Are juries consistent? (Yes, but only when they deliberate.)

What is it about?

Group decisions are ubiquitous: whether you're being tried by a jury, serving on a school board, or following the policies set by a governing council, society trusts groups with its most challenging problems. In this study, we examine the question of group consistency: that is, if the same group faced the same decision twice, would it produce the same outcome? Or are group outcomes specific to the social dynamics of the interaction --- making them arbitrary decision-makers, and no better than asking a single individual to decide? To answer this question, we conducted hundreds of online jury deliberations. In these deliberations, participants were asked to decide a question sourced from Reddit. They decided twice with the same group, and twice alone. When users worked in groups, we changed each user's pseudonym on the platform as they appeared to teammates, concealing the fact that they worked with the same team twice. This enabled us to directly compare the consistency of a group with the consistency of individuals. To our surprise, groups and individuals were equivalently consistent --- a result that is "good for democracy." Interestingly, however, we found that, when we aggregated the votes of people who did not deliberate, their decisions were virtually random. Thus, a key takeaway of this study is that group decisions are consistent, but only when they deliberate.

Featured Image

Photo by Tingey Injury Law Firm on Unsplash

Photo by Tingey Injury Law Firm on Unsplash

Why is it important?

Our research provides empirical evidence and key design implications for the critical problem of whether, and how, to use democracy to address a vast array of challenging problems. Recently, social media platforms have become interested in using group decision-making to address content moderation issues (e.g., the Facebook Oversight Board); policymakers have also become interested in crowdsourced policies via tools like "deliberative polling." This work establishes that platforms and leaders can proceed with confidence in implementing group decisions, but cautions that group decisions become volatile without deliberation.