Groups Can Be Smarter Than Individuals—If Structured Right

In 1906, the British statistician Francis Galton attended a livestock fair in Plymouth. He was there to observe a contest: guess the weight of an ox after it had been slaughtered and dressed. About 800 people submitted estimates—farmers, butchers, and random fairgoers with no expertise whatsoever.

Galton expected the crowd to be hopelessly wrong. He was an elitist who believed that the masses were too ignorant to make good judgments. But when he analyzed the data, he found something that challenged his worldview.

The median estimate was 1,207 pounds. The actual weight was 1,198 pounds. The crowd was off by less than 1%.

No individual came that close. The crowd, collectively, outperformed every expert.

Galton had stumbled onto something profound—and it would take another century to understand why it worked.

The Paradox of Collective Intelligence

Here's the puzzle: individuals are limited. We have bounded rationality—we can't process all relevant information, we take cognitive shortcuts, we're riddled with biases. Any single person's judgment is unreliable.

And yet, under the right conditions, aggregating those unreliable individual judgments produces something remarkably accurate. The errors cancel out. The signal emerges from the noise.

This is collective intelligence: the capacity of groups to perform cognitive tasks better than individuals.

It's not about finding the smartest person in the room. It's about structuring the room so that the group as a whole becomes smarter than any individual could be.

The phenomenon appears everywhere once you start looking:

Markets. Prices aggregate dispersed information about supply, demand, and expectations. No central planner knows what wheat should cost—but the market converges on a price that reflects countless individual assessments.

Science. No single researcher understands everything. But the scientific community, through peer review and replication, gradually converges on reliable knowledge. The process is messy, but it works.

Democracy. Individual voters are often uninformed and irrational. Yet democracies, on average, outperform autocracies on most measures of human welfare. Somehow the aggregation produces better governance than any individual ruler could provide.

Prediction markets. When people bet real money on future events, their collective predictions often outperform expert forecasts. The Iowa Electronic Markets have predicted election outcomes more accurately than polls.

Something is going on here. Groups can be smarter than individuals. But they're not always smarter. Sometimes groups are catastrophically stupid—groupthink, mob mentality, market bubbles. The same species that produced Wikipedia also produced the Salem witch trials.

The question isn't whether groups can be intelligent. It's when. Under what conditions does collective intelligence emerge, and when does it fail?

The Conditions for Collective Intelligence

James Surowiecki, in his 2004 book The Wisdom of Crowds, identified four conditions that enable collective intelligence:

Diversity of opinion. Each person should have some private information or interpretation—even if it's just an eccentric perspective. Homogeneous groups don't aggregate well; they just amplify shared biases.

Independence. People's opinions shouldn't be determined by others. The moment everyone starts copying the perceived expert, diversity collapses and the crowd becomes a herd.

Decentralization. People should be able to specialize and draw on local knowledge. Centralized systems lose the distributed information that makes aggregation valuable.

Aggregation. There must be some mechanism for turning individual judgments into a collective answer. Voting, averaging, markets—the mechanism matters.

When these conditions hold, crowds are wise. When they fail, crowds are mobs.

Galton's ox-weighing contest worked because all four conditions were met. The guessers were diverse (farmers, butchers, random visitors). They made independent estimates (no coordination, no copying). They drew on decentralized knowledge (some knew cattle, some knew weights, some just guessed). And there was a clear aggregation mechanism (the median).

The stock market often works the same way—until it doesn't. During bubbles, independence collapses. Everyone watches what everyone else is doing. Diversity disappears as contrarians get driven out. The aggregation mechanism (prices) starts reflecting collective delusion rather than collective wisdom.

Why Diversity Is the Key

Of Surowiecki's four conditions, diversity is perhaps the most important—and the most counterintuitive.

We tend to think that the best decisions come from the best people. If you want to predict an election, ask political scientists. If you want to value a company, ask financial analysts. Find the experts and listen to them.

But expertise often comes with blind spots. Experts share training, assumptions, and biases. A room full of political scientists might miss what a cab driver notices. A room full of financial analysts might miss the obvious flaw that an outsider would catch.

Diversity isn't valuable because ignorant people are smart. It's valuable because different people are wrong in different ways.

If everyone's errors are random and uncorrelated, they cancel out when you average. The signal—the truth—emerges from the noise. But if everyone's errors are correlated (because they share the same biases, the same information, the same training), averaging doesn't help. You just get the same error, more confidently.

This is why the best forecasting teams include people with different backgrounds, different expertise, and different ways of thinking. It's not about finding the single best forecaster. It's about assembling a diverse portfolio of perspectives that cover each other's blind spots.

Scott Page, a complexity scientist at the University of Michigan, has formalized this as the "diversity prediction theorem." In simplified form: Collective accuracy = Average individual accuracy + Diversity of predictions.

The more diverse the predictions, the more the errors cancel. Diversity isn't just nice to have—it's mathematically essential.

The Independence Problem

Independence is the condition that fails most often—and when it fails, collective intelligence collapses.

Humans are social animals. We watch each other. We copy each other. When we're uncertain, we look to others for guidance. This is usually adaptive—learning from others is efficient. But in the context of collective intelligence, it's disastrous.

Information cascades occur when people ignore their private information and follow what others appear to believe. If the first few people in a sequence make the same choice—even by chance—later observers might rationally conclude that those people must know something. They abandon their own judgment and follow. The cascade can lock in on the wrong answer, even when most individuals' private information pointed to the truth.

This is how bubbles form. Early investors buy an asset. Others see the buying and infer that the early investors must know something. They buy too. The price rises. Rising prices attract more buyers. Soon everyone is buying because everyone else is buying—and the actual value of the asset becomes irrelevant.

Independence is why secret ballots exist. If you know how your neighbors voted, you might change your vote to match. The aggregation loses information. Secret voting preserves independence—each person expresses their own judgment without social pressure.

Independence is also why good forecasting tournaments prohibit collaboration during prediction phases. If forecasters discuss their predictions before submitting, they'll converge. Diversity collapses. The collective becomes no wiser than a single individual.

The Aggregation Mechanism Matters

How you combine individual judgments determines what kind of collective intelligence you get.

Averaging works well when you're trying to estimate a quantity (like the weight of an ox). Errors tend to cancel, and you get a good central estimate. But averaging works poorly when there's no single right answer—when you need to select among options rather than estimate a value.

Voting works well for binary choices, especially when the majority is likely to be right. But voting can suppress minority information. If 51% of people think A and 49% think B, voting picks A—even if the B-voters have better information.

Markets work well when you can bet on outcomes. Prices aggregate not just opinions but confidence—people who are more certain bet more. Markets also update continuously as new information arrives. But markets require liquidity and can be manipulated by large players.

Deliberation allows people to share information and update their views. Good deliberation can improve on simple aggregation by letting people learn from each other. But deliberation can also destroy independence—people anchor on early speakers, defer to high-status participants, and converge prematurely.

The best collective intelligence systems often combine multiple mechanisms. Prediction markets aggregate initial estimates; deliberation refines them; voting selects final answers. No single mechanism works for everything.

When Collective Intelligence Fails

Collective intelligence fails in predictable ways:

Groupthink. When cohesive groups value unanimity over accuracy, dissent is suppressed. Everyone agrees on a course of action that no individual would have chosen alone. The Bay of Pigs invasion is the classic case—Kennedy's advisors convinced themselves of a plan that was obviously flawed in retrospect.

Herding. When people copy each other rather than thinking independently, the crowd amplifies rather than corrects errors. Financial bubbles are herding events—everyone buys because everyone else is buying.

Polarization. When groups of like-minded people deliberate, they often become more extreme. Conservative groups become more conservative; liberal groups become more liberal. The diversity that enables collective intelligence is replaced by ideological uniformity.

Motivated reasoning. When people have stakes in particular outcomes, their judgments become biased. Crowds of motivated reasoners don't converge on truth—they converge on whatever they want to believe.

Each failure mode involves the breakdown of one or more of Surowiecki's conditions. Groupthink kills diversity and independence. Herding kills independence. Polarization kills diversity. Motivated reasoning corrupts the signal that aggregation is supposed to extract.

The Engineering Challenge

The insight of collective intelligence research is that group cognition is designable. You can structure environments to make groups smarter or dumber.

To increase diversity: Include people with different backgrounds, training, and perspectives. Resist the temptation to only include "qualified" experts. Ensure that demographic diversity maps onto cognitive diversity.

To preserve independence: Collect judgments before discussion. Use secret ballots. Prevent people from observing others' choices before making their own. Structure deliberation so that all views get aired before convergence.

To enable decentralization: Let people specialize. Don't force everyone to know everything. Aggregate specialized knowledge rather than homogenizing it.

To improve aggregation: Choose mechanisms appropriate to the task. Use markets for continuous probability estimates. Use voting for discrete choices. Use deliberation carefully, structuring it to share information without destroying independence.

These aren't just academic considerations. Organizations design collective intelligence systems all the time—meetings, committees, prediction markets, voting systems, review processes. Most do it badly, defaulting to structures that suppress diversity and independence.

The organizations that do it well—and they exist—consistently outperform their competitors on complex cognitive tasks.

The Takeaway

Groups can be smarter than individuals—but only under specific conditions. Diversity, independence, decentralization, and proper aggregation are the keys.

When these conditions hold, crowds outperform experts. When they fail, crowds become mobs.

The question isn't whether to trust groups or individuals. It's how to structure groups so that their collective intelligence emerges. This is an engineering problem, not a philosophical one. We're just beginning to learn how to solve it.

Galton's ox-weighers got lucky—they stumbled into conditions that enabled collective intelligence. The challenge now is to create those conditions deliberately, at scale, for problems that actually matter.

Further Reading

- Surowiecki, J. (2004). The Wisdom of Crowds. Doubleday. - Page, S. E. (2007). The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton University Press. - Sunstein, C. R. (2006). Infotopia: How Many Minds Produce Knowledge. Oxford University Press.

This is Part 1 of the Collective Intelligence series. Next: "Wisdom of Crowds: Surowiecki's Conditions"