The More, the Wiser?

On May 5, 1787, an old lady by the name of Korbmacher was accused of witchcraft in Philadelphia. She was set upon by a crowd in the Spring Garden neighborhood and received a good deal of abuse from the mob, which eventually cut open her forehead in an attempt to dispel her sorcery. She escaped with her life. But Philadelphians did not forget her supposed offenses. Three months later, with the city in the grip of a heat wave, they pounced again. On July 10, in the New Market section of the city, another mob set upon her. The Pennsylvania Evening Herald reported, “We are sorry to hear that the poor woman who suffered so much some time ago, under the imputation of being a witch, has again been attacked by an ignorant and inhuman mob. On Tuesday last she was carried through several of the
streets, and was hooted and pelted as she passed along. A gentleman who interfered in her favour was greatly insulted, while those who recited the innumerable instances of her art, were listened to with curiousity and attention.”

Eight days later, Mrs. Korbmacher was dead. It seems likely that she was not convinced of the wisdom of crowds.

“The wisdom of crowds” first came into vogue in 2004 as the title of a James Surowiecki book. Drawing on a half-century’s worth of sociological and economic research, Surowiecki argued that masses of people are often able to make better decisions than even very smart individuals. Consider, for example, a jar filled with jellybeans. If a large number of people guess how many jellybeans are in the jar, the average of their guesses will be closer to the correct number than all but a few individual answers. This experiment has been performed numerous times, with the crowd always performing exceptionally well.

Filled with such case studies, Surowiecki’s book was a sensation, and “the wisdom of crowds” became a clever phrase that, like “tipping point” or “black swan,” was bandied about as a catch-all explanation for all manner of phenomena. The theory was particularly well received by business types and Internet enthusiasts, who saw it as an endorsement of economic and technological futurism.

It’s easy to see why they were seduced. As Surowiecki describes it, the wisdom of crowds is an elegant theory. He posits that, under the right circumstances, groups can make remarkably intelligent judgments. In the TV game show Who Wants to Be a Millionaire? for instance, contestants are asked a series of multiple-choice questions. They are also given a set of lifelines to help them in case they get stumped. One of the lifelines allows the contestant to call a particularly smart friend to ask for help. Another allows the studio audience to vote en masse on the question. Statistics show that the very smart friends provide a correct answer 65 percent of the time, but the audience gets it right 91 percent of the time.

From these humble beginnings — jellybean counting and game show questions — Surowiecki extrapolated his theory further. Crowds, he argued, exhibit superior abilities not only at cognition but also at coordination and cooperation. Examples include finding optimal prices, creating efficient pedestrian traffic, and identifying both weak and strong businesses. In each of these, the crowd finds better solutions than any one central planner would be able to.

To Surowiecki’s credit, he understood that not all crowds behave wisely, so he established a number of caveats. In order to reach good decisions, he said, a crowd must be large, diverse, and its members must be independent of one another. He warned of the effects of information cascades, in which the decisions of a small part of the crowd skew the judgments of the larger mass. And he allowed that “groups are better at deciding between possible solutions to a problem than they are at coming up with them.”

But in the intervening years, Surowiecki’s cautions have faded as the wisdom of crowds became a cultural mantra. Conservatives embraced the theory because they saw it as an endorsement of the free market. Liberals adopted it because they saw it as an egalitarian broadside against elitism. And nearly everyone associated with the Internet glommed on because they understood that it was, in large part, an exaltation of the new medium that placed the World Wide Web near the center of an entire worldview.

Surowiecki’s acolytes range from the silly to the serious, but what they all have in common is an unbridled ardor for futurism. In Smart Mobs Howard Rheingold praised the way mobile technology enables the masses to revolt against corrupt establishments (such as the uprising against Filipino president Joseph Estrada in 2001). Chris Anderson, the formidable editor of Wired magazine, is so enamored of the wisdom of crowds that he took to the pages of Nature to argue that scientific journals should let their readers, not in-house “experts,” do the peer reviewing of articles.

In the computer world, the wisdom of crowds was gospel even before Surowiecki’s book. In 1991, a Finnish programmer named Linus Torvalds created an open-source operating system to replace Unix. He called it Linux and promptly released the source code, for free, to the entire world. Other programmers began making additions and fixes to the code and sending them to Torvalds, who decided what to incorporate and what to reject. The effort, known as “open sourcing,” harnessed the wisdom of the crowd, with thousands of programmers contributing bits and pieces of code to the system. From it emerged a nerd koan: “With 10,000 eyes, all bugs are shallow.”

Deference to the wisdom of crowds has found all manner of practical applications. John Fluevog Shoes runs a project called Open Source Footwear, which solicits ideas from customers, the best of which are put into production. California congressman Mike Honda used a service called crowdSPRING to let the crowd design his website. Last year Doritos harnessed the crowd by asking fans to make their own commercials, two of which it paid to air during the Super Bowl. The financial services firm Marketocracy has 85,000 users managing their own investment portfolios. It then aggregates the portfolios of the top 100 investors into a crowd-sourced super-index that consistently outperforms the S&P 500.

Yet for all the triumphalism, relying on crowds for wisdom presents some problems. For starters, history is overstuffed with instances of crowds making very bad decisions. Or, more to the point, making bad decisions precisely because people were acting as a crowd and not as mere individuals. The easy criticism of the wisdom of crowds is to simply point to the Beer Hall Putsch. Or the tech bubble. Or the housing bubble. Or the harassment of Mrs. Korbmacher.

Whether you judge the wisdom of crowds to be a net positive is largely a matter of taste. There are dueling examples to support each view. Yes, crowds sometimes intuit truth. After the Challenger explosion in 1986, for instance, the stock price of Morton Thiokol, which made the ship’s solid-fuel booster, dropped four times as much as the stocks of the other companies integral to the shuttle program. The crowd was able to finger the general source of the problem faster than the experts. But sometimes crowds get fooled. A few months before Enron declared bankruptcy, its stock price was still over $40 a share. Or take the University of Richmond. In 2005 it raised its tuition price by a gob-smacking 31 percent in a single leap. Instead of shunning this higher price for what was an identical good — after all, the school itself did not change radically in the span of three months — applicants increased the following year.

Superficially, whether you find the wisdom of crowds an unalloyed good or a substantially conditional one depends on whether you give three cheers or two for the free market. Those who believe that decisions rendered by the market are by definition correct will rarely find cause to doubt the wisdom of crowds. Those concerned with market failures will be more circumspect.

But on a deeper level, the wisdom of crowds presents a much trickier question. Wisdom, in the philosophical sense, often does exist in “crowds,” just not the kind of crowds you can see and touch. G. K. Chesterton argued that societies have a duty to consider the opinions not only of the living, but also of their ancestors:

If we attach great importance to the opinion of ordinary men in great unanimity when we are dealing with daily matters, there is no reason why we should disregard it when we are dealing with history or fable. Tradition may be defined as an extension of the franchise. Tradition means giving votes to the most obscure of all classes, our ancestors. It is the democracy of the dead. Tradition refuses to submit to the small and arrogant oligarchy of those who merely happen to be walking about. All democrats object to men being disqualified by the accident of birth; tradition objects to their being disqualified by the accident of death. Democracy tells us not to neglect a good man’s opinion, even if he is our groom; tradition asks us not to neglect a good man’s opinion, even if he is our father.

Tradition is the storehouse of nearly all the goods that create the infrastructure of a healthy society. Honesty, justice, respect, love, mercy, charity — without these values (and others), the modern liberal order would be impossible. These mores are formed, shaped, and preserved by a crowd that transcends time and space and gives particular weight to the past. Yet those who preach the wisdom of crowds are uninterested in the democracy of the dead, or mere tradition. Their eyes are ever fixed on the future.

Part of that future is a more level world, where the old-fashioned notion of elitism has been plowed under. Surowiecki argues that diversity is more important than either intelligence or expertise and that a crowd made up of diverse individuals — that is, smart people and not-so-smart people —will reach better decisions than a group consisting only of smarties. “Bringing new members into the organization,” he writes, “even if they’re less experienced and less capable, actually makes the group smarter.” This is because according to the wisdom of crowds, expertise is a mirage.

All of which is well and good, except that it isn’t true. A man who is gravely ill does not crowd-source his diagnosis. Architects building a bridge do not open up design specifications to the Internet for voting. You can dismiss the importance of experts only when dealing with relative trivialities like counting jellybeans, creating a new shoe, or building a website. Or stock picking. In 1998 a group of six students at Cornell’s business school did a research project on Enron. Relying only on publicly available information but using a series of complicated statistical tools, they concluded that the company was a house of cards, taking outsized risks and manipulating its earnings. We devalue expertise at our own peril.

More important, though, even if crowds can reach wise decisions, they don’t create. Genius and inspiration are the province of individuals. Thousands of computer programmers have been able to refine and improve Linux over the years, but the program itself would not have been written without Linus Torvalds.

At the exact same moment that Mrs. Korbmacher was being chased through the streets of Philadelphia, a group of fifty-five men convened in a hall on the east end of town. They were a decidedly un-diverse group, the very cream of the young Republic. While the crowd of Philadelphians was hunting down a witch, this small band of elite experts wrote our Constitution, which would shape the course of human history for centuries to come.

The wisdom of crowds only goes so far.