The extraordinary good sense of people acting in crowds is the subject of a recently published volume, called "one of the most important books of the last year" by Will Hutton in Sunday's Observer. -- The theme of James Surowiecki's The Wisdom of Crowds (Doubleday, May 2004; Anchor paperback, August 2005) is that crowds show extraordinarily good collective judgement. -- But that does not mean you should follow crowds, because there's a condition: crowds are "wise" only "as long as we each make our individual decision as far as possible independently and are not too much influenced by wanting to follow experts or second-guessing others. It is when we follow the crowd that it turns into an irrational mob, creating stock market bubbles or lynching the innocent," Hutton writes. "But when crowd decisions emerge of our own aggregated free will, they are astonishingly accurate and, when values are involved, decent." -- Also, interestingly, "To be wise . . . the crowd's judgment has to include everyone's -- the expert, the stupid, the allegedly commonsensical, the wild, the analytic, the hunch. It's by comprehending the universe of possible outcomes in all their diversity and then averaging them that the wisdom emerges." -- As Hutton points out, there are important lessons for government officials to learn from The Wisdom of Crowds....
THE CROWD KNOWS BEST
By Will Hutton
** From cricket to fuel prices, our collective instinct invariably strikes the right note **
September 18, 2005
On Tuesday lunchtime, the English cricket team, triumphant winner of the Ashes, paraded in an open-top, double-decker bus from St Paul's to Trafalgar Square. Beforehand, I must confess I had my doubts. Trafalgar Square celebrations are for great national occasions, I had thought. Winning a Test series against Australia, although profoundly satisfying, was not on the same scale as winning the World Cup in football or rugby, or even the Olympic bid. It risked, I thought, descending into unjustified jingoism and triumphalism, undermining cricket's tradition of sportsmanship and respect for opponents.
I needn't have worried. Trafalgar Square was much less full and the event shorter than it had been to celebrate England's rugby success. Unlike then, the traffic flowed freely and pedestrians used the wide southern pavement to go about their business as if not much was happening. Although there was pride, it was much more gentle and less hubristic than I expected. In fact, the crowd calibrated the scale and tone of the response perfectly. The celebration turned out to be the right thing to do in the right way.
The crowd had been just as intelligent at the Oval. It knew exactly when the match was won and England's score had become too great for Australia to chase in the time available, when it was safe to stop mocking and joshing the Australian fielders and start congratulating worthy adversaries. Hours before victory had been formally sealed, the magnificent Shane Warne stopped being told he was the man who had dropped the Ashes (Kevin Pietersen would have been out had Warne held the catch, and England's eventual total would probably have been eminently beatable) and was told by thousands of voices rather that they wished he was English. The crowd had decided to become collectively generous, reflecting an aggregated instinct to do the right thing at exactly the right time that it largely managed throughout.
This latter ability of a crowd to home in on a crucial detail -- the point at which England was safe -- is grist to the mill of James Surowiecki, author of the subtle and intriguing The Wisdom of Crowds, one of the most important books of the last year. In his view, one of humanity's greatest assets is our unrecognized ability to get collective decisions right in crowds as long as we each make our individual decision as far as possible independently and are not too much influenced by wanting to follow experts or second-guessing others.
It is when we follow the crowd that it turns into an irrational mob, creating stock market bubbles or lynching the innocent. But when crowd decisions emerge of our own aggregated free will, they are astonishingly accurate and, when values are involved, decent.
Surowiecki starts his book with an exercise in ox-weighing mounted by English scientist Francis Galton in a west of England animal fair in the early 1900s: 787 breeders, sightseers, farmers, and laborers were asked to guess the weight of an ox after it had been slaughtered and dressed, with most of the respondents largely ignorant of anything to do with cattle. No individual got the weight right, but the average guess of the crowd was 1,197lbs. The actual weight turned out, to Galton's amazement, to be 1,198 lbs. The crowd, like that at the Oval and in Trafalgar Square, had shown an extraordinary collective judgment.
The Wisdom of Crowds is studded with similar examples and experiments, from audience advice to contestants on "Who Wants to be a Millionaire?" to asking students to guess the temperature of their classroom to finding a way through a maze. In every case, the crowd's decision turns out not only to be right, but consistently better than any individual's. Surowiecki cites the example an American naval commander used to find a sunken submarine lost at sea. After aggregating a mass of guesses of where it might be and then averaging them, he went on to search the location that the crowd thought the submarine would be. It was right to within some 200 yards and miles better than any individual guess.
To be wise, though, the crowd's judgment has to include everyone's -- the expert, the stupid, the allegedly commonsensical, the wild, the analytic, the hunch. It's by comprehending the universe of possible outcomes in all their diversity and then averaging them that the wisdom emerges.
Trouble starts, as Surowiecki readily concedes, when individuals start to adjust their first thoughts and reactions with what they think others are thinking, or when judgments are too little influenced by the diverse views of others. Stock market bubbles, classic exercises in collective irrationality, happen because, as Keynes famously argued, the art of successful stock market investment is to second-guess what the mass of other investors think, so causing prices irrationally to inflate.
And most governmental cock-ups, like Blair deluding himself, parliament, and the country that Saddam Hussein had weapons of mass destruction, are because a few people holding the same views dominate the decision-making process. One of the lessons of *The Wisdom of Crowds* is that privately generated government 'intelligence' by agencies like the CIA or MI6 has a bias to make mistakes because of the narrowness and lack of diversity of the views that they build upon to make their judgments. To assess the time and place of the next terrorist attack, the Met and the Home Office would be best advised to harness the wisdom of aggregated views, thinks Surowiecki, rather than rely on so-called intelligence.
An independent, diverse, and inquiring press is also fundamental to collective wisdom. Last week, the crowd nearly made a collective mistake -- panic-buying petrol, so generating a self-fulfilling shortage, because of the threatened blockade of petrol refineries. In fact, the blockade was always a bluff without anything like the necessary support. This was something that could have been quickly discovered had the (mainly right-of-center) newspapers that splashed with the story cared to do some elementary journalism. But that would have spoiled what proved an imaginary story. In fact, the crowd was wiser than the journalism, collectively guessing at the lack of the support, a view quickly validated by a diversity of sources ranging from the BBC to the internet. Petrol mainly ran out (and then only temporarily) in those towns where right-wing newspaper readership dominated, but not in those with more diverse newspaper sales and readerships.
Surowiecki's wisdom of crowds is thus very contingent. Get closer and the wisdom only manifests itself when there are lots of diverse opinions, plenty of independent sources of information, and masses of sturdy individuals reaching their conclusions unaffected by emotionally following the opinion of others. Trying to create the circumstances in which those conditions systematically hold is nigh impossible. But it's nice to know that if and when they do, the average of our aggregated judgments turns out to be right so often. There's no better case for pluralism, diversity, and democracy, along with a genuinely independent press. We have to continue aiming for such a world, even if it never arrives.