08 December 2008

About The Wisdom of Crowds

In his book “The Wisdom of Crowds – Why the many are smarter than the few”, James Surowiecki makes - indirectly but nonetheless powerfully - a very good case for Knowledge Management or the leverage of individual and collective knowledge.

Simply put this way, that the many are smarter than the few is hardly a contentious statement. After all, a croud of say 1000 individuals should be smarter than only 500 of this same croud most of the times. You have more minds available to solve a problem/find an answer. However, what Surowiecki means is that a croud of 1000 can be – with the right conditions – much smarter than the sum of its parts even when it acts/decides in a completely uncoordinated way (meaning each individual acts/decides in isolation from the others). In fact, such a group can be (and Surowiecki gives plenty of examples) smarter than the even best experts in a particular field!

The three conditions for this group wisdom to materialise according to Surowiecki, are that it must be diverse, independent and decentralized.

On diversity, Surowiecki writes (chapter 2, part III):
<<The fact that cognitive diversity matters does not mean that if you assemble a group of diverse but thoroughly uninformed people, their collective wisdom will be smarter than an expert’s. But if you can assemble a diverse group of people who possess varying degrees of knowledge and insight, you’re better off entrusting it with major decisions rather than leaving them in the hands of one or two people, no matter how smart those people are.>>This can be hard to believe but Surowiecki then makes the case for this point very well and I cannot find any reason to disagree with him.

On independence, he writes (chapter 3, part I):
<< First, [independence] keeps the mistakes that people make from becoming correlated.[..] One of the quickest way to make people’s judgments systematically biased is to make them dependent on each other for information. Second independent individuals are more likely to have new information rather than the same old data everyone is already familiar with. The smartest groups , then, are made up of people with diverse perspectives who are able to stay independent of each other. >>
I would think that this condition is in theory much less contentious than the first one on diversity. However, the problem with true independence is that in practice, it is rather difficult to obtain. Often, decisions in a croud are made sequentially with each individual influenced by his/her predecessors.Therefore, Surowiecki advises that <<If you want to improve an organization’s or an economy’s decision making, one of the best things you can do is make sure, as much as possible, that decisions are made simultaneously (or close to it) rather than one after the other.>>

On decentraization, he writes (chapter 4, part II):
<< [..] if you set a croud of self-interested, independent people to work in a decentralized way on the same problem, instead of trying to direct their efforts from the top down, their collective solution is likely to be better than any other solution you can come up with. [..] Decentralization’s great strength is that it encourages independence and specialization on the one hand while still allowing people to coordinate their activities and solve difficult problems on the other.>>
However, Surowiecki then cautions that : << decentralization’s great weakness is that there’s no guarantee that valuable information which is uncovered in one part of the system will find its way through the rest of the system.>> He then asserts that for a crowd of any kinds to allow << individuals to specialize and to acquire local knowledge [..] while also being able to aggregate that local knowledge and private information into a collective whole, [..] [it] needs to find the right balance between the two imperatives: making individual knowledge globally and collectively useful (as we know it can be), while still allowing it to remain resolutely specific and local. >>

Well, well, isn’t this where/when Knowledge Management should come in? In fact, for all intent and purposes, this is a definition of KM I am satisfied to work with in an organizational setting: any intentional and managed changes or activities with a conscious objective to facilitate/enable what is highlighted in blue above. But it then highlights a fundamental reason for organizational KM to have so often failed to deliver: the lack of management recognition that collective knowledge in practice is indeed always valuable, with the potential to be very often correct and effective. Leveraging knowledge is then not just about realizing (and doing something about it) that each employee’s knowledge is valuable (and that’s already hard enough for most senior managements) but that the collective knowledge of the whole or groups of employees is even more valuable.

I think that a cultural shift is needed here for this realisation to become the norm rather than the exception. This shift has already started with the ubiquitous nature and global reach of the World Wide Web enabling huge crowds to influence decisions directly or indirectly (eg. Obama’s election). This shift now needs to enter the board rooms en masse. According to Malcolm Gladwell, “the tipping point” (see his book with this title) should be reached when between 10 and 15% of board rooms will have formally acknowledged the value and power of individual and collective knowledge. I can safely predict this will happen even if I cannot say when.


At 11:39 AM, Blogger marnix said...

What if a newspaper suggests there might be a problem with the solvability of a bank.

The expert says : 'Don't panic, your money is save'.

The individuals in the crowd all want to save their money and start a run on the bank.
... individually correct decisions can lead to a crisis

At 12:14 PM, Blogger Peter-Anthony Glick said...

Thanks for the comment.

I did not say that crowds are always right!
However, in your scenario, why would you assume the experts to be right necessarily?
And if it is the right individual decision then couldn't it be the right collective one too? If it means for the bank to collapse well better have all our money out before it does later anyway.

At 12:26 PM, Blogger marnix said...

I am just looking for a few examples of not so wise crowds. The reactions to the financial crisis and similar panic reactions are typical.
It is not this simple, I know, crowds are never simple ;-)


At 2:40 PM, Anonymous sergiostorch said...

Hi Marnix
Let me suggest you just one fresh example of collective stupidity: take the crowds at both Israel and Palestine, who´ve been unable to perceive systemically the real danger both are running into in the mid-term, and keep electing radical leaders who conduct them to disaster (the syndrom of March of Folly, described by Historian Barbara Tuchman).

Lack of information? No. Just a very complex problem. Behind a simple decision (on the ballot) the appropriate social cognition process is far beyond what the tenets of the Wisdom of Crowds suggests.

Now a very common situation that each of us has seen in our daily lives: a team made up by PhD´s who behave collectively as an IQ 50 level. Why? leadership, egos, decision making processes, time management, trust etc.

At 10:47 AM, Anonymous Jon said...

A good debate.

But I have to say that for me ... the internet to full of people trying to tell each other what is right and wrong.

Sure there are examples of where a crowd can do good and bad things.

There are also examples of where individuals do great and bad things.

So how about just sharing something that worked ... without adding the telling everyone you have the right answer for all things?

At 10:51 AM, Blogger jon thorne said...

This comment has been removed by a blog administrator.


Post a Comment

Links to this post:

Create a Link

<< Home