Over a million developers have joined DZone.

Detecting potential bias in the wisdom of crowds

DZone's Guide to

Detecting potential bias in the wisdom of crowds

· ·
Free Resource

The wisdom of crowds is one of the more robust theories to have emerged in the social sciences in recent years.  The concept, which gained popularity with James Surowiecki’s book of the same name, has underpinned much of the growth in areas such as open innovation and crowdsourcing.

Surowiecki clearly stated the kind of conditions under which the wisdom of the crowds would exceed that of the wisest individual, one of which was that each person in the crowd has to act independently of the others.  Without this, groupthink can prevail and the crowd can actually be very stupid indeed.

A recent study conducted by a team of Spanish researchers aimed to tease out the role influence plays in crowd intelligence, and how one can mitigate against its impact.  The hypothesis tested by the researchers was that people have varying degrees of malleability, so some will be more likely to change their views as a result of extra information than others.

The idea is therefore that if you can figure out who those people are, and then separate them from the group, then the group can maintain its relative wisdom.  In other words, the researchers suggest, the wisdom of the crowd is reliant upon the crowd being confident enough in their beliefs to not seek external information.

To weed out these folks, the researchers used a previous study whereby people were given a range of tasks, such as estimating the length of the border between Italy and Switzerland.  After one task, some participants were given the aggregated answers of the other participants before continuing.  The end result was that this information influenced their second effort.

From this data, a mathematical model was born that charted how people tended to respond to new information.  With each individual giving a particular weighting to the extra information they received, they believe they could predict who the malleable folks are and how independent their thinking is.

“Our results show that, while a simple operation like the mean, median or geometric mean of a group may not allow groups to make good estimations, a more complex operation taking into account individuality in the social dynamics can lead to a better collective intelligence,” they say.

Of course, none of this really disproves the wisdom of crowds theory, as independence of information was always one of the criteria identified as key to success.  Nevertheless, the researchers next step is to try and test out their theory in a real world situation, which should be something to watch out for.

Original post

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}