False Wisdom of Crowds

False Wisdom of Crowds

 

If you had a choice of flying from Boston to San Diego in a plane piloted by a single machine or the combined intelligence of 3,000 people, which would you choose?

Perhaps you would want more information before making that decision.

If the machine piloting the aircraft was a well-designed piece of equipment that had been used as an autopilot for thousands of successful flights, and the 3,000 people were of average intelligence living in the Midwest, which would you choose?

What if the 3,000 people consisted of the combined intelligence of the best airline pilots in the aviation industry?

When working with large groups of people online, the wisdom of crowds is neither elevated to the smartest among them, nor is it diminished to the lowest levels. It hoovers somewhere in the middle.

But the cumulative influence of crowds, in today’s society, is very influential. And there is a pervasive notion that the crowd is always right. But what happens when it is wrong?

A History of Wrong-Headed Crowds

Leading up to 1929, it was a well-known fact that stocks were a great place to put your money.

In the 1950s, it became common knowledge that if a nuclear bomb went off in your city that you’d be safe if you simply learned to “duck and cover.”

In the 1970s, because of all the TV shows, virtually everyone knew about “quick sand” and how dangerous it was.

Up until 2007, it was a well-known fact that real estate was a great investment where you would virtually never lose money.

Today’s movies give most young people the idea that anyone can jump from a car going 30 mph, roll a few times, and be fine.

So much for the wisdom of crowds.

Drew Curtis, founder of Fark.com, has a very low regard for group intelligence.

"The 'wisdom of the crowds' is the most ridiculous statement I've heard in my life. Crowds are dumb," Curtis says. "It takes people to move crowds in the right direction, crowds by themselves just stand around and mutter."

Curtis points to his own experience moderating comments on Fark, where users present a rather humorous take on the news of the day. He says, “Only one percent of Web comments have any value and the rest are just garbage."

Leveraging the Thinking of Crowds

When Congress wants to study an issue, they don’t bring in average people to testify. They only want the experts.

On the other end of the spectrum, it really doesn’t work to poll people on their thoughts about winning lotto numbers to find the winning combination.

However, polling crowds can be a very good tactic for determining a product’s success if the crowd is closely aligned with that product’s target market.

While people are very good at knowing what they personally want, they are generally very bad at understanding the truths of the world around us.

A study last year by ETH Zürich found social influence is a huge factor that can greatly undermine the “wisdom of crowds.”

They found that influential people have a way of narrowing the diversity of opinions to the point where it undermines the wisdom of crowd effect in three specific ways:

  1. “Social influence” diminishes the range of opinions.
  2. This “range reduction effect” tends to polarize people’s opinions making group feedback less reliable in guiding decision-makers.
  3. Influencers tend to improve people’s confidence, but this so-called “confidence effect” will boost an individual’s confidence, while at the same time, decrease their accuracy.

Next Generation Super-Influencers

The Internet has made it far easier to gather information about people’s tendencies, inclinations, and opinions.

It has also made it far easier for super-influencers to polarize people’s judgement.

The whole democratic process in the U.S. today has really evolved into more of a battle of the super-influencers than the traditional gamesmanship by the candidates themselves. As a result we are seeing an increasingly polarized electorate.

Even though an elite group of super-influencers can be leveraged in the wrong way, marketing people are quick to get these same super-influencers to endorse their products and services in a more positive fashion.

Big Data’s New Wave of Decision-Making

In the past we have had the luxury of being able to take time making a decision.

Decision-making groups like planning boards, town councils, or some other overseer of a process, generally meet on a liesurely basis. Big decisions are often sliced into multiple smaller decisions and handled over longer periods of time. This time honored system for making decisions allows the general public to give their input and recommend changes along the way. In general, a fair way of doing business, but also a slow way of getting things done.

This slow determanistic model is being replaced with data mining techniques that can access the opinions of large constituencies or the buying whims of a target market almost instantly.

The speed of decision-making has become a competitive advantage in the business world and the latest form of bragging rights when it comes to communities. Yesterday’s taildraggingly slow processes are being replaced with an entirely new social norm.

According to a recent study by Capgemini, big data is becoming as fundamental to business as land, labor and capital.

“It’s not only through harnessing the many new sources of data that organisations can obtain competitive advantage. It’s the ability to quickly and efficiently analyze that data to optimize processes and decision making in real time, that adds the greatest value,” says Capgemini Global Sales Director Paul Nannetti. “In this way, genuinely data-driven companies are able to monitor customer behaviors and market conditions with greater certainty, and react with speed and effectiveness to differentiate from competition.”

Gems of real human intelligence will be captures as big data begins to comprehend larger trends and structures.

Capturing Real “Human” Intelligence

A few years ago I was involved in a search engine research project where we looked closely at the connecting strategy people used to analyse the relationship between a search phrase and the resulting website that they were eventually looking for.

Once users typed in a search phrase, we studied how the discernment process unfolded, with inappropriate sites being discarded before a final destination was chosen. Over time it became clear that the search path itself was layered with huge amounts of valuable data that should be captured and dissected for later use.

The information fragments that we were capturing were not merely data points along a line; we were capturing actual pieces of real human intelligence. Since real people were making the link between the search terms and the destination site, albeit a primitive association, it was indeed a useful nugget of human thinking.

Over time, artificial intelligence will give way to far more useful forms of real human intelligence.

Perhaps a better way to explain the capturing of real human intelligence is to explain my vision for the music player of the future. Since music is a very integral part of our lives, we can all relate to the power of listening to the right song at just the right time. But, for each of us, that perfect “right song” is always different.

So let’s imagine a music player that only played the “right songs”. One great song followed by another.

The ultimate music player will be capable of doing exactly this. It will assess our mood, our likes and dislikes, understand the contextual attributes of time, place, and people, decide whether we’re doing something that requires us to be physically active or just sitting comfortably in a chair, and it will anticipate our response to the music before it is ever played.

The ultimate music player will measure our heartbeat, brainwaves, biorhythms, stress levels, circadian rhythms, and a few other sensory inputs we haven’t even invented yet.

This future music player will only serve up songs that our body has a positive response to.

Technology like this will elevate our minds, and mental clarity, to a whole new level. It will be simultaneously energizing and relaxing, and give us motivation, endurance, and determination. In short, it will give us a reason to bound out of bed every morning to tackle a shining new day.

A device like this will require capturing real intelligence in real time.

As we move into the big data era, we will continue to uncover far more sophisticated ways of both capturing and leveraging these gems of real human intelligence, and finding unusual ways of using only those with the highest grade of intelligence behind them.

Final Thoughts

Today’s so-called “wisdom of crowds” will be replaced by three driving forces:

  1. Next Generation Super-Influencers
  2. Big Data’s New Wave Decision-Making
  3. Real Human Intelligence

While it will still be important to keep people engaged in a participatory fashion, the value of these three forces will quickly supercede the complexity of crowds.

When it comes to big data, IBM estimates that 2.5 quintillion bytes of data are created every day. Using this metric means that 90% of all the data in the world today has been created in the last two years.

Out of big data will come big strategy which will begin with effectively framing the problem.

A recent study by McKinsey and Company calculated an immediate shortage in the U.S. of 140,000-190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis.

The biggest gap is the lack of geek-savvy managers capable of making good decisions in this type of environment.

Over the coming months, the people-friendly “wisdom of crowds” is about to be railroaded by the oncoming big data train. To many it was a passing fad, to others, it never stood a chance.

By Futurist Thomas Frey

Author of “Communicating with the Future” – the book that changes everything

.

.

 

15 Responses

  1. Jon Jeckell

    I think you have crowdsourcing wrong here. It only works for specific classes of problems, and only with certain methodologies. Big data and crowdsourcing work different classes of problems and therefore are complimentary, not in competition. Both normally beat experts if the problem is framed correctly and used for the appropriate class of problem. Also, some recent research has thrown cold water on super influencers. People trust and are more influenced by people they know than celebrities, hence the whole viral marketing phenomenon.

  2. Nobody is smarter or dumber than everybody. Big data lets us consult with everybody before making our own decision.

  3. Jay Swartz

    Great article. I particularly like your ultimate music device. A sensor stream with that level of sophistication would be useful for far more than music selection.

    I think that while catchy, the term ‘big data’ does not convey the value of the concept. A term such as ‘dynamic synthesis’ would be more descriptive. We should bear in mind that analysis is the breaking down of data into details while synthesis is bringing it together. The ability to capture more detailed information more quickly and in greater volume is enabling. The real magic is the ability to convert the data streams into actionable information.

  4. Gary Mazz

    Good article !! Popular culture is never a substitute for critical thinking, or else we’d still be thinking the world is flat.

    And speaking of critical thinking… big data is nothing more than a “big farce”. There is no official definition for “big data”, so its another “do what you feel” buzz word. In many cases, its a bunch of old approaches to databases that were vetted by the market in the 1980s. If they failed in the past with small data sets, what makes anyone think they will magically work with 1000x the data ?

    As an industry, we are going to to need to take a critical look at knowledge interchange and interoperability to move forward.

    I like the music player concept. It an interesting problem. However, it may only work for a broad population. As we attempt to select the “perfect solution” for an individual, detection becomes susceptible to probabilistic models. When we approach a specific answer, we move from classical Euler-Lagrange to models similar to estimation models. This also applies to “big data”, where data and provenance is inconsistent across domain boundaries.

    Real human intelligence… agreed, is not far off, maybe in our lifetime.

    I think we will all be surprised to find the conscious we experience is in part the random selection of parallel thoughts modulated by personal experiences and situations. The brain activates several pathways to be more responsive to an unpredictable environment. Its a strategy to anticipate the resource needs of a potential situations.

    Before we can move forward with human intelligence, we will have to deeply reconsider how we think of data processing, integrate randomness with voting to assess prioritization and abandon determinism. We will need to accept data, an therefore outcomes, is not always consistent, correct and is influenced by others.

    We’ll also need to move away using data as the direct element for information, but that’s another story.

  5. Christina Venter

    Real human intelligence will always seek wise council from real people and the knowledge base they built from generation to generation. Only fools rely on their own understanding alone when it comes to rebuilding and advancing a new society. Thank you for this informative article. Choose those you allow to influence you with much care, wisdom and real intelligence. We know very little yet. The future(wisdom)to come can only be far beyond our expectations. Let us puch harder for a better world for all.

  6. Tom and Colleagues,
    Your statement and dialog miss a powerful option.
    Facilitated team brainstorming. Rather than get one expert opinion, put several experts (6-10) in a room and facilitate decisions.
    … Of course, it helps to have a suite of processes to guide that brainstorming. One for defining a business or product, one for developing strategy at multiple levels. One for culture, one for communications. One for early appraisal of technologies.
    … Visit InnovatorsEdge.com / ToolKits for more info.
    Best to all,
    Gary

  7. Arthur Collins

    The new paradigm that I now suggest and support is to become evermore products of our personal choices. instead of just being either some unforunat victim of consequence, much less the the product of our arrogant ignorance. Avoiding that typical elitist attitudes that condones first our complacency towards what others do or have done to them that then generates apathy then allows cruelty towards others. Wars are extremely wasteful and is never a good solution to anything. History tells us that tyranny always fails when the excessses of its leaders impoverish and needlessly endanger its suporters.
    We should look to something better that provides a reasonable return for our inputs of time, energy and resources. We can in fact have it all by stop being so wasteful and redirect our resources toward more profittable and meaningful ends.
    I now believe I can and should actually have anything I dsire but only as long as its honestly earned, positive, constructive and over all wholesome in nature. This also falls under my credo “to first do no harm.”

  8. Whether the wisdom of crowds is reliable depends upon the circumstances. It depends upon whether the problem to be solved is complex or complicated. Mechanical issues are generally issues of complication, and there’s no beating an expert’s ability for solving complicated problems. In an airplane at 30,000 feet, trust the pilot over the crowd. However, when the issues involve complex problems, diversity often trumps ability as we recently witnessed when a group of online gamers on Foldit solved in ten days a difficult puzzle in AIDS research that had alluded the world’s best scientists for ten years.

    The reliability of the wisdom of crowds also depends upon the presence of four critical conditions outlined by James Surowiecki in his book The Wisdom of Crowds: 1) diversity of opinion, 2) independent thinking, 3) local knowledge, and most importantly, 4) an aggregation mechanism. Without all four conditions, then the crowd can easily descend into an unintelligent mob or myopic groupthink. While not all crowds are wise, given a complex problem and the presence of the right conditions, there are many circumstances in which nobody really is smarter or faster than everybody.

  9. Let’s not forget that the ‘experts’ don’t have any monopoly on wisdom either: Rand Corporation in 1967 predicted experimental generation of fusion power by 1987, Alvin Toffler predicted in 1971 that by 2000 “society would be awash with free time”; in 1993 Lee Kuan Yew (Capo of Singapore) predicted unification of the Koreas by 2005. And it goes on: The Futurist Magazine’s “Best Predictions of 2001″ anticipate that by 2016 the world will be “freed from dependence on fossil fuels by nano-engineerd solar panels” and that by 2030 it will be possible to “record and replay our favourite dreams”. Possibly, but! Thus, we say in Australia, it will always be a case of “horses for courses”. The best process to use will depend on the question, the problem, the time frame, and the quality (or not) of your crowd.

  10. Gary Mazz

    There are significant distinctions between crowds, groups and individuals. Each has their own merits and excel at different tasks. For example, You wouldn’t use a wrecking ball for etching and engraving jewelry.

    Innovation rarely occurs in a collective environment. Collective dynamics often weeds out variation leaving homogeneity in its wake. The “I” (individuality)is moderated to mediocrity.

    Collectives (crowds) has very different role in society than providing vision and leadership. You rarely see innovation and progress from collective governments, and when it does happen, its an act of desperation.

    There are many models that can be used to explain these behaviors. Mainly, in crowds we don’t have the capability to process all the information. We aggregate and correlate to reduce the information needed to process, promoting the common ideas to precedence. The one idea, the “game changer” never gets heard. It’s lost in the noise of the crowd.

    The crowd is also much slower to respond than an individual. I’ve never seen a committee of 50 come to a conclusion quicker than a group of 3. How long would it take a crowd of 50,000 to reach consensus if each person had 3 questions ?

    No matter what type of collective or it’s size, vision starts with the individual. Others may add to it, foster and evolve it into something unrecognizable. Irrespective of the path it takes, a vision that can change the course of humanity still starts with one person.

  11. It’s an old truism — often applied nowadays to viewers of a certain cable news network — that gathering a lot of idiots in one place doesn’t improve the situation. The challenge of interactive crowdsourcing is to prove that truism wrong. It’s important that we do so, since we are the idiots whose situation needs improving. Yes, the rise of Big Data technologies will help highly-resourced decision-makers learn how to ask and answer various questions about customers, voters, system components, and so on. The challenge for a modern society is whether the people at large can to leverage these kinds of technologies for public purposes… democratic priority setting, for example. In technical terms, we need to learn how to establish and interrogate our own key performance indicators as collaborative decision-makers.

  12. Teppo Nieminen

    Rod Collins makes very good points. “Wisdom of crowds” is a relatively specific concept that is applied way too broadly with very little thought nowadays. That is pobably also why the title of the post includes those words. (The same misuse applies to another related favorite “crowdsourcing”, that covers such a wide range of practices that an umbrella term is pretty worthless).

    For example, a stock market can and will often start feeding on itself: bids and asks are not determined by the perceived worth of the underlying stock, but by participants’ perception of what other people will think the stock is worth. In other words, individual judgments that form the wisdom of crowds cease to be independent. Most prediction markets (HSX, IEM) rarely suffer from these rationality breakdowns because of different incentive structure.

    Super-influencers have always been with us. While certain people’s opinions and choices are now broadcast instantaneously, we also have a much wider variety of influencers to choose from.

    Wisdom of crowds is also not a substitute for a participatory decision-making process. If I continue with the town planning example, in most municipal decision-making there is no “right” answer that could be gleaned from either crowds or any amount of data. There are various conflicting interests that have to be either reconciled, accommodated or ignored, and some people will invariably be disappointed with the outcome. How disappointed, is to an extent determined by how satisfied they are with the process itself.

    The post sort of mixes predictions and skills. Piloting a plane is a skill, and thus hard to transfer into a crowd concept. In a simulator, a crowd can indeed “fly” a plane, but a pilot is hard to beat. And the more unstable the system and the shorter the cycle time, the bigger the advantage a pilot has. In simulated aerial combat a crowd is toast. (On the other hand, planes are already mostly flown by computers as witnessed by the Air France tragedy. When pilots have to take over, they don’t know how.)

    At the same time, Philip Tetlock’s research proves that experts are pretty bad at forecasting, and the more famous the expert the less accurate the forecast.

    The most intriguing point is about Big Data. I am certain that it will indeed change many aspects of our lives one way or another. But the algorithms still rely on past data, and try to find the best solution to fit on that data. The brief history of financial risk modeling is a good example.

    I am looking forward to a music player that senses when I could be in a mood for something completely new that I haven’t heard before. But I’m not holding my breath.

  13. Teppo Nieminen

    BTW, here’s a story that’s very pertinent to the big data issue raised by Thomas:
    http://bit.ly/PoC5TU

  14. “False Wisdom of Crowds,” Thomas Frey’s recent blog post, comes to a useful conclusion – that analytical skills are becoming increasingly important to take advantage of big data – but some of his comments regarding the “wisdom of crowds” are incomplete.

    For example, he says, “When working with large groups of people online, the wisdom of crowds is neither elevated to the smartest among them, nor is it diminished to the lowest levels. It hoovers somewhere in the middle.”

    In fact, this is true for groups of all sizes, as evidenced by a demonstration I often use. Here is a specific, recent result.
    http://sschuman.blogspot.com/2012/08/false-wisdom-of-crowds.html

  15. Does anybody know of a book that is entirely about the science behind the fact that the crowd is always wrong?

    A former friends brother wrote such a book and I’m trying to find it.

Leave a comment