Can you make better predictions than a monkey?
Let’s say. You have the idea to let families goes on their holiday to France in a self-driving car. They arrive more relaxed and - more likely - their journey was more safe. An innovative idea, don’t you think?
In this rather fast changing market, you want to be flexible and move towards your consumer. There are plenty ideas and possibilities to innovate. But how do you know whether you make the right decision - without making it rain? Wouldn't it be fantastic if you could know in an early stage if your idea will be successful.
Most people predict the future no better than a monkey making
selfies. Is there any way to peek into the future and make accurate
The experienced marketer
Admit it. We all know that we can predict the future ourselves. Don’t we? An experienced marketer knows, better than any other person, what is happening in ‘his’ market. After years of experience we judge our ideas on the basis of our ‘gut feeling’. Yet, we can depend on this less than we think. It seems that knowledge and experience of the market in practice reach the opposite effect.
Psychologist and researcher Philip Tetlock describe in his book ‘Superforecasters’ how easy it is to mis-interpret information, or worse to ignore information at all.
Don’t take this personal, but according to Tetlock, experts are bad forecasters. Especially experts who have developed their own vision, find it hard to step aside. They hear all arguments that conform their own vision and ignore arguments that refute their vision.
Are you the one that came up with this idea, then there is no chance that you can predict the future. According to Tetlock, marketers are very interested in positive predictions. They want their idea to be successful and that makes them no longer objective.
Traditional market research?
Ouch. Allright. What about market researchers, can they forecast? Yes, and no. For the answer on this question, I have to be honest with you. Market research is carried out to provide marketers with objective information. Traditionally, this is done by presenting a concept to the target group and then asking them about their purchasing intention.
However, based on theories of behavioural economics, we know that people are bad at predicting their own behaviour. Respondents will try to answer as rationally as possible. (I always take a short break after two hours driving). But in real life their behaviour comes down to pure routine and emotion. (We are almost there, I can easily drive one hour extra.)
Kahneman calls this in his book ‘Thinking, fast and slow’, system 1 (subconscious, the easiest choice) versus system 2 (rationally weighing up the advantages and disadvantages, which takes effort) thinking. By far the majority (90%) of all decisions are made by system 1.
If experienced marketers and traditional research are no longer able to make accurate predictions, how can we then peek into the future to look if our idea is successful?
We take a closer look at the theory of Philip Tetlock (the scientist that offended you earlier). Tetlock has shown, after years of study, that forecasts are more accurate if you use superforecasters. Superforecasters are ‘normal people’ who happen to be very good at forecasting.
In one of Tetlocks studies, he gathered a group of over 1.000 volunteers. Every one of them make, on a individual level, predictions of the future. Tetlock analyses these predictions and looked whom did it the best, and add importance to these forecasts. He found out that people who make better predictions are mostly the people who describe themselves as doubters. Other qualities of superforecasters are self-criticism, curiosity, cautiousness and the ability to view issues from a greater distance
As innovators in market research we were very enthusiastic to bring this theory into practice. In collaboration with the Royal Dutch Touring Club (ANWB), Blauw Research set up an experiment based on Tetlock’s book about Superforecasting. We identified superforecasters within the member panel of ANWB. We selected the ‘right’ persons by letting them fill in self-assessments.
In these assessments we add a whole bunch of character traits including the ones from Tetlocks theory. Over 2.000 people, within (and outside) the member panel, complete the assessment and we let them make predictions about the success of ANWB’s innovations. Therefrom we pick the top 5 percent of the members of ANWB who scored the best on qualities of a superforecaster.
Soon, if we catch the future, we found out if they were right, but the first results are encouraging:
- The forecasts made by superforecasters seem to be more realistic. (Superforecasters make different (more conservative) forecasts than the ‘average’ ANWB member.)
- The difference between the two groups is larger in the case of more complex propositions. (Due to his ability to analyse the superforecaster seems better able to deal with complicated issues.)
Forecasting is never easy. But this theory might help you to get more insights into the future. For example, organisations can use superforecasters to build more realistic business cases. It can save you a lot of costs. And truly promising innovations - such as going on a holiday in a self-driving car maybe - can be developed more confidently with superforecasters.