Better Forecasting with Historical Data and Judgment

A recent article on forecasting presents historical data and judgment as an either-or choice. We disagree. In our view, the art of forecasting requires both and the understanding of how much weight to place on each depending on the circumstances.

The article outlined research conducted by Dr. Matthias Seifert and his team at The IE Business School in Spain. They evaluated the value of historical data vs. judgement in forecasts for volatile sectors such as fashion and entertainment. These fast moving markets provide a compressed, microcosm to study how demand unfolds at a slower pace in other markets – similar to how geneticists study fruit-flies to understand the transmission of genes in humans. Their data indicate that in volatile markets, historical data provide limited value for predicting the success of new products. For example, forecasts based on a musical artist’s last album do a poor job of predicting the success of their next endeavor. The forecasts become more accurate when based on predictor variables such as the amount of marketing behind the first single and the other artists releasing music at the same time.

This may seem obvious, but that is due to the simplicity of the example. The dynamic exists in more complex markets and situations, it’s just harder to see. The business literature is strewn with examples of companies that based their forecasts and strategies on the status quo rather than look at variables that predicted a change in demand and paid the price (Kodak, Blackberry and Microsoft to name a few). The obvious ones to consider include the current competitive set, existing products, and marketing activity. Innovators and disruptors such as Steve Jobs and Elon Musk take the process a step further and evaluate factors such as products and competitors that may enter the market and what people are trying to accomplish rather than the specific products they use. Seeing the broader picture comes naturally for people like Jobs and Musk.  For the rest of us, the book Winning the Long Game: How Strategic Leaders Shape the Future by Steve Krupp and Paul J. H. Schoemaker offers some pointers and practices for looking beyond the status quo.

One of the primary challenges with identifying predictor variables is that they aren’t always evident. As a researcher Dr. Seifert had the luxury of comparing past actions and circumstances with outcomes. The second challenge is that it is easy to get carried away in identifying tenuous predictor variables and then assigning them more power than they likely have in order to create a plausible scenario that fits the outcome we are seeking. Using a reference-class comparisons and historical data provide a disciplined way to reign in this tendency. It involves systematically evaluating historical trends and what other organizations have experienced and asking the question: Why do we expect things to be different. This exercise helps put your hypotheses and variables in perspective and clarifies the degree to which they are likely to make the future different than the past.

At Isurus we include predictor variables and historical data when designing forecasting surveys. Predictor variables include brand awareness, information sources, price, etc. Pain points, desires and motivations fall into this category as well. It’s cliché, but people don’t want to buy a drill – they want a hole. Historical data and reference class comparisons aren’t perfect predictors of the future – but in most cases they are within a standard deviation or two from it: If consumers or businesses have always purchased on price, they are likely to continue to do so in the future.

We think the big take away is that the best forecasts use both historical data and judgement. To ensure you systematically include both, use these three questions when creating your next forecast:

• What would history predict the future would be?
• What factors got the market to its current state? (Level of marketing, competitors, technology, features and functionality, etc.)
• What might be changing or might we proactively change that would make the future different from the past? (New delivery model, combination of features, etc.)