Why Does My Forecast Look Like That?

Here at SkuBrain, customers often ask us, “Why does my forecast look like this?” The answer is (almost) always, “Because of your data”. SkuBrain uses historical time series data to predict future performance. Although it runs through lots of statistical algorithms to choose the best one, and does all the heavy lifting to reconcile forecasts at your chosen hierarchies, ultimately the reason why a forecast looks the way it does is because that’s what the data says.

However, in some cases the future is not captured in the historic data, and management action can often explain these disconnects between historical data that builds a forecast, and the performance of that forecast going forward. So first we will look at common examples of where forecasts fail, and why, and then turn to the strength of SkuBrain to get the forecast right under conditions where the future can be predicted, at scale, customized to the SKU level, on your sales and inventory data. The value of this forecasting is improved when accurate versus inaccurate results are segmented, reviewed by management, and new effectiveness trends are identified.

As examples of where management review and forecast accuracy segmentation can add value, here are some cases of where SkuBrain forecasts are inaccurate, with explanations and strong hypotheses as to why.

DISCLAIMER: Some of the data shown in the following examples have been time-shifted into the future to demonstrate the effects under discussion. That’s why you’ll see some “actuals” which are in 2017.

Forecast 1

Here is a case where a SkuBrain forecast was inaccurate, and the visualization shows why. With almost no sales in the two years preceding the forecast period (Oct 2016), but very strong sales in the three years prior, SkuBrain predicts modest performance. Instead, performance goes off the charts. Clearly SkuBrain was trying to strike a happy medium, and something else in the business process – or market demand that was not apparent for two years – must have changed for the sales to both increase and show repetitive three-month cycles.

Forecast 2

This forecast shows activity of an unpredictable drop in product demand. There appear to be two distinct patterns in the training data: a peak every three months that is trending upward forcefully; and the “normal” sales that are trending upward, but more moderately. SkuBrain’s best guess is a happy median, and as more data are acquired over time the stable cyclical trend in the data will be reflected in the forecast. But early on, twelve months of training data is not enough to tell one way or another, so SkuBrain picks a flat average forecast. This turns out to be really off, since sales for the product actually tank. Rather than ask “why is the forecast off?” the real question is why did the product tank, and whether that is due to issues that management can control.

Forecast 3

Nothing in the historical sales suggests that the recent, sustained, six-month upsurge of demand starting in July 2016 was going to continue, or stop. Management would hopefully know. With the data it does have, SkuBrain predicts a decline in near-term sales. (SkuBrain has produced a base forecast – the green shaded area – and a final forecast, indicated by the fine green line – to lean more about forecast hierarchies, click here.)

Forecast 4

With a single data point of sales, no algorithm in the world will give you an accurate forecast. In fact, SkuBrain’s internal algorithm would more than likely consider this to be a data error, or a one-off event, and so predict (correctly, in our view) no future sales. Management would need to identify which products should be excluded from forecasting so that the general results of the forecasting process are not biased negative because product segments that no forecast will predict accurately are included in the evaluation of the forecasting method.

Now that we’ve reviewed these examples, let’s look at some examples from the 90-95% of SkuBrain cases that show acceptable accuracy

This product has five years of data, with a lot of activity in the first year, followed by a slump, and stabilizing somewhat in the next three years (’14 to ‘16). As such, Skubrain bases its 12-month forecast on the more recent performance, and gets it right for the month of October 2016.

A Classic forecast

There is some seasonality (a dip every January) but no strong trend upwards or downwards. With three years of data, SkuBrain has produced a classic forecast – predicting September sales to within 1.39%. Few forecasts are this good!

Seasonal forecast

Very seasonal behaviour – perhaps, if you were a brewery, this would show behaviour by a summer beer (or a winter ale, if you happen to be in the southern hemisphere). The peaks are not obviously trending up or down, but July, August and September show reliable demand. October 2015’s forecast was highly accurate and we expect the forecast to be highly accurate in the future, because the underlying data trends are exactly what SkuBrain works best with.

Base vs. Final

Here is another seasonal product, but the sales trend is more nuanced. Again, SkuBrain has made two forecasts here – the Base Forecast at the SKU level (green shaded area) predicts a slight dip, but the Final Forecast retains the previous years’ performance, due to strong performance at the category level (not shown). To understand the difference between the base and the final forecast, have a look here.

Cyclicality

Here, the data shows some cyclicality (every six to seven months), as well as seasonality (a clear peak around March or February). However, the surge in Feb ‘16 is not repeated historically, so SkuBrain errs on the conservative side. Even so, the October 2016 forecast is within 3%, and over time the cyclical nature of the data will be reflected in the forward-looking forecast.

Phased out forecast

This product’s best days are behind it. Despite good years in ’14 and ’15, sales have clearly diminished in ’16, and SkuBrain picks that eventually sales will subside to zero, perhaps punctuated with a half-hearted resurgence or two before petering out. Naturally, management may know this product is already phased out and, if so, then no future forecast is needed and these results can be set aside.

Adjusting forecasts

This shows another one-off product that has seen better days. Of course, SkuBrain could be wrong, and in fact the product peak in February could indicate a seasonal product that requires forecasting adjustments. But nothing suggests that is the case in the (less than one year of) data at hand. In cases like this, the business planner will need to adjust the forecast before creating a plan – something that is easy to do with SkuBrain.

Forecasting with accuracy

This product has decent sales data stretching back three years, but with no recent sales in the last 12 months SkuBrain predicts no further sales in the next 12 months, and the results for October 2016 concur. Note that just as we might segment out difficult-to-predict products in order to assess general forecasting accuracy, one can argue that cases such as this should also be excluded, because product discontinuance is the proximal predictor of no future sales, and attributing accuracy to a forecast in that case will tend to inflate an estimate of future forecasting accuracy.

More data, better results

This product has five years of data, with a lot of activity in the first year, followed by a slump, and stabilizing somewhat in the next three years (’14 to ‘16). As such, Skubrain bases its 12-month forecast on the more recent performance, and gets it right for the month of October 2016.

Some cases (SKUs) will always result in prediction errors. Management action and expert knowledge often explains these cases. The insight is that management should actively segment the products into what is amenable to prediction, and apply expert knowledge and data visualization to improve that segmentation over time. Setting aside those difficult to predict cases, SkuBrain is very accurate on 90-95% of forecasts where data are stable. The algorithm tournament fits the best forecast given the data, and as the data mature the forecast will become more accurate.

Assessing an algorithm’s forecasting accuracy should exclude segments that are either impossible to predict (preferably pre-identified segments based on management input during the data warehouse build) or very easy to predict based on a management decision to discontinue a product last year and where SkuBrain will accurately predict zero sales in the future. What we are saying is that SkuBrain is an aid to improved forecasting, at massive scale, and management review, expert knowledge, and learning from the SkuBrain results over time will improve the value of forecasting.

SkuBrain is free to use for 7 days. Sign up here and start forecasting!