Forecasting Variability: Causes, Solutions, and Why It (doesn’t) Matter
Earlier in 2025, we started measuring forecasting variability (or stability) in our projects at SupChains. As we discussed the subject internally and with our clients, we focused on three main questions: how to measure stability, what is causing variability, and how important it is for supply chain planning. We conclude that humans might put too much emphasis and importance on forecasting stability, whereas its importance for inventory planning isn’t clear. This article presents our current views on the subject, which remains an active area of research.
This article is 100% human-written thanks to numerous reviewers. You can find their names in the acknowledgment section at the end of the article.
How to Measure Forecasting Stability
Forecasting Variability measures how forecasts (generated by the same model or process) change from one iteration to the next. Lower values indicate stable outputs from one planning cycle to the next. High values indicate that each new forecast differs substantially from the previous one.
At SupChains, to evaluate variability, we take the overlapping periods between two consecutive forecast sets and calculate the usual score metric (Score = MAE + |Bias|), treating one forecast as the actuals and the other as the prediction. Then scale the metric by the average of the two forecasts (using only the overlapping periods), resulting in a % (that can be higher than 100%). Mathematically,
where Fold/new denotes overlapping forecasts belonging to the old/new forecast sets, respectively.
You could also measure variability using any other forecasting metric you like — As long as it’s not MAPE or WMAPE. For example, this article by Pritularga and Kourentzes proposes another approach.
The fact that forecasts vary from one cycle to another is normal and expected. Conceptually, any new demand observation should result in a forecast update as you get new information. The only exception is if your forecast was 100% accurate.
Stability is simply one minus the variability.
This is a similar relationship to that between forecasting accuracy and error. You can use either one, as you prefer.
Note that the granularity level at which you measure stability also matters. Some forecasts might suffer from high variability at product x warehouse granularity (the one used to make supply and inventory decisions), but a much lower one at a higher aggregation level (the one mostly discussed in S&OP and IBP meetings).
Demand forecasts are pieces of information used by supply planning processes to make decisions. Forecasting variability (or variability in information) doesn’t always translate into variability in supply decisions. Decisions could, for example, be updated only if specific thresholds are met. For example, you would only cancel orders if the cumulative forecasts dropped by a consequential 50%. Furthermore, most supply chains make monthly (or weekly) orders for most of their products and raw materials. Resulting in perpetually renewed opportunities for adjustments.
What does Forecasting Variability look like?
To illustrate variability and stability, let’s take a real example from one of our recent case studies. One of our clients, Vantage, a US chemical manufacturer, regularly receives forecasts from its customers. We analyzed these customers’ forecasts and compared them with our own.
The table below illustrates how forecasts changed over time by displaying three forecast cycles (2024–06, 07, and 08) across the columns, with the first column containing the actual orders. By reading along a row, you can see how the Customer and SupChains forecasts changed over time.
You can read this table as follows: in 2024–06,
- the customer initially forecasted an order of 606 pieces for June (and 0 pieces for July).
- SupChains’ engine forecasted 38 pieces for June (and 87 for July).
Finally, the customer ordered 661 pieces in June (and 0 in July).
As you can see from the table, the customer tends to share order forecasts that can change drastically from month to month. On the other hand, our machine learning model delivers stable adjustments based on available data.
See that SupChains’ ML engine forecasts are consistently low for lag 1 because the model does not expect short-term orders when the customer has not yet provided a firm order. The order of 606 pieces in June 2024 is an exception: it wasn’t booked yet by the Customer on the 1st of June, so the ML model wasn’t aware of it when generating its 2024–06 cycle forecast.
If we can compute the variability of these forecasts, we would get the following results. You can replicate our results by downloading our Excel file here.
Insights about Forecasting Stability
We have been measuring the variability of different forecasting models and techniques across different projects. Let’s first look at the results we collected and discuss insights after (forecasts marked with a * were enriched using future confirmed orders).
From these projects, we observe that,
- 12-month moving averages (MA12) are naturally extremely stable.
- Our machine learning (ML) model delivers more stable forecasts than our statistical engine.
Most statistical engines (like the one we test for our clients) are often evaluation frameworks that evaluate various sub-models (such as exponential smoothing) for each product and select the best one. It means that, from one forecasting cycle to the next, the evaluation framework might choose a different sub-model to forecast a given product, which is likely to increase variability. And, even if you stick to the same sub-model, there is still variability from one cycle to the next. On the other hand, machine learning models (if properly trained and optimized) are inherently more stable because they are global models (i.e., trained on all products at once), considering various features to predict future demand.
- Enriching models with future confirmed orders (denoted by a * in the results above) increases variability.
This is especially true for statistical models that are enriched using simple rules (final forecast = max(stat forecast; confirmed orders). In our projects, enriching statistical models increased variability by more than 100%. The impact on the ML model is much lower: we always include confirmed orders in our ML models, and, as shown in the results, they always deliver more stable forecasts than statistical models — even without orders.
- In practice, accuracy doesn’t correlate with stability
In theory, you would expect that more accurate forecasts would be more stable forecasts: Ultimately, if your forecasts are 100% accurate, they should be 100% stable. In practice, we observe this relationship with moving averages, but not with our statistical engine or machine learning models. As shown in the table below, better accuracy (lower scores) doesn’t always result in more stability. In some cases, we could even observe a negative correlation: as accuracy increases, so does the variability. Overall, variability seems more a feature of a given model than of the underlying dataset.
Despite these results, variability should be considered as a byproduct of how much a model learned from the latest demand observations. If the observed demand lies within the forecast’s expected range, stability is expected. But when there is a discrepancy between the forecast and the actual demand (the forecast was wrong), the new forecast needs to adjust to account for this new information, introducing variability. Variability is a sign that the process is reacting to new information.
Finally, note that most (if not all) models are trained for accuracy, not stability. But you could, technically, optimize models for a combination of stability and accuracy. That’s what Van Belle et al. did in this paper.
How Variable are Humans?
As we investigated model variability in the previous Section, let’s now investigate how humans enrich these baselines.
The following table shows the Forecast Value Added (FVA, positive values denote accuracy improvement) obtained by our three clients, along with the increase in variability resulting from the human enrichments. We anonymized the results by showing only the FVA% (higher is better) of the consensus forecast relative to the baseline provided by SupChains, and the Variability increase as an extra % (lower is better).
Simply put, when enriching forecasts, two of our clients double the variability of the baseline. Humans seem to add more variability, on average, than ML-generated forecasts.
For this analysis, we measured variability per product x location (as per the previous table). Further analysis is required to compare stability at the global level: it may be that some planners do their best to stick to the same overall numbers despite changing forecasts for numerous products. This would result in high variability at the product x location level (the level needed for purchasing and inventory decisions) and high stability at the top level (the level most discussed in S&OP and IBP meetings).
Furthermore, variability, as we compute it, is only an average number that hides underlying variations. Forecasting models (ML and stat) update all their forecasts from one cycle to the next, resulting in numerous small adjustments. On the other hand, humans tend to focus on a few bigger adjustments while keeping many values untouched. So, even if we assume a similar overall variability, humans are likely to deliver fewer but bigger changes compared to baseline models.
Reducing Variability
Based on our analysis and experiences with our clients, we conclude that, to minimize variability, you should,
- Use ML models rather than statistical models (except for moving averages).
- Measure variability along with MAE and Bias in your Forecast Value Added (FVA) dashboard to identify which part of your process generates it. It is likely that humans will be mostly responsible for it. So, reducing variability might translate into reducing human inputs.
- As proposed by this article from Godahewa et al., you could ensemble historical forecasts with new ones to enforce stabilization. (You would have, for example, new forecast = 0.1 x old forecast + 0.9 x new forecast.) But this would come at the expense of extra post-processing that would be difficult to maintain as products and warehouses phase in or out.
But before acting on stability, we should ask ourselves, do we even need stable forecasts?
How Important is Forecasting Stability?
When comparing the added value of forecasting models or steps in the demand planning process, should we track and bother with stability?
Obviously, everything would be better with perfect forecasts that are 100% accurate and 100% stable. But such things do not exist. So, how important is forecasting stability?
Keep in mind that demand forecasting variability is not the same as variability in supply and purchasing decisions.
Is Forecasting Stability Useful for Inventory Planning?
From a pure inventory-planning perspective (which is our #1 priority: we mostly forecast demand to enable better inventory decisions), I doubt that stability would have a significant impact on policies. Accuracy does.
Most supply chains make monthly (or even weekly) orders for most of their products. It means that every month, supply chains get a new chance to order the right amount of goods based on the latest forecasts. When you have to place an order, you want the most accurate forecast possible over the relevant horizon. It doesn’t really matter how different the forecast was during the last order cycle — only the accuracy of the current forecast will impact the quality of your decision.
For example, let’s imagine that you make an order today. At the next order cycle, you realize that the forecast drastically changed. Again, it might be annoying for planners to cancel orders, but it’s best to cancel an order based on an updated forecast. In general, more recent forecasts are, on average, more accurate than older ones — especially short-term ones. And, if this forecast was updated, it means that new information has become available pointing towards lower future demand.
That said, it is not because the forecast changed that new decisions should follow.
Humans don’t like Variability
To put it simply, variable forecasts annoy humans: they cause endless changes during their S&OP meetings. No one wants noise and variability in their supply plans. On the other hand, refusing to cope with new updated numbers — for the sake of it — is like refusing to take one’s temperature. Forecasts should always be updated in the light of new information.
I am also afraid that humans see stability as a proxy for confidence. A forecast that doesn’t change (much) might appear to us as more correct, while variability could be interpreted as lower confidence or limited visibility into market dynamics. Humans also suffer from cognitive biases (conservatism bias, belief perseverance, preferences for consistency) that might explain why we tend to label variable forecasts as bad.
In practice, as new information becomes known, forecasts should change. Updating forecasts as new information emerges is essential to provide the best possible view of what lies ahead, driving the best possible supply and inventory decisions, allowing risks to be surfaced and mitigation options to be discussed during S&OP.
Would you blame the weather forecaster for updating their weather forecast in light of new information?
Seeing this new forecast, would you refuse to take an umbrella if you hadn’t initially planned to?
Or would you welcome the new forecast, thinking it might have saved you from getting wet?
When events change, I change my mind. What do you do?
Paul Samuelson[1]
[1] I have been wrongly attributing this quote to Keynes for years. Quote Investigator attributes it to Paul Samuelson, 1970 Nobel Prize winner in economics.
What Variability tells us about Forecasting Enrichments
Tracking variability in your FVA reviews might give you an extra edge to understand how planners react to information. High stability might mean that planners don’t trust the baseline enough (planners tend to stick to historical forecasts). At the same time, high variability might indicate that planners (or anyone enriching the forecasts) might overreact to the latest information or simply enrich too many products without supporting evidence.
Forecasting stability is still an active research area at SupChains and a topic of discussion with our clients. As we learn more about it (how to measure and increase stability and how important it is), we will continue to publish content on the subject.
Acknowledgments
Martin Edwards, Illia Babounikau, Justin Matthews, Charmaine Ng, Jacopo De Stefani, Martijn Mast, Jean-Michel Meyer, Jan Hesse, Kamini Garg, GP, Alexander Kononov, Mona Wappler, Jack Cox, Diogo Loureiro Afonso, Tianxiao Sun, Ivan Svetunkov
Citations
Godahewa, R., Bergmeir, C., Baz, Z.E., Zhu, C., Song, Z., García, S., and Benavides, D. (2025). On forecast stability. International Journal of Forecasting, 41(4), pp.1539–1558. Available at: https://doi.org/10.1016/j.ijforecast.2025.01.006
Van Belle, J., Crevits, R., and Verbeke, W. (2023). Improving forecast stability using deep learning. International Journal of Forecasting, 39(3), pp.1333–1350. Available at: https://doi.org/10.1016/j.ijforecast.2022.06.00
Pritularga, K., and Kourentzes, N, Forecast Congruence: A Quantity to Align Forecasts and Inventory Decisions (January 31, 2024). Management Science Working Paper №1., Available at SSRN: https://ssrn.com/abstract=4711817 or http://dx.doi.org/10.2139/ssrn.4711817
