How to set Forecasting Accuracy Targets

How should supply chain leaders set forecasting accuracy targets? In this article, I first review three techniques to set targets (industry benchmarks, demand variability, and statistical benchmarks). Then, I discuss why the forecast value-added mindset is a more robust and fairer approach than absolute accuracy targets — Actually, I don’t advise using accuracy targets. Finally, we assess how much added value we can expect from a forecasting engine and process.

Death and the Devil Surprising Two Women, Daniel Hopfer, ca. 1515

How accurate should your demand planning team’s forecasts be? This critical question is likely to keep supply chain managers pondering. As I’ve explored in a prior article, even a modest 10% increase in forecast accuracy can yield substantial business benefits. Accurate forecasts matter.

Supply chain managers want to set forecasting accuracy (and bias) targets for two main reasons,

• To Drive Team Performance. Setting clear, achievable targets helps to motivate and guide planners toward continuous improvement. (Nevertheless, pay attention that, as explained in another article, setting solid incentives to reach accuracy targets might be counterproductive.)
• To Gauge Achievable Accuracy. Understanding the realistically attainable forecast quality allows for more effective improvement-project selection and planning.

However, the challenge lies in setting these targets appropriately. Many Sales and Operations Planning (S&OP) managers default to using historical data or industry benchmarks to set these targets. As we’ll discuss in the following sections, these methods are not only outdated but can also be misleading. Indeed, sales predictability can vary significantly across products, channels, and geographical regions, making a one-size-fits-all approach ineffective.

Acknowledgments
Anupam Aishwarya, Karl-Eric Devaux, Christoph Koch, Jeff Carruthers, Leonardo Bozzo

Why Assessing Forecastability is Difficult

Forecastability[1] varies depending on the product, market, and sales channel. Several factors influence predictability:

• Market Conditions. As market conditions change, so does forecastability. Forecasting sales in 2020 posed unique challenges for everyone. Similarly, consider the complexity of forecasting sales when a main competitor is about to launch a new product line backed by a heavy marketing campaign.
• Product Life Cycle. Forecasting new product introductions is challenging and will leave most statistical models clueless.
  For a deeper dive into this, see this comprehensive guide.
• Business Drivers. Promotions, price changes, and shortages make sales patterns more volatile (hindering expected forecasting accuracy).
  If you want to learn more about these factors, I’ve published multiple case studies (case study 1, case study 2) with similar challenges.
• Intermittency. Low-volume items are challenging to forecast, particularly if they display demand intermittency.
  For more insights on how to forecast intermittent products, see this article.

Now that we understand why predictability differs for each product and changes over time, let’s review techniques to assess it.

[1] “Forecastability” is a term I introduce to describe the ease with which an individual SKU can be forecasted. A high forecastability indicates a greater likelihood of achieving accurate forecasting results. Essentially, it’s synonymous with predictability in the context of forecasting.

Demand Forecasting Best Practices

Poor and Good Ways to Assess Forecastability

If predictability varies by SKU and is subject to change, how can you set realistic accuracy targets?

Many S&OP managers default to using industry benchmarks (sold by various external organizations) or tracking the demand variability (or demand coefficient of variation, COV). However, these methods have significant drawbacks. Let’s delve into why.

• Industry Benchmarks. These are not reliable for several reasons:

1. Industry benchmarks often rely on inconsistent or flawed Key Performance Indicators (KPIs) — like MAPE.

2. Different companies operate in different markets and have unique product and distribution strategies, making direct comparisons unsuitable.

3. These benchmarks often measure accuracy at different levels of data aggregation or horizons.

• Demand Variability (COV). This is another misleading metric. COV is not a forecasting technique and fails to account for trends and seasonality. Unfortunately, many software vendors and consultants still use and recommend this obsolete and unsuitable indicator.

For more information, you can read this article or my book Demand Forecasting Best Practices for a detailed debunking of demand variability and industry benchmarks. Moreover, note that using COV to segment products in ABC XYZ categories is also not advisable. For more on this, check out my best practices video.

Instead, a more effective and straightforward method exists: using moving averages. As discussed in the next session, this approach allows you to understand the inherent demand variability and provides a more reliable basis for setting forecast targets.

Adopt Forecast Value Added (FVA)

Having established how to assess the forecastability of each SKU (by using moving averages as benchmarks), the next step is to shift our focus from forecasting accuracy targets to forecast value added (FVA) targets. Let’s break down the difference between the two approaches with a straightforward example:

• Forecasting Accuracy Target. If you set a 70% forecasting accuracy goal for all your markets, some planners may easily reach this target (thanks to stable market conditions). However, others might struggle because they must forecast demand in volatile environments. In short, flat accuracy targets are unfair.
• Forecast Value Added Target. Instead, consider asking your teams to produce 10% more accurate forecasts than the forecast baseline (generated by your forecasting engine). This approach levels the playing field, making the target more universally applicable. (Still, some markets will have an easier time adding value than others — we will discuss this further in the next Section.)

The key is transitioning from merely evaluating forecast accuracy at the end of your demand planning cycle to monitoring FVA by measuring the value each stage of the forecasting process adds to the overall accuracy. In my training courses, I always emphasize that implementing FVA should be the top priority for all S&OP leaders. For a deeper understanding of FVA, read my comprehensive guide here or watch our dedicated webinar with theory and case studies.

If you’re using forecasting tools like statistical models or machine learning, it’s crucial to measure not just the value your team adds but also the extra accuracy your model provides compared to statistical benchmarks (moving averages). I’ve spoken with numerous supply chain managers who, after following my advice, discovered their forecasting tools weren’t outperforming simple moving averages. For them, it was time to change.

Manage Your Teams with Added Value Targets

It’s clear that the focus should be on added value rather than just raw accuracy. This approach allows you to set more realistic, fair, and achievable targets for your teams. Fairness is a critical element in setting proper objectives.

What level of added value should you aim for? Here are some general benchmarks:

• Models. Based on our experience, advanced forecasting models can often outperform moving averages by a margin of 20% to 40%, depending on various factors. For more insights, you can read my article here. Let’s review quickly the main elements that will impact how much added value you can expect from your forecast engine.

1. Machine Learning. Properly implemented machine learning models can reduce forecast error by 10% to 20% compared to moving averages — even without any extra information.
2. Seasonality. When dealing with seasonal patterns, expect a 10% error reduction. Moving averages typically fall short of capturing seasonality, so your models should offer a noticeable advantage.
3. Promotions. If your business relies heavily on promotions and you incorporate them into your forecasting engine, anticipate an additional 5% to 10% boost.
4. Pricing. Price changes are similar to promotions but are generally less impactful. Pricing adjustments can contribute to a 1% to 5% decrease in forecast error.
5. Shortages. Removing historical shortages from your dataset could see another 1% to 10% improvement. Read this article for a more detailed overview of this technique.

• Planners. Based on our experience, skilled planners who adhere to best practices can usually reduce forecast errors by 5% to 15%. Your expectations for planners’ added value should be based on different factors:

1. How elaborate your forecasting engine is. It might be easy to enrich a forecast made by a simplistic model, but you will struggle to add value over a machine learning model using independent variables, such as pricing, promotions, marketing, and recent shortages.
2. Local business environment. In some markets, thanks to local information and demand dynamics, your planners might have an easier time adding value. For example, they might get information from your clients — making it easier for them to enrich the baseline forecast. On the other side, tender-driven markets might be more difficult to forecast.

It’s important to note that these are general guidelines. Actual results may vary based on specific circumstances. However, my experience across multiple projects indicates that these benchmarks generally hold true. Nevertheless, reaching these performance levels won’t happen overnight; it requires a commitment to ongoing, incremental improvements.

Subscribe here: https://mailchi.mp/supchains.com/newsletter

Key Takeaways on Setting Forecast Targets

1. Forecastability is Complex. Forecastability (or predictability) varies by SKU, market conditions, and other business drivers like promotions and pricing.
2. Industry Benchmarks and Demand Variability are Poor Indicators. Relying on industry benchmarks or the demand coefficient of variation (COV) is inadequate for setting realistic targets.
3. Focus on Added Value. For a more robust and fair approach, transition from setting forecasting accuracy targets to forecast value added (FVA) targets. Track the added value of each step in your process and measure it compared to statistical benchmarks such as moving averages.
4. Benchmarks for Added Value. Your forecast engine should deliver a 10–30% FVA compared to moving averages. Your planners should be able to enrich the forecast further and reduce the error by 5 to 15%.

Do you want to learn more about best practices for demand planning?

I wrote my latest book to help supply chain leaders improve their demand forecasting process.
You can download an extract using this link:

Demand Forecasting Best Practices