How wrong was the forecast? Measuring it honestly

The previous post in this series argued for writing forecasts down. This one is about the follow-through: scoring them. Because an unscored forecast is not a forecast, it is a mood — and the scoring is where all the improvement lives.

Two numbers cover a small business

Average miss (MAPE, if you like acronyms). For each scored forecast, take the miss as a percentage of what actually happened, ignore the direction, average them. "Our weekly product forecasts miss by about 18%" is a real fact about your business, and it sets how much safety stock and caution every downstream decision needs. Expect the number to be humbling. Everyone's is.

Bias — the one that actually changes decisions. Same misses, but *keep* the direction, then average. A forecast that misses 18% randomly in both directions is honest noise; you buffer for it. A forecast that misses 12% consistently *high* is not noise — it is systematic optimism, and it quietly over-orders every single week. Bias is the difference between "the world is uncertain" (true, live with it) and "we always guess high" (true, fixable this week by adjusting the guess).

Split both numbers by category and by who made the call, and the pattern sharpens: optimism usually lives somewhere specific — new products, or one buyer's patch, or everything seasonal.

The trap in judging single forecasts

One forecast cannot be judged by its outcome. The 80%-confident call that goes wrong was not necessarily a bad call — 80% means wrong one time in five. Judgement only shows in the batch: across everything you called 80%, did about 80% happen? That question — calibration — is the honest scoreboard for a forecaster, human or machine.

It is also the scoreboard we hold Kauzio to. Every prediction it makes is sealed before the outcome, scored after, and the running accuracy — including the misses — is shown to you, not summarised into flattery. We think any tool that forecasts for your business owes you its scorecard. Ask yours for its bias number. If it cannot answer, you have learned something important about it.

Score the forecast. Fix the bias. Buffer the noise. In that order.