Demand Forecasting for Inventory: The Loss Is Not Symmetric

A demand forecast is not an end product. It is an input to an ordering decision, and that decision is asymmetric: running out costs the lost margin, ordering too much costs the markdown. When those two numbers differ, the optimal order is not the expected demand, and the gap between them is computable exactly. The mean is the wrong number to forecast Almost every forecasting tutorial minimises mean squared error, whose optimal prediction is the conditional mean. That is the right target when errors in both directions cost the same. In inventory they never do. Take a product with a margin of £12 a unit and a markdown loss of £4 on anything unsold. Every unit you fail to stock costs £12 of margin you could have had. Every unit you overstock costs £4. Being short is three times as expensive as being long, so a forecast that is right on average is a policy that is wrong most of the time. The correction is not a fudge factor bolted onto the forecast. It is a different quantity to forecast: a quantile of the demand distribution rather than its centre. Which quantile is not a matter of taste, and it is the same move as deriving a decision threshold from a cost matrix - the economics decide the operating point, and the model is only asked to produce the number the economics need. The critical fractile, derived Let Cu be the underage cost (per unit short) and Co the overage cost (per unit unsold). Consider whether to order one more unit when you have already decided to order Q. That extra unit sells only if demand exceeds Q, which happens with probability 1 - F(Q), where F is the demand CDF. expected gain from the extra unit = Cu * (1 - F(Q)) expected cost of the extra unit = Co * F(Q) Keep adding units while gain > cost. The optimum is where they are equal: Cu * (1 - F(Q)) = Co * F(Q) Cu = F(Q) * (Cu + Co) F(Q*) = Cu / (Cu + Co) smallest Q with F(Q) >= 0.75 -> Q* = 120 Now the cost of each policy. Total expected cost is Co times expected leftover plus Cu times expected shortfall. Cost(Q) = 4 * E[max(Q - D, 0)] + 12 * E[max(D - Q, 0)] Q = 100 (roughly the mean) leftover: 30(.15) + 10(.20) = 4.5 + 2.0 = 6.5 shortfall: 20(.25) + 40(.15) = 5.0 + 6.0 = 11.0 cost = 4(6.5) + 12(11.0) = 26 + 132 = 158 Q = 120 (the critical fractile) leftover: 50(.15) + 30(.20) + 20(.25) = 7.5 + 6 + 5 = 18.5 shortfall: 20(.15) = 3.0 cost = 4(18.5) + 12(3.0) = 74 + 36 = 110 Q = 140 (order for the worst case) leftover: 70(.15) + 50(.20) + 40(.25) + 20(.25) = 35.5 shortfall: 0 cost = 4(35.5) = 142 = 142 Q = 90 leftover: 20(.15) = 3.0 shortfall: 10(.25) + 30(.25) + 50(.15) = 17.5 cost = 4(3.0) + 12(17.5) = 12 + 210 = 222 Ordering the fractile costs £110 a period. Ordering the mean costs £158. That is £48 a period, 30% of the total cost, obtained by changing which number the forecast reports and nothing else - no better model, no more data, no additional features. Across a thousand SKUs and fifty-two periods it is a large number, and it is the single highest return available in most forecasting projects. The 30% figure is arithmetic on the invented distribution above, not a claim about anybody’s inventory. What generalises is the method: compute your own Cu and Co, take the fractile, and price the two policies against your own demand distribution. The gap grows as the two costs diverge and vanishes when they are equal. Training a model to predict a quantile To get the 75th percentile from a model rather than from a table, train it on the pinball (quantile) loss, which penalises under- and over-prediction at different rates by construction. pinball loss at quantile tau: L(y, f) = tau * (y - f) when y >= f (under-forecast) (tau - 1) * (y - f) when y < f (over-forecast) at tau = 0.75: under-forecast by 10 units: 0.75 * 10 = 7.5 over-forecast by 10 units: (0.75 - 1) * (-10) = 2.5 Under-forecasting is penalised three times as hard as over-forecasting, which is exactly the 12:4 cost ratio the fractile came from. That is not a coincidence: tau/(1-tau) = 0.75/0.25 = 3 = Cu/Co. The last line is worth keeping. The quantile you train at encodes your cost ratio, so setting τ and setting Cu/Co are the same act, and any disagreement between the two means someone has overridden the economics by hand. from sklearn.ensemble import GradientBoostingRegressor CU, CO = 12.0, 4.0 tau = CU / (CU + CO) # 0.75 model = GradientBoostingRegressor( loss="quantile", alpha=tau, n_estimators=500, learning_rate=0.05, max_depth=4, random_state=0, ).fit(X_train, y_train) order_qty = model.predict(X_future) # fit a second model at a low quantile to get a planning interval low = GradientBoostingRegressor( loss="quantile", alpha=0.10, n_estimators=500, learning_rate=0.05, max_depth=4, random_state=0, ).fit(X_train, y_train) Evaluate a quantile forecast with the pinball loss, not with MAE. A 75th-percentile forecast is supposed to sit above the actual three quarters of the time, so its MAE looks terrible and its bias looks like a defect. Scoring it on the loss it was trained for is the only comparison that means anything, and the coverage check - what fraction of actuals fell below the forecast - should come out near 0.75. That coverage check is the regression analogue of a reliability table, and it fails for the same reasons. Lead time, and why the horizon is not one period The newsvendor above assumes one order covering one period. Real replenishment has a lead time L: an order placed today arrives in L days, so the quantity has to cover demand over the whole lead-time-plus-review interval, and the relevant distribution is of cumulative demand over that window. If daily demand is independent with mean m and sd s, then over L days: mean of cumulative demand = L * m sd of cumulative demand = s * sqrt(L) L = 9 days, m = 20, s = 6: mean = 180 sd = 6 * 3 = 18 75th percentile (normal approx) = 180 + 0.674 * 18 = 192.1 safety stock = 192.1 - 180 = 12.1 units Two consequences fall out. Demand uncertainty grows with the square root of lead time while mean demand grows linearly, so halving lead time cuts required safety stock by about 29% (1 − 1/√2) - usually a better investment than a better forecast. And the independence assumption is doing real work: correlated daily demand makes the cumulative standard deviation grow faster than √L, so a promotion or a weather effect that runs for a week breaks the formula in the direction of understocking. Intermittent demand breaks everything above A large share of real SKUs sell zero units on most days. Slow-moving spares, long-tail catalogue items, industrial parts. For these the demand distribution is a spike at zero plus a small continuous piece, and the usual machinery misbehaves: the mean is a number that never occurs, MAPE is undefined on the zero days, and a quantile model trained at τ = 0.75 will happily predict zero because zero genuinely is the 75th percentile. - Model the two parts separately. One model for whether there is demand at all in a period, one for how much given that there is. This is Croston’s method in its classical form and a two-stage classifier plus regressor in a modern one. - Aggregate the time bucket until the zeros thin out. A SKU that is unforecastable daily is often perfectly forecastable monthly, and if your reorder cycle is monthly the daily forecast was never needed. - Score on the decision, not the forecast. For these items the only meaningful evaluation is simulated: replay the ordering policy over history and total the stockout and holding costs. Every point-forecast metric is misleading on a mostly-zero series. Forecasts that have to add up Forecasts are usually needed at several levels at once - SKU, category, region, total - and forecasts made independently at each level do not sum. The finance team’s number and the sum of the warehouse numbers disagree, and both are defended. The cheap fixes are top-down (forecast the total, split by historical shares) and bottom-up (forecast each SKU, add them). Bottom-up is usually more accurate at the leaves and noisier at the top; top-down is the reverse. Optimal reconciliation methods take an independent forecast at every level and project them onto the set of coherent forecasts, which typically beats both - the family is well developed in the forecasting literature under that name, and the practical requirement is only that you pick one policy and apply it everywhere rather than letting each team round its own way. Whatever you choose, evaluate it against the seasonal-naive baseline at every level. A reconciliation scheme that improves coherence and worsens accuracy at the level where the money is spent has made things worse in a way that is very difficult to notice. Top comments (0)