A bid shading model was overcorrecting, costing 9% of winnable impressions

A bid shading model was overcorrecting, costing 9% of winnable impressions

Bid shading (the DSP lowering your first-price bid toward the expected clearing price so you do not overpay) is supposed to save money. One buyer found it was quietly losing them volume.

1. They ran an A/B holdout: 80% of traffic used the shading algorithm, 20% bid the full unshaded value.
2. They compared win rate and effective CPM across the two arms on matched inventory.
— Shaded arm: 18% win rate, $2.41 eCPM.
— Unshaded arm: 27% win rate, $2.88 eCPM.
3. They reconstructed the shading curve and found it was shading 31% off the bid on average, far past the actual clearing price.

The model had been trained on a period of soft demand. As competition returned, the expected-clearing estimate lagged the real market, so shaded bids landed below winning thresholds. The 9-point win-rate gap was pure overcorrection: impressions they valued and could have won at a profit, surrendered for a 47-cent saving that wasn't worth the lost conversions.

Why it matters: shading is a forecast, and forecasts decay. A periodic unshaded holdout is the only way to see what the model is leaving on the table when market conditions move underneath it.