<b>Sometimes the right test isn't "is B better" but "is B not meaningfully worse."</b>

<b>Sometimes the right test isn't "is B better" but "is B not meaningfully worse."</b>
When you simplify code, drop a vendor, or remove a feature, you want a non-inferiority test — proof the metric stays within a tolerance margin, not proof of a lift.
— Set the margin first (e.g. "no worse than −0.5%"). The CI must sit entirely above it.
— Standard significance tests answer the wrong question here and stall good cleanups.
Not every test is hunting for up.
Read the number, not the story. [non-inferiority margin −0.5%]