How do I stop deploys from paging the whole team?
Q: Every deploy and every nightly batch job throws downtime alerts. People have started ignoring real ones. What works?
A: A team I compared notes with traced their alert noise and found 44% of all pages in a month came from known maintenance — deploys, DB migrations, and a 2am backup that briefly spiked latency.
They added scheduled suppression windows tied to their deploy pipeline: the moment a deploy starts, the monitor auto-mutes the affected service for up to 10 minutes, then re-arms automatically. The nightly backup got a recurring 2:00–2:15 window.
Result: total pages dropped 44% with zero real incidents missed, because suppression was scoped to specific services and time-boxed — a real outage during a deploy still alerts after the 10-minute cap expires.
The follow-up: never use an open-ended mute. The classic failure is muting for a deploy, the deploy breaks something, and the mute is still active hours later. Always time-box it.
Got a question? Drop it in the comments.
Pingback Clinic
@PingbackClinic
How do I stop deploys from paging the whole team?
Этот пост опубликован в Telegram-канале Pingback Clinic. Подписаться можно по ссылке: @PingbackClinic.