Q: My on-call person missed an alert and we were down for an hour. How do I prevent that?

Q: My on-call person missed an alert and we were down for an hour. How do I prevent that?

A: You need an escalation policy, not just a louder first alert. The pattern: if the primary on-call doesn't acknowledge within N minutes, the alert automatically escalates to a second person, then a third, then a whole-team channel. Nobody being awake should never equal nobody being paged.

Concrete shape:
— Page primary immediately on confirmed down.
— No ack in 5 minutes, page secondary.
— No ack in 10 minutes, page the lead and post to a team channel.

And use a real paging channel (phone call or push that overrides silent mode), not email or a Slack message that sleeps with the phone.

Follow-up worth fixing now: test the escalation chain quarterly with a fake incident. Most teams discover a wrong phone number or a muted integration only during the real outage.

Got a question? Drop it in the comments.