Alert Volume Is the Metric You Can Show Leadership. Actionability Is the One That Matters.

TL;DR Most engineering teams measure success by the reduction of total alert volume, yet few track whether the surviving alerts actually change a responder's behavior. By adopting the "actionability" metrics used in emergency medicine, SRE leaders can identify when alert filtering has successfully reduced noise versus when it has merely concentrated complex, undiagnosed problems for the on-call engineer.

The number you can show leadership and the number that matters

Engineering leaders often celebrate when a dedicated "alert hygiene" project results in a 50% drop in total page volume. It is a clean, quantifiable win to present to a VP of Engineering. However, a recent account from an SRE in India highlights a sobering reality: after eight months of successful alert reduction, the incidents that survived were not easier to handle—they were significantly harder.

When you aggressively filter for actionable alerts, you remove the trivial "low-hanging fruit." What remains are the edge cases where a correlation engine might hand a sleep-deprived engineer three plausible root causes simultaneously. As one commenter on the thread noted, filtering does not remove judgment; it concentrates it. The "win" shown to leadership—fewer interruptions—masks a new, unmeasured debt: the increased cognitive load required to perform hypothesis testing in the middle of the night.

Emergency medicine ran the study

Software operations is not the first field to struggle with alert fatigue. In June 2020, Fleischman et al. published a study in the American Journal of Emergency Medicine that should be mandatory reading for every SRE lead.

Researchers placed a physician observer in an urban academic emergency department (ED) for 53 hours. They recorded every monitor alarm, the staff’s response, and—crucially—whether the alarm changed clinical management. The results were staggering. Across 146 patients, they recorded 1,049 alarms (a median of 18 alarms per hour). Of those 1,049 alarms, only 8 resulted in a change in clinical management. That is an actionability rate of just 0.8 percent (95% CI 0.3 to 1.3).

Unlike software industry surveys that rely on self-reported feelings of burnout, this study used direct observation to measure a binary outcome: Did this signal change what the human did?

Why 0.8 percent is not the shocking part

The truly alarming finding was that staff did not observably respond to 63 percent of the alarms at all. This is not a failure of individual diligence; it is a rational adaptation to a system that cries wolf 99.2% of the time.

According to the Agency for Healthcare Research and Quality (AHRQ), The Joint Commission estimates that between 85 and 99 percent of clinical alarms are non-actionable. This has life-or-death consequences. The Joint Commission’s sentinel event database recorded 80 deaths and 13 permanent injuries associated with alarm events between 2009 and 2012. In medicine, alert fatigue is not a productivity drain—it is a mortality metric. Because the stakes are so high, the field was forced to stop measuring "noise" and start measuring "change in action." Software operations, which often shares the same "always-on" criticality, has yet to adopt this rigor.

Key Takeaway: The only metric that validates an alert's existence is whether it changed the responder's subsequent action.

The measurement, translated

To move beyond volume-based reporting, engineering leaders can implement an "Actionability Audit" over a two-week window. This should be treated as a blameless quality improvement project, not a performance review.

The Actionability Runbook:

1. Real-time capture: For every page, the responder records four data points immediately (post-incident recall is notoriously unreliable):
   • Trigger: What did the page say?
   • First Look: What was the first dashboard, log, or command checked?
   • Action: What was the first corrective action taken?
   • Divergence: Did this alert change what you were going to do, or did you already know about the issue through other channels?
2. Compute the Action Rate: Divide the number of "action-changing" alerts by the total number of alerts.
3. Measure Time to First Action: Track the delta between the page firing and the first diagnostic command.

If responders feel this is a test of their efficiency, they will inflate the numbers. It must be clear that a low actionability rate is a failure of the monitoring system, not the engineer.

What medicine did next

The Fleischman study didn't just recommend "fewer alerts." The researchers suggested widening alarm thresholds, monitoring more selectively, and—most importantly—customizing alarm parameters to the individual patient’s clinical status.

In software, this translates to moving away from global thresholds (e.g., "Alert if 5xx > 1%") toward state-aware monitoring. An alert should fire based on a service's current context, such as its recent deployment history or current traffic profile. While most observability vendors sell deduplication and smarter routing, these are merely presentation layers. They make the noise organized, but they don't make the signals more actionable. True improvement requires the harder discipline of per-service tuning and monitoring fewer things more deliberately.

The second-order cost nobody budgets

When you successfully eliminate non-actionable alerts, you inadvertently increase the average difficulty of an on-call rotation. The remaining alerts are rarer and less familiar, meaning engineers cannot build the "muscle memory" that comes with frequent, simple fixes.

We must accept that grouping signals is a presentation, not a diagnosis. A tool that tells you ten signals fired at the same time is helpful, but it still leaves the engineer with a hypothesis-elimination task. If you do not budget for this, your senior engineers will burn out on "low volume" rotations because the three alerts they did get required four hours of deep forensics each.

To support this new reality, every surviving alert must have:

• An attached runbook with the first three diagnostic commands.
• A clear dependency model to separate upstream causes from downstream effects.
• Explicit recognition that a "grouped" alert is a request for complex investigation, not a simple restart.

What to report upward

To provide a true picture of operational health to leadership, report these three numbers in tandem:

1. Pages per Rotation: The standard volume metric (ideally trending down).
2. Actionability Rate: The percentage of pages that changed a responder's behavior (ideally trending up).
3. Time to First Diagnostic Action: A measure of how quickly an engineer can orient themselves (ideally staying flat or decreasing).

If volume falls but the time to first action rises, you have concentrated judgment. You have successfully filtered the noise, but you now owe your team the diagnostic support to handle the complexity that remains.

At Operate, we focus on this post-filter reality. The work left after the noise is removed is often a difficult hypothesis elimination task across code, infra, and logs. Operate’s Context and Root Cause agents perform this investigation automatically, providing a verification agent to check conclusions before they reach a human. This doesn't necessarily reduce the number of pages, but it drastically reduces the cognitive load required to determine what to do once one arrives.

Sources & further reading

• According to Fleischman et al. (2020), only 0.8 percent of emergency department monitor alarms resulted in a change in clinical management.
• According to AHRQ PSNet, The Joint Commission estimates that 85 to 99 percent of clinical alarms are non-actionable.
• According to KFF Health News, alarm fatigue was associated with 80 deaths in a three-year window of Joint Commission data.
• According to a recent discussion on r/sre, reducing alert volume can lead to a concentration of complex, harder-to-diagnose incidents.