When Mayo Clinic re-examined referred diagnoses, 21% changed completely. Root cause analysis has the same failure mode and none of the safeguards.

The 21% Problem: Why Second Opinions are Critical for Root Cause Analysis

TL;DR: Research from organizations like the Mayo Clinic reveals that when a second physician re-examines a diagnosis, 21% change completely and only 12% remain unchanged. Applied to engineering, this suggests that the majority of initial Root Cause Analysis (RCA) findings are likely incomplete or incorrect due to cognitive biases, necessitating a formal "independent second read" protocol for high-severity incidents.

A diagnosis walks into a referral clinic

In 2017, the Mayo Clinic published a study that should haunt every engineering leader who signs off on a postmortem report. Researchers examined 286 patients referred from primary care providers to the clinic’s General Internal Medicine division. They weren't looking for rare tropical diseases; they were looking at how often the first professional opinion held up under scrutiny.

The results, published in the Journal of Evaluation in Clinical Practice, were a sobering wake-up call for diagnostic accuracy. According to the Mayo Clinic study, only 12% of patients received a final diagnosis that was the same as their initial one. In 66% of cases, the diagnosis was refined or redefined. Most strikingly, in 21% of cases—one in five patients—the final diagnosis was distinctly different from the original.

This is the "21% Problem." In the high-stakes world of medicine, one out of five people is being treated for the wrong ailment until a fresh set of eyes intervenes. In the world of software engineering, specifically during incident response, we face a nearly identical diagnostic challenge—yet we lack the protocol of a second opinion.

When a production system fails, the pressure to restore service is immense. We scramble for a narrative that explains the telemetry. The moment we find a plausible culprit, we stop looking. If medicine, with its centuries of diagnostic rigor, faces a 21% complete-error rate, we must ask: What is the error rate for our root cause analysis when we rely on the first engineer who happens to be on call?

Why first diagnoses fail: Anchoring and premature closure

In both medicine and incident response, the primary blockers to accuracy aren't lack of data, but cognitive shortcuts. The Human Factors research in healthcare identifies several key biases that contribute to the 21% problem, all of which mirror the frantic environment of a Slack incident channel.

Anchoring (Root Cause Analysis Accuracy)

Anchoring occurs when a diagnostician relies too heavily on the first piece of information offered. In a hospital, this might be a specific symptom mentioned by the patient. In a DevOps environment, this is often the "Recent Deploys" list. If a service starts throwing 500s three minutes after a frontend deploy, the team "anchors" on that deploy. Even if the telemetry later points to a regional AWS networking blip, the team may burn hours trying to revert a clean frontend build because they cannot un-anchor from the initial correlation.

Premature Closure

This is the most common diagnostic error in medicine. It is the tendency to stop the diagnostic process once a potential explanation is found. According to researchers, this often happens because the brain seeks the relief of "certainty" during high-stress events. Once an engineer finds a spike in CPU usage that correlates with the incident, they declare "Resource Exhaustion" as the root cause and stop investigating why the CPU spiked or if the spike was a symptom rather than the cause.

Availability Bias

We diagnose based on what we’ve seen recently. If a team spent all last week battling a memory leak in the billing service, every latency spike this week will be viewed through the lens of a memory leak. This "diagnostic momentum" creates a feedback loop where teams solve the same perceived problem over and over, while the actual underlying architectural flaw remains invisible.

The RCA equivalent: The first plausible Grafana correlation wins

In the software world, we treat the postmortem as a record of truth, but it is often just a record of the first plausible story that satisfied the Room. We see this play out in "The Deploy Did It" narrative.

Imagine a Sev1 incident. The system is down. The on-call engineer sees a spike in database locks. They also see a database migration was run an hour ago. The "Root Cause Analysis" is written: Inefficient migration caused lock contention. The fix is applied, the system recovers (perhaps coincidentally, or because the traffic load dropped), and the incident is closed.

But what if the migration was fine, and the true cause was a hidden retry-storm from a third-party API that just happened to saturate the DB at the same time? Without an independent second read, the false "Root Cause" becomes part of the company's permanent technical mythology. We spend engineering cycles "fixing" the database migration tool when the actual reliability risk—the retry logic in a microservice—remains a ticking time bomb.

Key Takeaway: To avoid confirmation bias in incident response, engineering teams must treat the first root cause as a hypothesis to be challenged rather than a conclusion to be documented.

What an independent second read looks like

A true second opinion, as practiced in the Mayo Clinic study, requires independence. It is not enough for a senior engineer to "review" a junior engineer’s postmortem. If the senior engineer reads the draft first, they are already poisoned by the initial author’s anchoring.

An independent second read implies a separate path of evidence.

Information Isolation: The second reader should ideally start with the raw telemetry (logs, traces, metrics) and the timeline of events, but without reading the initial hypothesis.
Re-derivation: The goal is to see if a different person, starting from the same data, arrives at the same conclusion.
Alternative Hypotheses: In medicine, doctors are taught to ask, "What else could this be?" An independent RCA process should mandate the listing of at least two alternative theories that were disproven.

In the context of the burgeoning field of AI-assisted operations, this is even more critical. If you use an LLM or an automated agent to generate a root cause, you cannot ask that same model instance to verify its own work. This is "self-grading," and it is the fastest way to hallucinate a false narrative that feels technically sound but is factually empty.

A lightweight protocol: Second-read checklist for Sev1 postmortems

You don't need to double your headcount to implement a second-opinion culture. You only need to apply it where the stakes are highest. For any "Severity 1" or "Business Critical" incident, implement this 15-minute second-read protocol:

The "Clean Room" Reviewer: Assign one engineer who was not on the active incident response to act as the second reader.
The Evidence Challenge: For every "Root Cause" identified, the reviewer must ask: "Is there any data point in our logs that contradicts this?"
The Pro-Forma Alternatives: The RCA document must contain a section titled "Disproven Hypotheses." If the team can't name two things it thought it was but wasn't, they haven't looked deep enough.
Verification of Fix: Does the proposed fix actually address the evidence, or just the symptoms? For example, if the cause was "High CPU," does the fix just add more CPU (symptom management) or address the infinite loop (root cause)?

Where this is heading: Verification as a first-class stage

As we move toward 2026, the complexity of distributed systems is outstripping human ability to hold the entire diagnostic context in one head. We are increasingly relying on AI to parse millions of logs and traces. However, the risk of "automated premature closure" is high.

According to research on clinical decision support, humans tend to over-trust automated suggestions (automation bias). If an AI tells an engineer the root cause is a specific pod restart, the engineer is even less likely to seek a second opinion than if a human teammate had suggested it.

The future of reliable operations isn't just about faster AI diagnosis; it's about building a multi-agent architecture where verification is a distinct and adversarial step. We need systems that act like the Mayo Clinic’s referral department: taking the "referred" diagnosis and attempting to systematically tear it apart to see if it holds.

Sources & further reading

According to the Mayo Clinic, the value of second opinions is most significant in complex cases where a misdiagnosis can lead to significant harm or wasted resources. Their study found that the 21% of cases where the diagnosis was changed entirely represented a massive opportunity for better patient outcomes and cost savings.

According to further research in the Journal of Evaluation in Clinical Practice, the secondary benefit of the second opinion is not just the correction of errors, but the "refinement" of 66% of cases—providing a more nuanced understanding that leads to more effective, targeted treatment.

Solving the 21% Problem in Production

At Operate, we've integrated this philosophy of the independent second read into our core architecture. When our platform investigates an incident, it doesn't just produce a single hypothesis. Operate's Verification agent specifically re-examines the Root Cause agent's hypothesis using a different model and a separate analysis of the evidence—effectively acting as the engineering embodiment of the independent second read. By ensuring the "investigator" and the "verifier" are distinct, we help engineering teams move past the first plausible answer and find the one that actually prevents the next outage.