Verifying AI Root Cause Analysis the Way Pilots Cross-Check
Verifying AI root cause analysis needs a checker that did not produce the answer. Pilot monitoring shows how to build one that reads raw evidence and says no.
Quick Answer
An AI-generated root cause deserves the same treatment a flight deck gives every critical action, a second party that checks it without having produced it. That checker should go back to the raw logs and data rather than the first summary, look actively for evidence that contradicts the diagnosis, and have the authority to reject it outright. A model reviewing its own answer is not independent and tends to confirm its first story.
Verifying AI root cause analysis means testing a machine-proposed cause of an incident against evidence before anyone acts on it. We think the best working model for that check comes from aviation, where one pilot flies and the other monitors, and the monitor's whole job is to catch what the flyer misses.
Most AI incident tools still verify a diagnosis by asking the same model to review its own work. In cockpit terms, that is a pilot reading their own altimeter twice and calling it a cross-check.
In this post we cover why self-review fails, what pilot monitoring and dissimilar redundancy teach, a five-step cross-check procedure, the questions to ask any tool, and where the cockpit analogy stops holding.
Why an AI Cannot Grade Its Own Root Cause
A model that investigates an incident forms a story early, usually from the first few log lines it reads. When we ask the same model to review that story, it reviews it with the same context and the same blind spots.
Research on language models supports the worry. A study of self-correction in large language models, presented at ICLR 2024, found that "LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction."
The failure pattern we see in incident work looks like this:
- The model sees a 504 error, notices a database span in the trace, and blames a lock.
- Asked to review, it rereads the same trace, skims past a network timeout flag, and confirms the lock.
- The on-call engineer receives a confident, plausible and wrong diagnosis, and spends the next half hour remediating the wrong thing.
Nothing in that loop was dishonest. It simply had no second source of truth.
What Pilot Monitoring Teaches About Independent Checks
Airline crews split two jobs deliberately. The pilot flying controls the aircraft's path, and the pilot monitoring watches, calls out deviations and intervenes if needed.
The FAA's advisory circular on standard operating procedures and pilot monitoring duties defines the role plainly. The pilot monitoring "monitors the aircraft state and system status, calls out any perceived or potential deviations from the intended flightpath, and intervenes if necessary." It also notes that studies of crew performance, incidents and accidents have identified inadequate monitoring and cross-checking as vulnerabilities for aviation safety.
One line in its description of effective monitoring maps almost exactly onto AI diagnosis. A monitoring pilot keeps in mind that the displays might be showing incorrect information and stays "always on the lookout for other evidence that confirms or disconfirms the information the displays are providing." We read the AI's root cause as one of those displays.
Dissimilar Redundancy and Why Airbus Builds Two Kinds of Computer
Aviation applies the same idea to machines. Two identical flight computers running identical software can fail together from a single design fault, so Airbus deliberately makes its primary and secondary flight control computers different.
A paper by Airbus engineers on fly-by-wire dependability describes the approach. On the A380, the primary computers run on PowerPC processors and the secondary computers on SHARC processors, with different automatic coding tools and different functional specifications, the secondary set being simpler and more robust.
The same paper reports that Airbus moved from letting the two design teams choose components independently to deciding together to choose different ones, because a shrinking pool of suppliers made accidental overlap likely. It cites an A320 flight on which both primary computers were lost after an air conditioning failure raised the temperature, as one of two in-service cases where the dissimilarity paid off.
The lesson we take for AI diagnosis is that difference has to be designed in. Two calls to the same model with slightly different wording are the software version of two identical flight computers, and they will tend to fail in the same way on the same incident.
Verifying AI Root Cause Analysis in Five Cross-Check Steps
These are the steps we apply to any AI-proposed root cause before it reaches a human as a conclusion. Each one borrows a specific habit from the flight deck.
1. Separate the Proposer From the Checker
The component that checks a diagnosis must not be the one that produced it. A different model, a different prompt with different context, or a deterministic rule set all count, but rereading the same conversation does not. This is the pilot monitoring rule, the monitor never flies.
2. Send the Checker Back to Raw Evidence
The checker should read the logs, traces, query results and code itself, not the first model's summary of them. A summary carries the proposer's framing, and checking against it only tests internal consistency. Raw evidence is what lets the check disagree.
3. Make the Checker Look for Disconfirming Evidence
We instruct the checker to find the observation that would make the diagnosis false, such as a timeout that started before the suspected deploy or an error rate on hosts the lock could not touch. Confirming evidence is easy to find for almost any story. The disconfirming search is the one that catches mistakes.
4. Give the Checker Authority to Reject
A checker that can only add caveats is decoration. It needs a path to mark the diagnosis as unsupported and send the investigation back, or to escalate to a human with low confidence. This is where an AI agent that investigates with evidence differs from one that only explains, because a rejected answer never reaches the engineer as a conclusion.
5. Record What the Checker Saw
Every verdict should list the evidence the checker examined and what it concluded from each piece. When a verified diagnosis later turns out wrong, that record shows whether the checker missed something or never had access to it.
How often two independent reviewers actually disagree is its own question, and we cover the evidence in why AI root cause analysis needs a second opinion. The procedure for the human part of the check sits in running root cause analysis past the first why.
Cockpit Practices and Their Root Cause Analysis Equivalents
We use this mapping when we evaluate any AI tool that claims to verify its diagnoses. Each row turns an aviation habit into a question with a checkable answer.
| Flight deck practice | Equivalent in AI root cause analysis | Question to ask the tool |
|---|---|---|
| The monitoring pilot never flies | The checker did not produce the diagnosis | Does the same model instance propose and verify? |
| Cross-check against other instruments | The checker reads raw telemetry | Does the checker see the logs or only a summary? |
| Dissimilar hardware and software | A different model or method checks | What makes the check different from the proposal? |
| Call out deviations and intervene | The checker can reject or escalate | What happens when the check fails? |
| Standard callouts on the record | The verdict lists its evidence | Can we see what the checker examined? |
A tool that cannot answer the last two questions is usually running a self-review and calling it verification.
Where the Cockpit Analogy Breaks Down
The analogy is useful, but we try not to stretch it past what it can carry. Three differences matter.
• Dissimilar Is Not the Same as Independent
In a 1986 experiment, Knight and Leveson had 27 versions of one program written independently from the same specification at two universities. Each was highly reliable, but the number of tests in which more than one version failed was substantially more than expected.
Different checkers still share blind spots, which is why we anchor every check in evidence a human can inspect. We look at the same lesson from the hardware side in redundancy versus diversity after Ariane 501.
• Software Changes Under the Investigation
An aircraft's physics do not change while the crew diagnoses a fault. Production systems do, through autoscaling, retries and the responders' own mitigations, so a check run ten minutes later may be looking at a different system.
• Not Every Incident Needs a Full Cross-Check
For a low-impact issue with an obvious cause, a single investigation plus a quick human read is often enough. We reserve the full procedure for diagnoses that will drive a production change or a customer-facing decision.
Treat Verifying AI Root Cause Analysis as a Separate Job
We would start by asking who checks each AI diagnosis today and what that checker can see. If the answer is the same model rereading its own summary, the first fix is a separate checker with raw evidence and the power to say no.
That separation is the core of verifying AI root cause analysis, and it is how Operate is built. It brings the evidence from logs, databases and code together through read-only adapters, and a separate agent on a different model checks each proposed root cause against that evidence.
Frequently Asked Questions
Pilot monitoring is the flight deck role that watches the aircraft and the other pilot rather than flying. We find the FAA's definition useful well beyond aviation, because the monitor checks system state, calls out deviations from the plan and intervenes when needed, without ever taking the controls.
The pilot flying controls the aircraft's path, while the pilot monitoring checks that path against the plan and the instruments. We borrow the split for AI diagnosis, where one component proposes a root cause and a separate one checks it, so neither grades its own work.
Dissimilar redundancy means backing up a system with one built differently, so a single design fault cannot disable both. We point to Airbus flight computers, where primary and secondary units use different processors, coding tools and specifications. Different builds reduce shared failures but do not remove them entirely.
We verify an AI answer by checking it against the original evidence, not against the answer itself. A separate reviewer reads the raw data, looks for anything that contradicts the claim and can reject it. For incident work that means logs, traces and queries, never just the model's summary.
We trust an AI answer only as far as its evidence goes. A claim that cites specific logs or query results, and has survived an independent check, can be acted on. A fluent answer without evidence is a hypothesis, however confident it sounds, and we treat it that way.
About the author
Operate Team
The team behind Operate
Operate Team builds Operate, enterprise AI agents for production operations. Operate answers production questions from Slack and Microsoft Teams, investigates alerts, and reviews performance, security and cost. It finds the root cause with evidence and proposes a fix for an engineer to review, with controlled access to your systems.
