The Postmortem Records the Answer and Throws Away the Search. The Search Was the Incident.

TL;DR: Standard incident reviews record a linear timeline of system events, effectively discarding the effortful, iterative search that responders actually performed. To improve reliability, organizations must evolve their postmortem schema to record the hypotheses formed, the signals that misled them, and the instruments that were missing during the investigation.

Two documents from the same incident

Imagine an incident review for a major database outage. The final postmortem document is clean and authoritative. It lists a sequence of system events: at 10:00 UTC, a configuration change was deployed; at 10:05, connection latencies spiked; at 10:45, the change was reverted; at 10:50, the system stabilized. It identifies the "root cause" as a malformed connection pool setting.

Now, consider the reality of the responder’s hour. They spent the first fifteen minutes investigating a suspected network partition because a dashboard showed a misleading green status for the database nodes. They formed three wrong hypotheses—blaming the load balancer, then the disk I/O, then a recent application code change—before finally finding the configuration error. They spent twenty minutes in the wrong service because a p99 metric was aggregated in a way that hid the true source of the latency.

Only the first document—the skeletal timeline of the system—gets saved. The second document—the record of the human search—is thrown away.

What the researchers actually observed

In March 2017, the SNAFUcatchers consortium—a group of researchers and practitioners including David Woods and Richard Cook—convened a workshop in New York to study how engineering teams cope with complexity. The resulting STELLA report remains one of the most significant, yet under-read, primary research documents in the history of SRE.

According to the STELLA report, when experienced researchers observed real anomaly responses at real companies, they found that "neither the sources of the anomaly nor the route to correction were immediately apprehended." Instead, the response was characterized by surprise, uncertainty, and an "effortful and iterative" search for information.

Responders do not simply look up an answer; they generate hypotheses, revise their representations of the system, and navigate through a fog of ambiguous signals. The report catalogues this process not as a failure of the responder, but as a fundamental characteristic of coping with complex, "below-the-line" systems that cannot be observed directly.

And what those same companies wrote down about it

The turn in the STELLA report is jarring. After documenting the messy, iterative nature of the search, the researchers observed the postmortem process at these same companies.

According to the STELLA report’s section 3.5, the typical review is a one-hour meeting held days or weeks after the event. Facilitators gather machine-generated logs and chat histories to present a "skeletal timeline" using presentation software.

The mismatch is total. The actual response was a nonlinear journey through uncertainty and hypothesis generation. The artifact preserved is a linear history of what the machine did. By recording only the answer, the organization discards the incident itself—because the search was the incident.

Why the schema is what it is

We should be fair to current practices. The industry’s focus on "blameless" postmortems was a hard-won victory that made the process safe for humans. Once safety was established, defensibility became the primary goal.

Timelines are easy to reconstruct from machine logs. They are objective and difficult to argue with. The search, however, is difficult to capture; it happens in the heads of responders, with only fragments appearing in Slack or Zoom. A skeletal timeline is legible to stakeholders who weren't there, whereas a record of three failed hypotheses can feel like a record of incompetence to the uninitiated.

The standard schema is a rational response to the difficulty of capturing human cognition. It is also, nine years after the STELLA report, the wrong schema for a learning organization.

Key Takeaway: Incident reviews fail to improve diagnostic speed because they record the system's eventual state rather than the iterative, hypothesis-driven search that consumed the responder's time.

What gets lost, specifically

When we discard the search, we lose four specific types of operational intelligence:

1. Which signal was checked first and why: The ordering of a diagnosis contains the essence of expertise. If an expert checks the thread pool first, that is a heuristic that junior engineers need to learn.
2. Which hypotheses were formed and eliminated: The "wrong" hypotheses are often more useful than the right one. They reveal what the system’s representations made plausible at the time.
3. Which representation misled: A green dashboard that should have been red, or a p50 metric that hid an outage, is a defect in the observability layer. In a skeletal timeline, these are invisible; in a record of the search, they are high-priority bugs.
4. What did not exist: The query no one had written, or the log field that hadn't been added yet. These are the highest-value action items, but they are rarely captured when we only focus on what the system did.

A schema that records the search

To fix this, you do not need to replace your postmortem process. You need to add four fields to your existing template:

• The Hypothesis Log: A table of each hypothesis, what evidence suggested it, and what evidence eliminated it.
• The Misleading Signal: A field for any metric or dashboard that pointed the wrong way or looked "fine" while the system was failing.
• The Missing Instrument: What tool or data point would have collapsed the search time from forty minutes to four?
• Time to First Correct Hypothesis: Record this alongside MTTR. The gap between "Incident Start" and "First Correct Hypothesis" is your diagnostic cost.

Crucially, these must be recorded during the preparation phase—by interviewing participants and reading chat logs—not debated in the room. Once everyone is in the room, hindsight has already begun to rewrite the search into a linear path.

What this changes

By recording the search, you move beyond "preventing the exact incident we just had." You begin to see patterns in how your team is misled. If the same load balancer dashboard shows up as a "misleading signal" across three unrelated incidents, you have identified a systemic weakness in your observability that a skeletal timeline would never have surfaced.

You also gain a real metric for engineering maturity. You can finally answer: Is our diagnosis getting faster, or are our incidents just getting easier?

The nine-year gap

The observation that the postmortem artifact is a "skeletal timeline" was published in 2017. In the nine years since, the industry has automated the creation of these skeletons with LLMs and incident management bots. But if the schema itself is insufficient, automating it only makes the loss of information more permanent.

We have spent a decade making postmortems safe. It is time we made them useful.

Why this matters: As incident response becomes more complex and automated, the diagnostic path—the "search"—is the only part of the process where true learning occurs. If we continue to discard it, we ensure that every new engineer must repeat the same "effortful and iterative" search from scratch.

Sources & further reading:

• According to the STELLA report, anomaly response is characterized by evolving system representations and hypothesis generation.
• The concepts of "Above the line / Below the line" were further popularized by Woods and Cook in their O'Reilly Radar preview.
• Further details on the SNAFUcatchers consortium can be found at their projects page.

In the context of modern SRE, the need for auditable, reproducible investigations is paramount. A system that investigates—whether human or AI-driven—must record its search to be reviewable: which signals it read, which hypotheses it formed, and which it eliminated based on evidence. Operate’s pipeline is designed this way by construction; it uses a separate verification stage to check conclusions against evidence before any PR is drafted. When you move toward making investigations truly auditable, you realize that the skeletal timelines we’ve used for years were never enough.