Back to blog
    Incident Management

    Incident History Retrieval Was Not the Three Mile Island Gap

    Written by:OperateOperate TeamUpdated 6 min read

    A near-identical sequence happened at Davis-Besse eighteen months before Three Mile Island. What it says about searching incident history versus acting on it.

    Incident History Retrieval Was Not the Three Mile Island Gap

    Quick Answer

    The initial sequence at Three Mile Island had already happened about eighteen months earlier at another reactor of the same design, and the lesson did not reach the operators who needed it. Searching past incidents was not the gap. Teams need a close-out step that forces a decision on each lesson, with an owner and a deadline, so that a written analysis does not sit unread.

    Incident history is easier to search than it has ever been, and many teams now plan to index every postmortem and surface the relevant ones with retrieval tools. We think that is useful and incomplete. The 1979 accident at Three Mile Island is a clear case where the knowledge existed in the organisation and still did not change what the operators did.

    In this essay we describe the precursor, explain why retrieval was not the bottleneck, and give four questions to ask about your own incident history. We also say when search is exactly what a team needs.

    The Same Sequence Eighteen Months Earlier

    The US Nuclear Regulatory Commission's fact sheet on the accident records the March 1979 sequence at Three Mile Island Unit 2. A relief valve "should have closed when the pressure fell to proper levels, but it became stuck open." Operators misjudged the water in the core, and the primary system was starved of emergency cooling water.

    Wikipedia's summary of the accident states that the initial causal sequence "had been duplicated 18 months earlier at another Babcock & Wilcox reactor." That reactor was Davis-Besse in Ohio. We rely on that summary for the precursor, and we have not verified the detailed internal memos that other retellings quote.

    The Information Existed and Did Not Reach Operators

    The same summary says Babcock engineers recognised the problem and that the company failed to clearly notify its customers of the valve issue. It also records the criticism that important safety information was not communicated.

    The structure of the failure is the point. A correct diagnosis existed inside one organisation. It did not turn into an instruction that changed what operators did at another site. We read that as a close-out failure and not a retrieval failure.

    Key Takeaway. An incident analysis only has value if it is attached to a mandatory decision that prevents it from being treated as routine.

    Why Incident History Retrieval Was Not the Bottleneck

    Today's version of this argument is that we can index every postmortem, run semantic search over the archive and let tools surface the right one at the right moment. We find that attractive.

    The Three Mile Island record suggests it is not enough. In the accounts we have read, the knowledge existed inside the organisation. What was missing was the step that forces a yes or no on the proposed change, and an owner who must carry it out.

    Google's SRE book reaches a similar view about postmortems. It defines a postmortem as "a written record of an incident, its impact, the actions taken to mitigate or resolve it, the root cause(s), and the follow-up actions to prevent the incident from recurring." It also warns that "An unreviewed postmortem might as well never have existed." The full chapter on postmortem culture sets out how to review them.

    Four Questions to Ask About Your Own Incident History

    We would put these four questions to the last resolved incident your team had, and answer them from the record.

    1. Was the analysis written so that a stranger could find and understand it?
    2. Did anyone outside the immediate team review and sign off on the findings?
    3. Did the analysis imply a specific change, and does that change have an owner and a deadline?
    4. If a change was proposed and rejected, who was authorised to record the rejection, and where is it written?

    Most teams can answer the first two. The third and fourth are where lessons die, and our piece on how blameless postmortems record the answer and lose the search looks at what a record should keep.

    We think the output of an investigation should end in a concrete proposed change that a person must accept or reject. That is the reason we favour dedicated AI agents that propose fixes for your team to review, so that the decision stays with people.

    When Search Is Exactly What You Need

    We would not wave away retrieval. A team with no usable incident archive, scattered documents and no tags will repeat itself for lack of memory, and search is the first investment.

    Search also helps at the moment of an incident, when a responder wants to know if this symptom has been seen before. Our notes on escalation metrics and answerability rate describe how to tell if the right information was reachable in time. The argument here is only that retrieval and close-out are separate controls, and that the second one is the more often neglected.

    Close the Loop on Every Lesson

    Three Mile Island is a reminder that knowing is cheap compared with acting. Index your incidents, and then give every proposed change an owner, a deadline and a recorded decision.

    Your incident history is only as good as its close-out rate. We would start with the last five postmortems and check how many of their action items were completed or explicitly rejected. Our write-up of incident closure rate versus recurrence rate shows how to measure if the loop is closing.

    When you want help turning an investigation into a reviewable change, Operate generates a .patch file that an engineer reviews and applies manually.

    Frequently Asked Questions

    We rely on a published summary that says the initial sequence at Three Mile Island had been duplicated eighteen months earlier at another Babcock and Wilcox reactor. That reactor was Davis-Besse, and operators there also faced a stuck relief valve.

    We read the record as a communication and close-out failure. A published summary says the reactor designer failed to clearly notify its customers, and critics pointed to a lack of communication of important safety information, so no mandatory step turned the analysis into guidance.

    We use Google's definition, which is a written record of an incident, its impact, the actions taken, the root causes and the follow-up actions. We add that it only helps if someone reviews it and the follow-up actions have owners.

    Not by itself, in our view. Search helps a responder find a similar case, and it does not make anyone act on a lesson. We pair retrieval with a close-out step that forces a recorded decision on each proposed change.

    We give each action item an owner and a deadline, review postmortems regularly and track completion and recurrence. Rejected items get a written reason, so a decision to do nothing is visible and can be challenged later.

    About the author

    Operate

    Operate Team

    The team behind Operate

    Operate Team builds Operate, enterprise AI agents for production operations. Operate answers production questions from Slack and Microsoft Teams, investigates alerts, and reviews performance, security and cost. It finds the root cause with evidence and proposes a fix for an engineer to review, with controlled access to your systems.

    Share: X LinkedIn
    #incident response
    #SRE
    #engineering management
    #postmortem

    Keep reading