Retrieval Was Never the Bottleneck: What 1977 Says About Searching Your Own Incident History
TL;DR: The diagnosis for the Three Mile Island accident was written, distributed, and escalated eighteen months before the event occurred. The failure was not a lack of searchable incident history, but the absence of a mechanism to force an authoritative decision on known risks. For modern engineering teams, this proves that indexing past incidents is secondary to establishing a mandatory close-out process for the lessons they contain.
24 September 1977
On a Saturday afternoon in 1977, the Davis-Besse nuclear plant in Ohio experienced a sequence of events that should have changed the industry. A spurious half-trip of the steam and feedwater rupture control system closed a startup feedwater valve. As the steam generator level fell, a pilot-operated relief valve (PORV) lifted nine times and then stuck open from rapid cycling.
High-pressure injection (HPI) started automatically, as designed. However, the operators—mistakenly believing the pressurizer level was too high—manually throttled the injection. According to the NRC’s Accident Sequence Precursor (ASP) description, the plant was only at 9 percent power at the time. This lower heat load allowed the crew to close a block valve twenty minutes later and prevent a catastrophe. Because the core did not melt, the event (labeled LER 346-77-016) remained a footnote for over a year.
The engineer who wrote it down, five weeks later
By November 1, 1977, the retrieval problem was already solved. J. J. Kelly, an engineer at Babcock & Wilcox (B&W), the reactor's designer, wrote a memorandum titled "Customer Guidance on High Pressure Injection Operation." As reproduced in Understanding Failures in Organizational Discourse by Herndl, Fennell, and Miller, Kelly described the September Davis-Besse event in two sentences and recommended specific changes to operator guidelines.
He closed the memo with a clear request: "I would appreciate your thoughts on this subject." He sent it to seven named individuals within the organization. This was the moment the diagnosis became official. The information was not hidden; it was indexed, correct, and distributed to the people responsible for reactor safety.
The refusal, the escalation, and the sentence
The organizational response was a study in friction. On November 10, a manager named Walters denied the need to change instructions. This triggered an escalation. On February 9, 1978, Bert M. Dunn, Manager of ECCS Analysis, wrote to Jim Taylor, Manager of Licensing, regarding "Operator Interruption of High Pressure Injection."
In that memo, transcribed in the Staff Report to the President's Commission, Dunn wrote the sentence that would haunt the industry:
"Had this event occurred at full power and other than insignificant burnup, it is quite possible, perhaps probable, that core uncovery and possible fuel damage would have resulted."
Dunn added that this was a "very serious matter" and deserved "prompt attention." By August 1978, the disagreement reached a department head, Karrasch. In later testimony, Karrasch admitted he thought the memo raised "rather routine questions" and delegated the follow-up. According to the Kemeny Commission findings, Karrasch finally answered the memo seven months later—not in writing, but in a brief conversation at an office water cooler. The warning was never converted into an instruction for the utilities.
Three other copies of the same warning were also in the building
The Kelly and Dunn memos were not isolated anomalies. The Rogovin Report (NUREG/CR-1250) identifies a consistent chain of precursor warnings that were documented but not acted upon:
- The Dopchie letter (1971)
- The Beznau incident (1974)
- The Michelson report (September 1977)
- The Rancho Seco incident (March 1978)
The Michelson report, authored by a TVA engineer, reached B&W in April 1978. B&W took nine months to respond and failed to notify its customers. Even more striking, an NRC inspector raised the Davis-Besse HPI question independently in 1978. After five months of inaction, he used the NRC's "open-door" policy to meet directly with two commissioners. That meeting took place exactly one week before the accident at Three Mile Island.
28 March 1979
Eighteen months and four days after the Davis-Besse precursor, Three Mile Island Unit 2 suffered the exact failure Kelly and Dunn had predicted. The same valve stuck; the same instrument readings confused the operators; the same manual throttling of HPI occurred. The only difference was the power level: 97 percent instead of 9 percent.
The Kemeny Commission finding B.a stated: "The warning, issued thirteen months before the accident, if heeded, could have prevented it." The retrieval of information was successful, but the institutional response was nonexistent.
Why retrieval was never the bottleneck
This historical chain is a direct answer to the modern argument for "vector databases for SREs." The current industry trend suggests that if we simply index every postmortem and use RAG (Retrieval-Augmented Generation) to surface them, we can prevent recurring incidents.
The 1977 records prove that searchable incident history is not a safeguard. The Davis-Besse event was recorded in a licensee event report and indexed. The analysis was written, correct, and specific. It reached the desks of seven engineers, a licensing manager, a department head, and two federal commissioners.
Key Takeaway: An incident analysis only has value if it is attached to a mandatory decision-making mechanism that prevents it from being classified as "routine."
The bottleneck was not retrieval; it was the lack of a mandatory close-out. No one had the unilateral authority to change the operating instructions, and no system forced a "yes" or "no" decision on the proposed fix.
What the industry built afterwards
The nuclear industry eventually realized that hardware logs were insufficient. The Rogovin Report noted that incidents were previously assessed "almost entirely from the perspective of the hardware with little concern about what the operator saw or did."
While the Lewis Committee had recommended systematic precursor analysis in late 1978, the NRC's formal Accident Sequence Precursor (ASP) program only began work on June 15, 1979—eleven weeks after the accident. In the NRC's FY2005 ASP annual report, the original 1977 Davis-Besse event was retrospectively scored at a conditional core damage probability (CCDP) of 7 x 10⁻², significantly higher than the 10⁻³ threshold for a "significant precursor." The data was there; the process for respecting that data was not.
The four questions worth asking about your own incident history
For engineering leaders, the lessons of 1977 suggest that "finding the past" is the easy part. To prevent your next major outage, ask four questions about your last resolved incident:
- Was the root cause analysis written in a way that a stranger could find and understand it?
- Did anyone outside the immediate team have to review and sign off on the findings?
- Was there a specific code or process change implied, and does that change have an owner and a hard deadline?
- If a change was proposed but rejected, who was authorized to record that rejection as "routine"?
In 1977, B&W and the NRC satisfied the first two questions, yet the accident happened anyway. The failure was in the transition from knowledge to action.
At Operate, we believe the output of an investigation shouldn't be a document that disappears into a searchable pile. This is why our investigation pipeline ends in a concrete patch file that a human must explicitly review and merge. The incident isn't "solved" when the cause is found; it is closed only when a human makes a decision to change the system.
Sources & further reading
- According to the Kemeny Commission (1979), findings A.7 and B.a, the failure to provide information to utilities about correct operator actions was a primary contributor to the accident.
- The NRC FY2005 ASP annual report (SECY-05-0192) confirms the high precursor significance of the 1977 Davis-Besse event.
- As detailed in Herndl, Fennell, and Miller (1991), the "water cooler" testimony illustrates the informality that undermined formal warnings.
- According to the Rogovin Report (NUREG/CR-1250), multiple precursor events were documented across the industry throughout the 1970s.