Google Says Troubleshooting Is Learnable and Teachable. It Also Says the Hiring Bar Is Coding and Systems.
TL;DR: Most engineering hiring loops measure construction (coding) while production capacity is consumed by diagnosis (troubleshooting). While Google’s SRE doctrine treats debugging as a teachable skill rather than a selection criterion, recent selection research suggests that job-specific work samples—like a dedicated debugging interview—are more predictive of performance than general coding rounds.
The round that is not in your loop
Imagine a major production incident. The system is failing in a way that defies the runbook. The engineer who eventually finds the answer doesn't do it by writing an elegant algorithm or refactoring a service. They find it by ruling things out—methodically isolating variables, questioning their own assumptions, and forming a mental model of a system they didn't build.
Now, look at your hiring loop. What stage in that process would have detected that ability? For most organizations, the honest answer is nothing. We spend hundreds of hours interviewing for the ability to build, while we pay a massive, daily tax on our inability to diagnose.
Google says both things, in the same book
The most prominent literature on this contradiction comes from Google’s own SRE handbook, which presents a curious internal tension. According to Google SRE Book Chapter 12, troubleshooting is often viewed as an innate skill, but Google explicitly believes it is both learnable and teachable, formalizing it as an application of the hypothetico-deductive method.
To support this, Chapter 28 outlines an intensive training regimen: the "Wheel of Misfortune" roleplays, a class on "Reverse Engineering a Production Service," and exercises like "burning a search cluster to the ground." The goal is to build three attributes: reverse engineering skills, statistical thinking, and the ability to improvise when procedures break.
Yet, when you look at Chapter 1, the hiring bar is described almost entirely in terms of coding and systems internals. Fifty to sixty percent of SREs are hired via the standard software engineer procedure; the rest face a bar of coding plus UNIX internals and networking. Troubleshooting judgment is named repeatedly as a training objective, but almost never as a selection criterion.
This is a coherent position: if you believe a skill is teachable, you train for it rather than narrowing your hiring pool to find it. But for the average engineering leader, the question is: are you running the training half, or are you just skipping the selection half?
What diagnosis costs you, with the dates attached
The cost of this "diagnosis gap" is staggering, though the data requires careful handling. According to the 2016 State of DevOps Report (co-authored by DORA and Puppet), even high-performing teams spend roughly 21% of their time on unplanned work or rework. Low performers spend 27%. While this data is nearly a decade old and vendor-presented, it establishes a floor: the best teams in the industry spend at least one day a week just fixing what is broken.
More recent (though self-reported) data from the 2024 Stack Overflow survey shows that 63% of developers cite technical debt as a top frustration, and 61% spend more than thirty minutes every single day just searching for answers. The exact percentage of the "build-vs-operate" split is contested, but the order of magnitude is not: a massive share of your payroll is spent on diagnosis.
The evidence that does not exist
If we spend so much time debugging, why don't we interview for it? The common defense is that coding ability and debugging ability are perfectly correlated.
However, this is the section where we must be honest: there is no study, old or new, that establishes a direct correlation between coding-interview performance and debugging ability.
Key Takeaway: There is currently no peer-reviewed evidence proving that high scores in traditional algorithmic coding interviews predict a candidate's ability to diagnose complex production failures.
The irony is that the "10x programmer" myth originated in a 1968 study by Sackman, Erikson, and Grant, which was primarily a debugging experiment. They found a 25-to-1 difference in debugging time between individuals. But as critics like Sheil (1981) have noted, that study was methodologically flawed and has never yielded usable hiring criteria. We are using construction-based interviews to solve a problem that the industry's founding research identified as a diagnosis-based variance.
What the selection literature actually supports
For decades, the "gold standard" for hiring was general cognitive ability (GCA), based on 1998 research by Schmidt and Hunter. However, the selection literature reversed in 2022.
According to Sackett, Zhang, Berry, and Lievens (2022), the range-restriction corrections used in those old meta-analyses were substantially overcorrected. When the math was fixed, GCA's validity coefficient dropped from .51 to .31.
What emerged at the top? Structured interviews (.42) and job-specific predictors. The authors explicitly state that predictors specific to individual jobs, such as job knowledge tests and work sample tests, fare better than general psychological constructs. This is a peer-reviewed argument for a debugging interview: sampling the actual work of diagnosis rather than testing for general "smartness" via a bug squash interview.
What a diagnosis round looks like in practice
Few companies actually do this. According to developer Zhenghao He, who interviewed at over ten major tech firms in 2022, only Stripe offered a dedicated debugging interview.
Stripe calls it "Bug Squash." Candidates sit side-by-side with an interviewer to solve a real historical bug in an open-source project. They use their own laptop and their own tools. As practitioner Jake Zimmerman notes, this scores hypothesis formation and the ability to build a mental model of a codebase, rather than just tracing lines of code.
Importantly, Zimmerman argues that this format is "AI-proof." Even if a candidate uses a model to find the fix, they still have to explain the "why" and demonstrate the diagnostic path. However, he also lists seven significant costs to this round:
- High authoring cost for the problem.
- Difficulty calibrating the "sweet spot" of complexity.
- Needing multiple bugs per project to prevent leaks.
- Needing different questions for different languages.
- Multi-platform build issues.
- Slow build speeds on candidate hardware.
- Constant upkeep as the snapshot bitrots.
Designing the round: The SRE Rubric
If you decide to add a diagnosis round, it must not become a trivia round about your specific codebase. Instead, use a real historical bug and score the search, not the answer. Use the four common troubleshooting pitfalls from the SRE book as your grading rubric:
- Chasing irrelevant symptoms: Does the candidate get distracted by noise in the logs?
- Lack of system knowledge: Do they fail to understand how to safely test a hypothesis?
- Wildly improbable theories: Do they jump to "cosmic rays" before checking the config?
- Latching onto past causes: Are they trying to solve the last incident instead of this one?
The standard you are asking for
A diagnosis round tests whether an engineer can distinguish an observation from a conclusion under pressure. This is the core discipline of an SRE.
It is also the same standard you should hold your tools to. When a platform claims to have found a "root cause," it should be required to show its work. This is why Operate utilizes a four-agent pipeline where the verification of a fix runs on a different model than the one that generated the hypothesis. The result is not an automated change to production, but a patch file with evidence attached for a human to review—applying the same diagnostic discipline to AI that we expect from our best engineers.
Sources & further reading
- According to Sackett et al. (2022), job-specific work samples and structured interviews are more valid predictors of performance than general cognitive ability.
- According to the Google SRE Book, troubleshooting is a teachable skill that requires statistical thinking and reverse engineering.
- According to the 2016 State of DevOps Report, even elite teams spend over 20% of their capacity on unplanned work and diagnosis.
- According to Jake Zimmerman, a "Bug Squash" interview measures mental models and hypothesis testing, but requires significant maintenance.