← Blog · SRE & Engineering Efficiency · July 18, 2026 · 7 min read · By Senior Technical Writer, Operate

Reduce MTTR by attacking the 80 percent: diagnosis, not repair

About 80 percent of MTTR is spent finding the cause, not fixing it. A diagnosis-first playbook for reducing MTTR across code, database, infra, logs, and CI.

Reduce MTTR by attacking the 80 percent: diagnosis, not repair

Reduce MTTR by attacking the 80 percent: diagnosis, not repair

To reduce MTTR (Mean Time to Resolution), engineering teams must focus on the "Mean Time to Identify" (MTTI) or diagnosis phase, which typically accounts for 80% of the total incident lifecycle. While detection and remediation are often automated, the bottleneck remains the human-intensive process of correlating disparate signals across code, databases, and infrastructure to find the root cause.

Most engineering leaders are taught that to reduce MTTR, they need better alerting or automated rollbacks. While these are valuable, they target the edges of the problem. If an incident lasts 100 minutes, detection usually takes five minutes, and the fix (a revert or a restart) takes another ten. That leaves 85 minutes of "searching." This is the diagnosis gap. To truly reduce mttd and mttr, you must build a playbook that treats diagnosis as a structured forensic process rather than a creative brainstorming session.

MTTR has four clocks; one eats the other three

Mean Time to Resolution is not a monolithic metric. It is the sum of four distinct phases:

  1. Detection (MTTD): How long before an alert fires?
  2. Mobilization: How long until the right engineer is in the "war room"?
  3. Diagnosis: How long until the cause is identified?
  4. Remediation: How long to apply the fix and verify it?

What the data says: 80 percent is diagnosis

Industry data and historical benchmarks consistently show where the time goes. According to The Visible Ops Handbook, cited by New Relic, roughly 80% of the MTTR for unplanned outages is spent identifying which change or component caused the failure. Identification is inherently harder than fix-application because modern distributed systems have an almost infinite state space.

Consider the Railway incident on July 2, 2026. This case study illustrates the diagnosis bottleneck perfectly. The service experienced a significant US-East outage. While the initial routing issue was identified and fixed by 08:59, the system remained degraded because of stuck storage connections. It took until 10:45 to identify that specific secondary cause. In a 257-minute total incident window, 106 minutes were spent in pure diagnosis of the "stuck-state" side effect.

Key Takeaway: Instrument and measure diagnosis time separately; most MTTR wins come from faster cause identification across the whole stack, not faster restarts.

Why detection and repair got all the attention

Historically, vendors have focused on detection (Observability) and repair (CI/CD and Infrastructure as Code).

Diagnosis, however, is messy. It requires deep context, the ability to read code, and the patience to query database internals. Because diagnosis was seen as a "human" skill, it was left out of the automation roadmap. To reduce mttr in incident management, we have to stop treating diagnosis as an art and start treating it as a workflow.

The diagnosis playbook, surface by surface

When an incident hits, the clock is running. A diagnosis-first playbook moves through the stack in order of probability.

What changed: deploys, config, dependencies

In a stable system, failure is usually preceded by change. Your first diagnostic step should be a "diff of the world."

Database: slow queries, plans, schema drift

If no code changed, the database is the next suspect. Often, "high MTTR" is caused by a sudden shift in query execution plans.

  1. Check for Schema Drift: Did someone run a manual ALTER TABLE that didn't go through CI?
  2. Slow Query Logs: Look for queries that were fast yesterday but are slow now. This often points to a "tipping point" where a table size hit a threshold that invalidated a cached plan.
  3. Locks: Use pg_stat_activity or equivalent to see if an uncommitted transaction is holding up the queue.

Logs: exception clustering and silent failures

Standard logging often fails during major incidents because the volume spikes. To reduce mttr, you need to cluster logs. Instead of looking at a stream, look at the delta in exception types.

Infra and CI: capacity, cost, flaky signals

Sometimes the cause is external to the application logic.

Answering common MTTR questions

What causes a high MTTR?

A high MTTR is typically caused by "black box" components, lack of searchable audit trails, and siloed knowledge. If only one person knows how the caching layer works, your MTTR is tied to that person's sleep schedule.

What is a good MTTR value?

According to DORA (DevOps Research and Assessment) benchmarks, "Elite" performers have an MTTR of less than one hour. "Medium" performers range between one day and one week. If your MTTR is consistently over 4 hours, your diagnosis phase is likely blocked by a lack of observability.

Is lower MTTR better?

In isolation, yes. However, a lower MTTR should not come at the cost of "bandage" fixes that lead to higher incident frequency (MTBF - Mean Time Between Failures). The goal is stable resolution, not just a quick restart.

How to improve MTTR and MTBF?

To improve both, you must conduct blameless postmortems that focus on "how we could have diagnosed this 50% faster." This often leads to adding better instrumentation or "debug endpoints" that provide instant visibility into internal state.

Measuring diagnosis time explicitly

To reduce mttr, you must measure it. Standard incident management tools like PagerDuty or Rootly allow you to add "milestones." Start tracking:

  1. Incident Start
  2. First Responder Linked
  3. Cause Identified (Diagnosis End)
  4. Fix Applied
  5. Resolved

If the gap between "First Responder" and "Cause Identified" is the largest segment, stop buying better alerting tools and start investing in diagnostic tools.

Where AI assistance actually fits

The current wave of "AI for SRE" often misfires by trying to automate the fix. Giving an LLM write access to your production environment is a security and stability risk many aren't ready to take.

Where AI actually provides value is in hypothesis generation and verification. An AI agent can scan 50,000 logs, 100 recent PRs, and 10 database metrics in seconds—something a human cannot do. It can present a hypothesis ("I think the 10:05 deployment of the billing service caused a lock on the users table") and provide the evidence.

The final step—applying the patch—should still be a human-in-the-loop process. This "AI-assisted diagnosis" shrinks the 80% gap without sacrificing the safety of the production environment.

Common Pitfalls

Sources & further reading

  1. According to New Relic, 80% of MTTR is spent on identification.
  2. The Railway July 2026 Incident Report demonstrates the long tail of diagnosis in complex outages.
  3. LogicMonitor notes that identification in complex stacks can be 3-4x longer than remediation.

Solving the Diagnosis Bottleneck with Operate

If you find that your teams are stuck in the "searching" phase of every incident, it may be time to look at a dedicated diagnosis platform. Operate is built specifically for the 80% problem. It acts as a self-hosted AI SRE that watches your production environment and, the moment an incident occurs, kicks off a proactive Root Cause Analysis (RCA).

Operate uses a four-agent pipeline to investigate code changes, database performance, and logs. It doesn't just alert you; it finds the evidence and drafts a fix as a PR for your team to review. Because Operate is self-hosted, you maintain complete control and privacy over your data while gaining the diagnostic speed of an automated system. It’s an auditable, read-only approach to shrinking MTTR by making diagnosis instantaneous.

#SRE#Incident Management#MTTR#Observability#Engineering Management