Code Yellow in Software Engineering Is a Lagging Indicator

A code yellow in software engineering is a formal, company-wide declaration that a problem needs concentrated, cross-functional attention now, before it becomes a crisis. We think it is one of the most useful escalation tools an engineering organisation has, and also one of the clearest signs that something was missed earlier.

The trigger is rarely a single outage. It is usually a slow build-up of degradation that nobody was staffed to diagnose, until customers stop distinguishing between one bad week and the next.

This post covers what code yellow and code red mean, how one company ran its first code yellow, the three signals that fire months before a declaration, how to write the protocol in advance, and the failure modes that turn it into theatre.

The Pattern Before the Protocol Is Months of Unexplained Flakiness

Most engineering leaders know the state well. Nothing is fully down, but background jobs lag on certain days, a search index drifts stale, and a few large customers report intermittent timeouts that support cannot reproduce.

None of those issues justifies a high-severity incident on its own, so none of them gets an incident commander or a war room. We call this the gray zone, the unstaffed middle between healthy operations and active firefighting.

Because each issue is sub-critical, engineers investigate briefly and return to roadmap work. The team's understanding of the system decays, workarounds become tribal knowledge, and eventually the friction is high enough that feature delivery slows. That is usually the moment someone proposes a code yellow.

James Stanier describes exactly this build-up at Provet. Towards the end of 2025, he writes, the company could barely get through a week without some infrastructure degradation, outages were hard to diagnose, and when a major AWS outage hit in October, customers did not separate Amazon's problem from Provet's.

What Code Yellow and Code Red Mean and Where the Terms Came From

The terms come from Google. In his essay Code Yellow, Code Red, Stanier recounts Steven Levy's story from In The Plex, in which the name came from a yellow tank top owned by engineering director Wayne Rosing. Whoever wore it led the effort and could pull anyone at Google off their current project to help.

The practice has since spread. LinkedIn's SRE team presented their version at SREcon19 Americas, describing Code Yellow as their process for "righting the ship" for teams that are struggling under operational load. Stanier also lists Meta, Shopify, Instacart and OpenAI as users of variations.

Dimension	Code yellow	Code red
Severity	Serious but not existential	Existential threat or critical failure
Purpose	Preventative intervention	Emergency response
Typical duration	Weeks to months	Days to weeks
Normal work	Deprioritised but not fully stopped	Fully paused

We want to correct a common confusion, since a code yellow is not the same thing as an error budget policy.

Google's SRE workbook gives an example error budget policy in which, if a service has exceeded its budget for the preceding four weeks, the team will "halt all changes and releases other than P0 issues or security fixes until the service is back within its SLO." That is a pre-agreed, per-service rule. A code yellow is a broader organisational declaration, often across many services and teams.

How Provet Ran Its First Code Yellow

Stanier's account is useful because it is specific. Provet announced its first code yellow to the whole company in writing, explaining the concept, the structure, the exit criteria and the reasons. The target was eight consecutive weeks of zero downtime with tighter SLAs.

The first week was chaotic by his own description, with daily stand-ups, daily war rooms and a full audit of monitoring, logging and alerting. The team walked the stack from the databases upward, looking at access patterns, slow queries, timeouts and rate limits, and also from the outside in, looking for gaps in monitoring.

Two structural choices stand out to us. Updates went to a public channel so anyone could follow the work, and people working on the code yellow had authority to ask any owning team to reprioritise immediately. The company met its target, with one brief degradation in the period.

Stanier's own reflection is that he would have pushed even harder in the first two to three weeks, since that is where most of the work happened.

Three Code Yellow Warning Signs That Fire Months Earlier

A declaration is a lagging indicator. By the time leadership agrees to pause roadmap work, the degradation has usually been visible in the data for a long time. We watch three signals, and none of them needs new tooling.

1. The Same Symptom Keeps Coming Back

Look for repeat-symptom incidents, the same class of failure in different weeks with the same temporary mitigation each time. A latency alert cleared by recycling pods three times in a quarter is not three incidents. It is one undiagnosed problem that the team keeps paying for.

The cleanest way to see this is to record a failure signature on every incident and count repeats, the approach we set out in incident recurrence rate versus closure rate. When repeats climb while closure rate stays high, the gray zone is growing.

2. Time to Diagnose Rises While Time to Fix Stays Flat

When a system becomes flaky, the change that fixes a problem usually stays small. What grows is the time it takes to prove why the problem is happening. If the diagnosis portion of your incidents trends upward over a quarter while the repair portion stays flat, the system's complexity is outrunning the team's ability to observe it.

Few teams record those two clocks separately, which is why this signal is missed. We explained how to split them in our guide to reducing MTTR by shortening diagnosis.

3. Unplanned Investigation Work Crowds Out the Roadmap

Track the time engineers spend on investigation that does not end in a fix, such as reading dashboards, tailing logs and running one-off queries on problems that are not formal incidents. This work rarely appears in sprint planning, so it is taken from the margins.

We know of no reliable threshold that predicts a code yellow, so the trend is the signal. When that share grows for two or three months in a row, a declaration is getting closer. Our post on unplanned work as roadmap capacity shows one way to measure it.

Write the Escalation Protocol Before You Need It

The time to design a code yellow is before the gray zone forces one. Stanier lists four components a team needs before starting, and we agree with all four.

1. A problem statement simple enough that anyone in the company can repeat it.
2. Exit criteria that are specific, measurable and time-bound, defined before entering the code yellow.
3. A timeframe with a start and expected end, no longer than a quarter.
4. An authority structure that says who can pull people in, who reports progress and who can change scope.

Communication sits on top of those four. Stanier treats company-wide announcement as essential rather than optional, because a code yellow that nobody else can see is just one stressed team working harder.

We would add one item of our own, the trigger. Decide in advance which signals, at what trend, prompt the conversation about declaring one, so the decision does not wait for a customer escalation.

Staff the Gray Zone So the Protocol Stays Unused

We do not think the goal is to run excellent code yellows. It is to need fewer of them. That means treating gray-zone diagnosis as planned work rather than something done between features.

• Reserve explicit capacity for investigating recurring, sub-incident problems, and review what it found each month.
• Give at least one engineer a goal tied to diagnosis time, not only to uptime.
• Reduce the manual cost of each investigation, so more of them reach a cause in the time available.

That last item is where tooling helps most. Teams can use agents that investigate recurring findings and cite the evidence they used, so a slow query or a repeated error gets a documented root cause analysis instead of another restart. The point is the same with or without software, which is that diagnosis has to be staffed before it becomes a crisis.

Code Yellow Failure Modes to Watch For

Stanier names four ways a code yellow goes wrong, and we have seen each of them.

• Overuse, where frequent declarations turn the signal into background noise.
• Performative escalation, where war rooms and dashboards multiply but priorities do not actually change.
• Failure to de-escalate, where a code yellow never ends and becomes indistinguishable from normal operations.
• Shortcut temptation, where new technical debt is taken on to hit the exit criteria, setting up the next code yellow.

His point about ending is the one we would underline. Exhaustion is not an exit criterion, and neither is things seeming better. If a team cannot exit, the criteria were wrong.

When Declaring a Code Yellow Now Is the Right Call

Leading signals are for avoiding the next code yellow, not for delaying a needed one. If customers are already losing trust, as Provet's were after the AWS outage, we would declare rather than spend another quarter measuring trends.

The same holds when the problem spans many teams and no single owner can reprioritise enough work alone. As Stanier puts it, "If you’re waiting for the dramatic failure, you’ve already waited too long." A declared, time-boxed code yellow is far better than a quiet one where a single team absorbs the cost without organisational support.

Treat Your Next Code Yellow as a Missed Signal

A code yellow is a good tool and a bad habit. Run it with a clear problem statement, exit criteria, a timeframe and real authority, and then use its retrospective to ask which earlier signal was missed.

We would track repeat symptoms, the diagnosis clock and unplanned investigation work, because that is how most of the next declaration can be avoided. For the investigation behind those signals, Operate runs dedicated AI agents that monitor production systems, investigate root causes with cited evidence and propose fixes for engineers to review.