Your LLM Fallback Was Not Independent on September 3

An LLM fallback is the plan to route model calls to a second provider or model when the first one fails. We think most of those plans rest on an assumption nobody has tested, which is that the two providers do not share anything that can fail at once.

The morning of 3 September 2026 is a useful case because three providers failed close together and the public record still has no shared cause. That gap is the point. A team that cannot see its providers' dependencies cannot claim its fallback is independent.

This post covers what the providers actually said, why the most popular explanation was unconfirmed, what independence would require, how model APIs fail differently from databases, and what to instrument on your own side.

What the Providers Said About the September 3 Outages

The Register's report on the outages collected the official statements. We list them as the providers gave them, without joining them into one story.

• OpenAI told The Register that "a routing error starting around 7:43 am PT" made ChatGPT and Codex unavailable for some users, and that a solution was implemented by about 8:17 am PT.
• Anthropic's status page showed a three-hour six-minute outage with elevated error rates on Sonnet 5 and other models, and a spokesperson said service was restored at 16:16 UTC after an infrastructure issue.
• SpaceX's Grok status page showed an investigation starting at 6:30 am PT. That afternoon SpaceX blamed "an outage at our Memphis compute center" and apologised to its "impacted compute partners."
• Cloudflare, which all three use in some capacity, told The Register its systems were operating normally, and the AWS, Google Cloud and Azure status pages showed no relevant problems.

We see three different stated causes, a routing error, an infrastructure issue and a compute center outage, inside one morning. None of the providers has published a joint postmortem.

The Explanation That Spread Fastest Was Never Confirmed

When several services fail together, people look for one shared cause. On the day, 9to5Google suggested the problem might involve Microsoft Azure, which was also seeing a spike in user reports. The same article noted that Microsoft said this was not the case.

We think that pattern deserves attention, because it happens inside companies too. Under pressure, a single tidy cause feels more satisfying than three unrelated ones, and a plausible shared cause can spread faster than the evidence for it.

The same habit produces wrong conclusions in internal root cause analysis. Our guide to deciding if a provider is down or it is your own system covers how to test that kind of claim against your own telemetry before acting on it.

The Compute Partner Apology Matters More Than the Cause

The most revealing line of the day came from SpaceX's apology to its compute partners. It confirms in public that model providers can be tenants on someone else's specialised hardware.

Engadget, in its coverage of the apology, noted that Anthropic and the SpaceX AI business had signed a deal earlier in the year for Anthropic to lease compute, and that Anthropic had not said if its issue was related. We want to be plain about this. A shared cause is an inference, not a confirmed causal chain.

The argument does not depend on the inference being true. It depends only on the fact that a customer cannot see who supplies its provider's compute, network or upstream services, and so cannot rule out a shared failure.

LLM Fallback Assumes an Independence Nobody Can Test

Most teams handle model-provider risk by adding a second provider and a failover path. That plan is only as good as the independence between the two, and in the model layer independence is almost entirely hidden.

Cloud providers at least publish regions and availability zones, which lets a customer place workloads to avoid shared failure. Model providers publish far less about what sits underneath them. From the outside, we cannot see the following.

• Which compute clusters or data centres each provider rents, and from whom.
• Shared network paths, content delivery or DNS providers in front of the API.
• Shared dependence on the same hardware suppliers or upstream platforms.
• How the fallback provider behaves when traffic from a failed competitor arrives all at once.

The last item is the one teams forget. Even two genuinely independent providers can fail together if every customer of the first one fails over to the second at the same moment. Google's SRE book describes the general mechanism in its chapter on addressing cascading failures, where load shifted from a failed cluster overwhelms the survivor.

None of this makes an LLM fallback useless. It means the fallback is a hypothesis, and a team should treat it like one until it has evidence.

Model APIs Fail in Three Partial Ways, Not All at Once

Traditional monitoring looks for an up or down signal. Model APIs often fail in partial ways that a simple availability check passes. On 3 September, 9to5Google reported that Claude recovered most models while Opus 4.8 and Opus 5 were still affected, so the outage depended on which model you called.

We watch for three partial failure shapes.

1. Latency Inflation

The API still answers, but tail latency climbs. Requests hold connections and worker slots longer, pools drain, and the provider's slowness becomes your outage. Our p99 latency guide covers why averages hide this.

2. Silent Quality Change

The response is a valid 200 OK with well-formed JSON, but the content is worse or wrong. No error rate moves. We wrote about this class in AI agent failures that return 200 OK.

3. Shifting Rate Limits

A provider under stress may tighten capacity, so intermittent 429 responses appear. They look like a problem with your own quota or configuration, which sends the investigation in the wrong direction.

What to Instrument on Your Side of the Model API

A provider's status page is a lagging, provider-written summary. We would not rely on it to decide when to fail over. The truth has to be measured from inside your own request path.

1. Record per-model latency percentiles, error classes, timeouts and 429 rates yourself, tagged by provider and model version.
2. Run a small set of canary prompts with known-good answers continuously, because only a correctness check sees a well-formed wrong answer.
3. Decide in advance which features degrade, which queue work for later and which fail hard, so nobody makes product decisions during the outage.
4. Set strict timeouts and circuit breakers on every model call, since a slow provider holds your resources longer than a dead one.
5. Send a small, steady share of real traffic through the fallback, so credentials, prompt formats and token limits are proven to work.
6. Measure the switch itself, including how long failover takes and what it costs in latency and tokens.

Retries deserve special care during a provider incident. Uncapped retries against a struggling API multiply load at the worst moment, which we explain in our post on retry storm amplification.

When a Second Provider Is Still Worth Having

We are not arguing against a second provider. On 3 September, OpenAI's stated cause was a routing error inside its own service, and its window was about 34 minutes. A fallback to another provider would have covered that failure, whatever was happening elsewhere.

Provider-specific bugs, bad deploys and model-specific degradation are common, and a tested fallback handles them well. The claim we push back on is broader, that having a fallback makes a team safe from correlated failure. It does not, and the honest position is to say what the fallback covers and what it does not.

Nor do we think this is a case for every team to self-host models. Running inference hardware is a large operational burden. Where control over the model matters, tools that let you bring your own AI provider, including a model hosted in your own network give a middle path, because the choice of provider stays with you.

Treat Your LLM Fallback as a Claim That Needs Evidence

The general lesson reaches beyond model APIs. Payment processors depend on banks, CDNs depend on transit providers, and both halves of a redundancy can share something nobody listed. The useful question is what would have to be true for both halves to fail at once, and if you would see it from the outside.

For an LLM fallback, we would answer it with data from your own request path, a rehearsed switch and a clear statement of what the fallback does not cover. When a degradation appears that no status page acknowledges, Operate runs dedicated AI agents that investigate root causes with evidence from your own systems and propose fixes for engineers to review.