The Delhi Data Center Fire Is a Blast-Radius Audit You Haven't Run Yet

TL;DR: The June 2024 fire at the STT GDC/Tata data center in Delhi serves as a critical warning for infrastructure teams about hidden shared-fate risks. Even major providers like Google Cloud faced weeks of degradation, proving that disaster recovery plans must account for the physical destruction of single-site dependencies like POPs and battery rooms. These data center fire lessons emphasize that if your primary and backup systems share a building, you don't have a backup.

On June 5, 2024, a fire broke out at the STT Delhi 2 data center, a facility operated by STT Global Data Centres (STT GDC) and Tata Communications. While many localized outages are resolved within hours, this event was different. The physical destruction of the facility led to weeks of service degradation and potential permanent data loss for major enterprises. For engineering leaders, this isn't just another headline—it is a live case study in blast-radius mismanagement and the fragility of "the cloud" when the physical layer fails.

What actually happened (verified timeline: June 5 fire, battery units, extensive damage, ongoing Google Cloud impact into July)

According to Data Center Dynamics, the fire originated in the lithium battery units of the STT Delhi 2 facility. Lithium fires are notoriously difficult to extinguish, and the resulting damage to the 1.1MW facility was described by Tata Communications in a filing with the National Stock Exchange (NSE) as "extensive."

The timeline reveals a slow-motion disaster for affected customers:

• June 5: The fire breaks out, leading to immediate power cuts and hardware damage.
• June 24: Reports from Reuters indicate that Tata Communications informed customers the damage was so severe it was hampering recovery efforts.
• Late June/Early July: Matrix Cellular, a major international roaming provider, expressed fears to Business Standard that they might lose 20 years of customer data stored at the site.
• July: The Google Cloud Status Dashboard continued to track incidents related to "network connectivity issues in the asia-south1 region," stemming from the loss of a Point of Presence (POP) located within the damaged building.

This was not a software bug or a configuration error. This was a "lower-level" failure that bypassed high-level redundancies. It exposed a hard truth: many Disaster Recovery (DR) plans are built on the silent assumption that while a server or a rack might fail, the building itself will survive.

Three failure classes in one event: total data loss, partial degradation, dependency opacity

The Delhi incident highlights three distinct classes of failure that can bypass traditional SRE guardrails.

1. Total Data Loss (The Fate-Sharing Problem): When Matrix Cellular reported the risk of losing two decades of data, it signaled a fundamental DR failure. If 20 years of records can be wiped out by a single battery fire, it suggests that the "backups" were either physically co-located with the production data or were not replicated across a geographically distinct "blast zone."
2. Partial Degradation (The Residual Latency Problem): Google Cloud is world-class at redundancy, yet its traffic was still "limping" weeks later. This happened because a specific Delhi facility served as a critical interconnect node. When that node vanished, traffic had to be rerouted, leading to increased latency and packet loss that affected users across Northern India.
3. Dependency Opacity (The "Where is my data?" Problem): Many teams didn't even know they were dependent on STT Delhi 2. They were customers of Google Cloud or other ISPs. They only discovered the dependency when their services began to lag, leaving them to wait for a third-party root cause analysis (RCA) while their own customers suffered.

Key Takeaway: The Delhi fire demonstrates that physical facility failures create "shared-fate" risks that can degrade even the most sophisticated cloud architectures for weeks.

Audit question 1: does any backup share a building, a room, or a battery bank with its primary?

The most basic lesson from the delhi data center fire is the danger of physical co-location. It is common for teams to feel secure because they have "redundant power" or "off-site backups" that are actually just in a different room of the same data center.

• The Audit: Verify the physical address of your primary storage and your backup storage. If they are in the same building, they share a fire suppression system, a power grid connection, and a battery room. In the event of a lithium fire—which can burn for days and release toxic gases—your "redundancy" is effectively zero.
• The Action: Move critical backups to a different region or, at the very least, a different availability zone (AZ) that does not share a physical facility.

Audit question 2: what single facilities does your traffic silently depend on (POPs, DNS, peering)?

The stt gdc fire google cloud disruption was particularly insidious because it didn't just kill servers; it killed a path for data. Many engineering teams audit their "Compute" and "Database" locations but ignore their "Edge" locations.

• The Audit: Map your network's entry points. If your traffic from Northern India enters the global internet via a single POP in Delhi, you are one fire away from a regional blackout.
• The Action: Implement multi-region ingress. Ensure your DNS can intelligently route traffic away from a degraded region without manual intervention.

Audit question 3: when the provider's status page says degraded, what does your own telemetry say?

During the June outage, many users on r/developersIndia noted that while Google’s status page eventually turned "yellow," their actual application performance had been "red" for hours. Cloud providers often use broad aggregated metrics for status pages, which can mask severe issues affecting specific customer subsets.

• The Audit: Compare your internal Service Level Indicators (SLIs) to your provider's public status page during the next minor blip. If your latency spikes by 300ms but the provider says "all systems green," your observability is properly decoupled. If you rely on their status page to tell you something is wrong, you are flying blind.
• The Action: Set up "canary" probes that simulate user journeys from specific geographic regions. This allows you to detect blast radius infrastructure failures before the provider acknowledges them.

Audit question 4: whose RCA are you waiting on, and what can you investigate yourself?

One of the most frustrating aspects of the Delhi fire was the "wait and see" period. According to Outlook Business, multiple entities were waiting for an "independent technical root cause analysis" that took weeks to materialize. If your business is paralyzed while waiting for a vendor’s PDF, your incident response process has a dependency bug.

• The Audit: When an outage happens, does your team have the tools to prove exactly where the failure lies? Can they differentiate between a local ISP issue, a cloud provider's internal routing failure, or a physical hardware fault?
• The Action: Invest in tools that provide high-resolution evidence of root causes. The goal is to move from "Google is down" to "Google's Delhi POP is dropping 40% of packets on port 443," which allows for much faster decision-making.

The over-engineering trap: protecting against everything vs protecting against sharing fate

A common reaction to news of a data center fire is to demand "Active-Active Multi-Cloud Everything." However, as discussed on r/sre, this often leads to a complexity monster that causes more outages than it prevents.

The goal isn't to build a system that can survive a nuclear strike. The goal is to avoid shared-fate. You don’t need a secondary cloud in another country to survive a Delhi fire; you just need to ensure your data and your connectivity aren't tied to a single physical battery room.

A one-hour tabletop exercise for your next ops review

To turn these data center fire lessons into action, run this simple tabletop exercise with your infrastructure team:

1. The Scenario: It is 3:00 PM on a Friday. A fire has broken out in the main lithium-ion battery room of your primary data center (or your cloud provider's primary region for your stack). The facility is evacuated and power is cut. It will be offline for at least 14 days.
2. The Question: What is gone forever? (Check your backups).
3. The Question: How do we tell our users? (Check your status page triggers).
4. The Question: How do we route around it? (Check your DNS and POP dependencies).

Why this matters: The shifting nature of site reliability

The delhi data center fire is a stark reminder that infrastructure is still physical. As we move toward more abstracted cloud services, we lose visibility into the physical hardware that actually stores our bits. The transition to high-density lithium-ion batteries in data centers has increased the risk of "catastrophic physical failure" compared to the lead-acid batteries of the past.

Engineering leaders must recognize that disaster recovery single site strategies are a ticking time bomb. The "it's in the cloud" mentality can lead to a dangerous sense of complacency regarding physical blast radii.

Common pitfalls in blast-radius planning

• Trusting "Zones" blindly: In some regions, "Availability Zones" may be physically closer than you think, sometimes even sharing the same utility substation.
• Ignoring the POP: You might have multi-region compute but only a single-region network entry point.
• Lazy Backup Verification: Having a backup is not the same as having a recoverable backup. Matrix Cellular’s data loss fears stem from the inability to physically access or verify the integrity of drives in a damaged building.
• Over-reliance on Vendor RCA: Waiting for a provider to tell you why you are slow is a strategy for failure. You need the telemetry to identify the "where" and "how" of a failure independently.

Sources & further reading

• According to Data Center Dynamics, the fire at STT Delhi 2 caused major damage and was linked to lithium battery units.
• Reuters reported that the damage was so extensive it delayed recovery operations for weeks, impacting multiple high-profile tenants.
• The Google Cloud Status Page documented the persistent network issues in India following the June 5 incident.
• Business Standard highlighted the specific risk of 20 years of data loss for Matrix Cellular, illustrating the "shared-fate" risk of single-site storage.
• Discussions on r/developersIndia and r/sre provide a practitioner’s view on the gap between provider status updates and real-world impact.

Taking control of your incident response

When a major facility fire or region-wide degradation occurs, the difference between a minor headache and a business-ending event is the speed of investigation. Teams that rely solely on a provider's status page are at the mercy of the provider's communication speed—which, as the Delhi fire showed, can lag behind reality by days or weeks.

This is where a shift in operational philosophy becomes necessary. Instead of waiting for an external RCA, engineering teams should aim for high-resolution, self-hosted observability that can pinpoint the root cause of a regression with evidence. When your team can independently verify that a specific network path is failing due to a facility-level event, they can pull the trigger on a failover hours before the official "Incident" notification arrives.

Operate provides this level of self-hosted, evidence-based investigation, allowing SRE teams to identify root causes and draft fixes with full auditability. In an era where a single battery fire can degrade the cloud for a month, the ability to investigate your own infrastructure's silent failures is no longer a luxury—it’s a requirement for survival. By automating the investigation of latency regressions and connectivity gaps, you can ensure that your team stays ahead of the blast radius, rather than being caught in its path.