Case Study
Google Cloud September 2026: every fiber path unplugged at once, and 4h 11m of us-central1 degradation
On 1 September 2026, routine hardware maintenance in Google Cloud's us-central1 (Iowa) region went wrong before any automation could help: a technician replacing optical transceivers was told to disconnect every fiber path at once instead of one router at a time. Within about 13 minutes the zone lost its inter-datacenter network capacity, and 19 products degraded for about 4 hours 11 minutes. Coming just twelve days after the August us-west1 outage, it is the second regional GCP network failure of the summer - and a clean example of why a maintenance runbook that can remove all your redundancy simultaneously is itself a single point of failure.
By Oliver Wakefield-Smith · Published September 2026 · Source: Google Cloud incident report (status.cloud.google.com/incidents/J5ia5t9p3g9Q5Wi7r8Ev).
Incident window
~4h 11m
07:41 to 11:52 PDT, 1 Sep 2026
Scope
us-central1-b
19 products, one zone
Disclosed cost
None
Google published no figure
Timeline
What happened
| Time | Event |
|---|---|
| Before onset, 1 Sep | Routine hardware maintenance is under way in us-central1 (Iowa): a technician is replacing optical transceivers across a set of routers, expected to be non-disruptive because the zone carries multiple redundant fiber paths |
| ~07:41 PDT, 1 Sep | A procedural error issues the complete list of transceiver replacements across all routers at once, without instructions to sequence the work one router at a time; the technician begins disconnecting fiber paths near-simultaneously |
| First ~13 minutes | All affected fiber paths are disconnected within about 13 minutes, collapsing inter-datacenter network capacity in us-central1-b; traffic-flow drop rates reach as high as 100% at the peak |
| During the incident | Virtual machines become unreachable with elevated packet loss; 19 products degrade across us-central1-b and a small fraction of us-central1-f, including compute, databases, analytics, and networking services |
| Recovery | Engineers reconnect the disconnected fiber and restore network capacity, then let traffic and dependent services drain back to normal, so recovery time varies by product |
| ~11:52 PDT, 1 Sep | Incident resolved; Google puts the overall window at about 4 hours 11 minutes (07:41-11:52 US/Pacific) |
Times and the 4-hour-11-minute window are taken from Google's published incident report (tracking ID J5ia5t9p3g9Q5Wi7r8Ev), which gives the window in US/Pacific. Intermediate steps are approximate where Google did not publish a minute-by-minute log; impact duration varied by product as traffic was restored.
Affected Services
Selected products impacted in us-central1
Google lists 19 affected products; the grouping below is representative, not exhaustive. Because the failure was a loss of physical network capacity in the zone rather than a single service bug, the impact spread across compute, databases, analytics, and networking at once.
| Product group | Observed impact |
|---|---|
| Compute Engine / GKE | Instances unreachable and cluster operations degraded in us-central1-b with elevated packet loss |
| Cloud SQL / Cloud Spanner / AlloyDB / Bigtable | Managed database connectivity and operations returned errors and elevated latency in the zone |
| BigQuery / Cloud Dataflow | Analytics queries and data-processing pipelines degraded for the region |
| Cloud Run / App Engine / Apigee | Serverless request handling, deploys, and API management degraded |
| Cloud Filestore / Looker | File storage and BI query workloads in us-central1 were affected |
| VPC / Cloud NAT / Cloud Interconnect / Cloud VPN / Hybrid Connectivity | Network reachability, egress, and hybrid links into the zone dropped as fiber capacity collapsed |
| Google SecOps SOAR | Security orchestration workloads in the region degraded during the incident |
Two vendors publicly acknowledged downstream impact during the window - Cockroach Labs (on its own status page) and cloud-management firm DoiT International - though neither is part of Google's incident report.
Root Cause
A maintenance step that pulled every fiber at once
Per Google's incident report, the disruption began during routine hardware maintenance in us-central1 - specifically, a technician replacing optical transceivers on network routers. Transceiver swaps are meant to be non-disruptive: the zone carries multiple redundant fiber paths, so the plan is that work on one path is absorbed by the others while it is offline.
The failure was in how the work was issued. Google describes it as "a procedural error in the manually orchestrated upgrade process" that caused "the complete list of transceiver replacements across all routers to be issued to the technician without instructions to sequence the work one router at a time." So instead of taking one path down, replacing it, and confirming health before moving on, the technician disconnected all affected fiber paths in the same short window - within about 13 minutes. The redundancy never had a chance to absorb the loss because every redundant path went away almost together.
With inter-datacenter capacity in us-central1-b gone, traffic-flow drop rates reached as high as 100% at the peak. Virtual machines became unreachable, packet loss climbed, and 19 products degraded across the zone (and a small fraction of us-central1-f). Recovery was a matter of physically reconnecting the fiber and restoring capacity, then letting dependent services drain back to normal, which is why impact duration varied by product. The overall window was about 4 hours 11 minutes.
Economic Impact
No disclosed figure, and it depends on where you ran
Google published no cost for this outage, and no independent analyst issued an insured-loss estimate, as CyberCube did for the AWS October 2025 event ($38 million to $581 million). Like the August us-west1 event and unlike the global June 2025 Service Control failure, this was confined to one region, so the honest measure of exposure turns on a single question: did your workloads run in us-central1-b, or did you have live capacity elsewhere?
For a business whose us-central1-dependent revenue runs at $100,000 per day, a total 4-hour-11-minute regional outage is roughly $17,400 of exposure before any failover mitigation. A business that ran active capacity in a second region and could shed us-central1 traffic would have seen a fraction of that. That gap - between the exposure and what a working failover would have cost you - is the entire business case for multi-region.
As with every provider outage, Google's SLA credit is not the number that matters. GCP returns a percentage of the affected service's monthly fee, never a percentage of your revenue, so even a maximum credit is a fraction of the business loss from the same window. See our SLA credit asymmetry analysis, and use the downtime cost calculator to apply your own revenue and architecture to a 4-hour-11-minute outage.
Architectural Lessons
Redundancy a single procedure can remove all at once is not redundancy
The August 2026 us-west1 outage taught that spare capacity is worthless if the automated failover onto it does not fire. September 2026 is the failure one layer earlier: the redundancy was there, the paths existed, but a single maintenance instruction let a technician take them all down together, so there was nothing left to fail over to. The two events, twelve days apart, are the same lesson from opposite ends - the resilience you designed only counts if the operational reality around it, human and automated, cannot defeat it in one move.
The lesson generalises past Google. Redundant links, replicas, and zones are supposed to survive maintenance because you touch them one at a time. When a runbook, a script, or a bulk change can act on all of them at once, they stop being independent and become a single fate-shared unit. Guardrails that enforce sequencing - change tooling that refuses to drain more than one path at a time, health checks between steps, blast-radius limits on bulk operations - are what keep redundancy actually redundant. And none of that protects a customer who lives in one zone: the only defence you control is a failover path in a different region that does not share the failing dependency, tested under load.
For how this event compares with the August us-west1, June 2025 global, AWS us-east-1, and Azure Front Door failures, see the AWS vs Azure vs GCP outage comparison and the full GCP outage history. For the twin regional event twelve days earlier, see the GCP August 2026 case study; for the June 2025 global control-plane failure, see the GCP June 2025 case study. For the cost-benefit math on multi-region and failover, see our business case builder.
Frequently Asked
Common Questions
What caused the Google Cloud outage of 1 September 2026?
How long did the September 2026 GCP outage last?
Which region and services were affected?
How much did it cost?
How was it different from the August 2026 GCP outage?
What is the architectural lesson?
Related