How One Bad Config Push Erased Facebook From the Internet

Last Monday’s outage is still worth picking apart, because it’s a genuinely great case study in how fragile the internet’s plumbing can be. Facebook, Instagram, and WhatsApp all went dark globally for roughly six hours — the worst outage the company has had in years — and the root cause wasn’t a hack or a hardware fire. It was a routine-sounding configuration change to Facebook’s own backbone routers.

Here’s the short version of what happened. Facebook relies on BGP (Border Gateway Protocol) to announce to the rest of the internet which IP address ranges it owns and how to reach them. A maintenance command intended to assess the capacity of the global backbone network apparently had a bug that took down the entire connection, which in turn caused Facebook’s routers to withdraw those BGP announcements. Once the routes were withdrawn, the rest of the internet simply had no path to Facebook’s servers anymore. It’s the digital equivalent of a company ripping its own street address off every map on Earth at once.

That withdrawal had a nasty side effect: it also killed Facebook’s authoritative DNS servers, because those DNS servers were only reachable via the same routes that just vanished. So even if some fragment of the network had a cached path to Facebook’s infrastructure, DNS lookups for facebook.com, instagram.com, and whatsapp.com started failing outright. That’s why people weren’t just seeing error pages — the domains looked like they didn’t exist at all.

The part that made it worse

What’s stuck with me most is the reporting that Facebook’s own employees got locked out of internal building-access systems during the outage, because those systems apparently depended on the same internal network that had just evaporated. When your outage response team can’t badge into the building where the servers are, you’ve discovered a special kind of single point of failure. It’s a reminder that “internal tools” and “internet-facing services” are often more entangled than anyone realizes until the entangling becomes the story.

There’s a broader lesson here that goes beyond Facebook. BGP is, by design, a trust-based protocol — routers announce routes and their peers largely believe them. That’s part of why BGP hijacks and leaks (accidental or malicious) have been a recurring theme in internet security for years. Most of the time the blast radius is contained to a region or an ISP. This time, because of how centralized and self-contained Facebook’s infrastructure is, a single internal mistake cascaded into a global blackout of some of the most-used apps on the planet.

If there’s a silver lining, it’s that outages like this are useful forcing functions. Expect to hear more from Facebook about DNS resilience, out-of-band emergency access, and safer rollout procedures for backbone changes. Six hours offline for a company this size is expensive and embarrassing enough that I’d be surprised if the postmortem doesn’t produce some real engineering changes.