What Takes You Offline? Mapping Single Points of Failure

Start with an exercise. Run it before reading on, because producing the list is the actual lesson. Write down every vendor whose failure, alone, takes your website fully offline. Not degraded. Down. Your uptime is the product of every vendor you trust. Most teams have never multiplied that out.

The registrar, whose lapse or compromise takes the domain itself. The DNS provider, since no resolution means no site, no matter how healthy your servers are. The CDN, if it fronts everything. The host or cloud region. The TLS certificate chain, since an expired certificate is an outage with a timestamp you set years earlier. And the entry most lists miss until prompted: that one third-party script loading synchronously in the head, whose vendor’s bad afternoon becomes your blank page.

Most teams produce this list slowly, with visible surprise at entries they had never registered as dependencies. The slowness is the finding. The average institutional website’s full-outage surface has never been listed by anyone currently employed there. The recent outage seasons (the DNS incidents, the CDN wobbles, the cloud region events) have made fragility a live topic. This post is the practical version: the map, the cheap wins, and the framing that keeps the subject sane.

Why the budget chases the wrong risks

Our infrastructure reviews keep finding the same pattern. Resilience spending piles up on the dramatic scenarios: multi-region failover, disaster-recovery contracts, the binder on the shelf. The cheap, high-probability exposures go unfixed, because drama is fundable and hygiene is not.

The postmortem record of the past two years reads the opposite way. The incidents that actually took sites down were mostly the mundane tier. The DNS mistake made worse by a day-long TTL. The certificate that expired on a subdomain nobody watched. The single synchronous tag from a vendor having an incident. The CDN config change that propagated wrong.

Each of these has a fix measured in hours and dollars, not quarters and capital budgets. The discipline is simply matching spend to the real exposure: map first, fund the cheap wins, and only then have the multi-region talk, which most sites do not actually need. This is insurance thinking, not doomsday thinking. You cannot buy zero risk at any price. But you can know your exposures, price your tolerance honestly, and buy down the biggest risks with the smallest checks first.

The high-yield list, in order of value

• DNS TTLs, set on purpose: the time-to-live values on your records decide how fast you can move when something fails. Repointing to a backup host is useless if the world caches your old records for a day. Audit them this week, set them as decisions rather than defaults, and write down the reasoning. Cost: an afternoon. Coverage: every failover you will ever run.

• Secondary DNS: a second provider serving the same zone turns the single worst failure point, resolution itself, into a survivable event. The major incidents of recent seasons made the case. The annual cost is a rounding error against the exposure.

• The static failover page, built before it is needed: a maintained page, hosted somewhere else, saying what happened and where to go. DNS or CDN config can promote it in minutes. The difference between an outage and a crisis is often whether the organization can still say something, somewhere, calmly.

• Third-party scripts, loaded async or deferred: the synchronous head tag is the classic borrowed outage. The tag census plus a loading-attribute pass turns a vendor incident from your outage into your telemetry gap. The performance budget collects the same fix as a speed win.

• Certificate monitoring, automated: expiry alerts on every certificate the census knows about. The calendar outage is the most preventable kind on record and still among the most common.

Running the map as a living document

The workshop’s output deserves upkeep, because a map that is not maintained is a snapshot that lies more each quarter. The format that works: one page per full-outage dependency. Vendor, what fails when it fails, blast radius. How you detect it, the response play, and an owner.

Review it on the same quarterly calendar as the security and dependency audits. Update it when vendors change. Rehearse it once a year with a tabletop hour: pick one dependency, declare it failed at 9 a.m. on your busiest Tuesday, and walk the room through detection, decision, and response with the map open.

The rehearsal always surfaces the same two findings. A response play that assumed a person who no longer works there. And a detection gap where the team would learn of the failure from a stakeholder’s phone call rather than its own monitoring. That second finding is what funds the monitoring. None of this is sophisticated, and that is the point. Resilience at this scale is mostly listing, hygiene, and rehearsal, because that keeps being what actually works.

Against the binder

A word about the disaster-recovery binder, since the workshop conversation always reaches it. The binder, the formal DR document produced for an audit or a board, fails in practice for a predictable reason. It documents everything at equal depth, so nobody reads it, updates it, or can find the right page during a real incident.

The one-page-per-dependency map beats it not by being more thorough but by being used. Short enough to review quarterly. Specific enough to act from. Owned line by line by people who know they are named in it. If the audit requires the binder, generate the binder from the map, not the reverse.

And if this month’s exercise produces a list with surprises on it, and it will, take the surprises as the cheap gift they are. Every dependency you find in a workshop is one you will not find on a status page during your admissions deadline or your giving day. An afternoon of mapping. A page per exposure. A rehearsal a year. The sites that stay up are not the ones that bought zero risk. They are the ones that knew where they stood and had decided in advance what to do about it.