Performance & Infrastructure
June 25 Broke the Internet. Nobody Saw It Coming. That's the Lesson.
Twice this year we have written about traffic events you could see coming. The inauguration with its years of notice. The campaign spikes with their media calendars. June 25 was the other kind.
When the news of Michael Jackson’s death broke on a Thursday afternoon, the surge arrived with no countdown, no capacity meeting, no rehearsal window. Google’s traffic monitoring briefly classified the explosion of queries as a possible automated attack. Twitter spent hours failing and recovering. News sites large and small slowed to a crawl or dropped entirely. For a stretch of the afternoon, a meaningful slice of the web simply performed badly, all at once.
The internet’s biggest unplanned traffic day to date is worth a careful postmortem precisely because ‘unplanned’ is the operative word. It tests a different virtue than the scheduled events do, and most organizations have not built it.
Why the scheduled-event playbook doesn’t cover this
Our January inauguration analysis ended with a checklist: model the peak, rehearse against it, pre-position the fallbacks. Every item on it assumes a date. June 25 had no date.
The organizations that fared well were not the ones with a Jackson contingency plan. Nobody had one. They were the ones whose everyday architecture already assumed surprise. That is the distinction this event teaches. Scheduled peaks reward preparation. Unplanned peaks reward defaults.
When the surge arrives mid-afternoon with zero warning, whatever state your infrastructure is in at that moment is your plan. The question is what state it is in on an ordinary Thursday.
The failure anatomy, briefly
The patterns are familiar ones, compressed by suddenness. Origin servers assembling dynamic pages per request hit the math wall fastest. The same work-per-page math from our caching coverage, at speeds no scaling reflex could chase. Databases saturated under comment storms and update-checking as users refreshed obsessively.
And the interdependence wrinkle showed its teeth. Sites that stayed technically up crawled anyway. Embedded third-party pieces (widgets, ad calls, social buttons) pointed at struggling services and hung page loads while their own hosts drowned. Your Thursday performance is partly a portfolio of other people’s Thursdays. That fact is worth an audit line all its own.
Resilience as a default setting
So what does surprise-ready architecture look like? Nothing exotic. It is the familiar toolkit with the dial turned from ‘special occasion’ to ‘always.’
Aggressive caching as the everyday setting. Full-page caching for anonymous visitors should not be the thing you enable when trouble looms. It should be the standing configuration, because trouble no longer announces itself. The performance gain on ordinary days is the bonus. The survival margin on extraordinary ones is the point.
The CDN already in the path. Edge delivery you activate during a crisis is edge delivery arriving after the crisis started. With utility-priced options now available, per November’s CloudFront piece, ‘already offloaded’ is affordable as a permanent state. Permanent is the state that counts on a Thursday.
Lightweight templates, pre-built and one switch away. The stripped-down page versions that looked so composed on inauguration day earn their keep doubly here. They are only useful in a surprise if they exist before it. Build them in calm. Test the switch quarterly. Hope to never need them, in the way one hopes about fire extinguishers.
A degradation ladder someone can actually climb at 3 p.m. The written order of what switches off, and, critically, who is authorized to act without a meeting. Unplanned events compress decision time to minutes. Authority that requires convening is authority that arrives late. The ladder plus a name is a plan. The ladder alone is a document.
The capacity question, reframed by surprise
One planning consequence deserves its own airing. Unplanned peaks break the capacity-modeling math that scheduled ones permit. January’s method, audience times response rate across a known window, has no inputs when the window is ‘whenever news breaks.’
The substitute discipline is margin-thinking. Know your tested ceiling. That is the load test you have run in calm. You have run one? Know your typical peak, and treat the ratio as your surprise budget. A site cruising at a tenth of its tested capacity can absorb an ordinary surprise. A site routinely near its ceiling is one news cycle from an outage, and should treat that fact as a standing work order, not a fun fact.
The cloud-computing story we have been following all year is relevant here too. Utility infrastructure’s genuine promise is exactly this scenario: capacity summoned in minutes rather than procurement cycles. Our sorting-exercise cautions stand, but the surprise-peak case is the one that will eventually move sober organizations up that adoption curve. When your peak cannot be scheduled, elasticity stops being a keynote word and starts being an insurance category.
The question communications teams should sit with
Here is the uncomfortable planning exercise this event hands every organization, and we mean it as a genuine assignment rather than rhetoric. If your organization became the story tomorrow, for wonderful reasons or terrible ones, would your website survive its fifteen minutes?
The wonderful versions happen. The research finding that goes everywhere. The alumna in the headlines. The viral moment nobody engineered. The terrible versions happen too. That is precisely when your site becomes the source of first resort (the statement, the facts, the response) at the exact hour it is drawing the most load of its existence.
A crisis-communications plan that assumes the website will simply be there has assumed the conclusion. The resilience setup above is what makes the assumption true. We have started raising this in continuity conversations with clients, and the room always goes usefully quiet.
What we changed after June 25
Practicing what we are preaching: this month earns three amendments to the standard build checklist. Page caching moved from ‘recommended’ to ‘default, with documented exceptions.’ The lightweight-template deliverable was added to launch scope for news-prone clients, meaning institutions whose subject matter touches public events. And our maintenance reviews now include a third-party dependency check: what does this page wait for, and what happens when those services have their own bad Thursday?
None of these cost much. All of them price the lesson of an afternoon nobody scheduled.
The scheduled peaks will keep coming, and the January checklist still governs them. But June 25 is the better teacher of the deeper principle. The web’s biggest days increasingly announce themselves after they have started. Architecture that is calm by default is the only kind that is calm when it counts. And calm is a thing you build in advance or do not have.