Eleven Lines of Code Broke the Internet: Dependency Lessons from left-pad

On March 22, a developer named Azer Koçulu unpublished all of his modules from npm. He was aggrieved by a trademark dispute over one of his packages and by the registry’s handling of it. npm is, the package repository at the center of modern JavaScript development.

Among the more than two hundred removed packages was one called left-pad. Eleven lines of code that pad the left side of a string with spaces or zeroes. Your application stands on a tree of other people’s decisions. Know the tree.

Within hours, build systems began failing worldwide. Babel, the compiler a substantial fraction of modern JavaScript flows through, depended on it, transitively. Projects at every scale, React applications included, stopped building. The registry took the extraordinary step of un-unpublishing the package to stanch the damage, igniting a governance debate that will run for months. The industry spent a day trading jokes about the web being held up by eleven lines nobody had read.

The jokes were earned. The implications deserve longer than a day, because left-pad is the cheapest possible tuition for a lesson we have been assembling since the Heartbleed spring. This post is the paid-in-full version for the people who commission software rather than write it.

What the incident actually revealed

The mechanics generalize far past one package manager. Modern applications are assembled, not written. A project declares a few dozen direct dependencies. Each of those declares its own. The resolved result is a tree of hundreds or thousands of nested dependencies: packages your packages need. No human on the team selected or reviewed them, and in most cases nobody knows they exist.

The tree’s properties become your application’s properties. Its availability: left-pad’s lesson, a node can vanish. Its security: a compromised node ships inside your build. Its licensing: obligations propagate whether or not anyone read them. And its maintenance health: unmaintained nodes age into all three prior problems.

None of this is an argument against assembly. The economics of shared components built the modern web, our stack included. The alternative (writing everything) fails worse. It is an argument that the tree is infrastructure and has never been managed as such.

The week’s failures sorted the industry cleanly into teams that could answer ‘what’s in our build?’ and teams discovering the question. The sorting question is now permanent, and it has a client-facing form we will get to.

The hygiene, now standard on our builds

• Lockfiles, committed. The dependency tree resolved and pinned to exact versions, recorded in version control, so builds are reproducible and yesterday’s working build cannot silently become today’s different one. This was best practice before March 22. It is table stakes after.

• Dependency inventories and audits, scheduled. The tree listed (the tooling makes this trivial) and reviewed on a schedule for abandoned packages, known holes, and license surprises. The same site-inventory habit our governance series applies to sites and Heartbleed applied to libraries. You cannot manage what you have not listed, at any layer.

• Private mirrors for the mission-critical. Builds that matter should not depend on a public registry’s availability or a stranger’s continued goodwill. A caching proxy or private mirror holds copies of everything your builds consume, converting registry incidents from outages into news items. An afternoon of setup. The whole insurance category.

• Proportionality at the door. The intake question our tag-governance piece installed for third-party scripts applies to packages word for word. Does this dependency’s value justify its place in the tree? Eleven-line utilities are precisely the case where writing it yourself beats adopting a stranger’s node. The culture that reaches for a package by reflex is the culture that owns a thousand-node tree by accident.

A worked example: a typical tree

To make the abstraction concrete, consider a representative project. A mid-sized institutional site with a modern front-end build.

Direct dependencies declared deliberately: 31, each one a considered choice. Resolved transitive tree: just over 900 packages.

Of those 900, our post-left-pad audit found a familiar spread. The large majority actively maintained and unremarkable. A few dozen not updated in over a year, including two in moderately critical positions. A handful of single-maintainer utilities of the left-pad species sitting under popular tooling. One license surprise: a copyleft-licensed package deep in the build toolchain, consequential for some clients and not others, now flagged in the inventory either way. And zero packages anyone on the team could have listed from memory before the tool printed them.

The fixes were proportionate rather than heroic. Three trivial utilities replaced with a dozen lines of our own code, under the proportionality rule. The stale critical packages scheduled for migration to maintained alternatives. The license question documented for client counsel. The mirror stood up in an afternoon, as promised.

Total effort: roughly two developer-days, most of it judgment rather than typing. That is the entire practice at working scale. Not paranoia, not paralysis. A census and a short list of decisions, repeated on a calendar. The tree did not get smaller. It got known, which was the deficit all along.

The client-facing question

For the readers who commission rather than build, the incident writes one due-diligence question worth adding to every vendor conversation, ours included: what is in your build, and what happens when a piece of it disappears?

The answer you want has the shape of the list above. Pinned, inventoried, mirrored, proportionate. The answer that should worry you is any version of ‘that won’t happen,’ because March 22 is now the standing citation that it does.

The governance debate the registry’s action opened matters, and we will track it. Who may unpublish? What does the commons owe its users? How do naming disputes resolve? But its outcome changes the local lesson not at all. Your applications stand on a tree of other people’s decisions. The tree can be managed or merely inherited. And the week the industry laughed at eleven lines was the week managing it became everyone’s job description.

Cheap tuition, as promised. The next incident at this layer will not be about string padding, and the teams that treated this one as the drill will read about that one the way our patched clients read about Drupalgeddon. As news about other people’s week. Enroll now. The course fee only goes up.