First Notes on ChatGPT, Written in the Week Everything Got Interesting

We are writing this five days after OpenAI released ChatGPT to the public, fully aware that it will be dated by the time many of you read it, and publishing anyway. We have learned across sixteen years which moments deserve notes written from inside them. The notes written inside the iPhone week. The March 2020 field report. The handful of others where the honest record of what it was like before anyone knew mattered more than the polish of hindsight. This is one. It drafts like a capable junior colleague. Which means everything it writes still needs a senior read.

A free chat window wrapped around a large language model (GPT-3.5, from the transformer family tracked here since BERT) is producing fluent prose drafts, working code, and structured documents on request. In plain back-and-forth chat. For anyone.

It has spread faster than any technology demonstration in our memory. The million-user milestone fell in days. The screenshots are everywhere. The people sending them to us this week include clients, board members, our own staff, and relatives who have never previously forwarded us anything about technology. The hands-on testing justifies the noise, the caveats are as important as the capabilities, and our usual approach applies, with one honest amendment recorded below. Notes first, in the order the week taught them.

What the first week of testing surfaces

Drafting and summarizing help is immediately, unmistakably useful. The first-draft problem, the blank page that costs knowledge workers their hours, is genuinely addressed. Meeting notes structured. Dense documents boiled down well. Workmanlike prose produced from prompts in seconds, tone adjusted on request. The output needs editing, always. It needs it the way a capable junior colleague’s draft needs it, which is a sentence we do not write lightly about software.

The code assistance makes Copilot’s trajectory suddenly steeper. Our engineers’ week of testing produced working functions, useful debugging chat, and solid explanations of unfamiliar code, with the same need for oversight. The implications for development economics go on the watchlist rather than into predictions.

And the hallucination problem appears structural rather than incidental, which is the finding we most wants in writing while the observation is fresh. The system fabricates. Citations that do not exist. Confident factual claims that are wrong. Plausible details invented to complete a pattern. And it does so in exactly the same fluent, authoritative voice as its accurate output. So far as the architecture is understood, the fluency and the fabrication come from the same source. The model produces likely text. Likelihood is not truth. There is no internal signal separating the two for the reader. Every institutional use case this readership will consider inherits that property, and our coverage to come will treat it as the central design constraint, because it is.

The questions forming faster than answers

• Attribution and provenance: the model was trained on the web’s corpus, we presumably included. The output cites nothing by default. The publisher, licensing, and payment questions are forming in real time with no settled framework. Counsel’s discomfort is appropriate.

• Training-data currency: the model’s knowledge has a cutoff and no live awareness. That bounds entire categories of use and is being widely misunderstood in exactly the ways the fluency invites.

• The search question: a system that answers questions directly, conversationally, without a results page, aims at the architecture of the machine-mediated-surfaces question we have tracked since the voice era. The incumbent’s response, the quality dynamics, and the citation economics are now the most consequential open questions in our oldest active file.

• Content economics: the marginal cost of fluent text is collapsing while the marginal value of verified expertise is not. That is either a crisis or a moat, depending on how much credibility an institution has been building. Readers of our E-A-T coverage know our bet.

The protocol, and the honest amendment

The emerging-tech instruments apply, and we are applying them in the open. The sorting question (what job does this do better, for whom, at what cost?) is already producing real answers in the drafting tier. The falsifier question, what would change our assessment, is engaged. And the advice against FOMO-priced adoption stands at full strength, because the vendor gold rush now assembling will make the web3 season look understated.

But the honest note, recorded in the moment because that is the entire function of these notes: this one differs in kind from the recent seasons. The 2017 bots were scripts behind a chat window. Bounded decision trees whose failure mode was the seams showing. Web3 was infrastructure in search of applications.

This is a general capability, language itself, handled competently across domains, whose applications are arriving faster than anyone can catalog them. The precedents for that shape are not the recent seasons but the foundational ones. The browser. The search engine. The smartphone. The technologies that did not add a channel but changed the terrain. We record that assessment now, at five days, with humility and the plainness the moment deserves. We believe this generalizes. We are proceeding accordingly. We are writing it down so you can check us.

What happens next

What proceeding accordingly means, concretely, starting now. Structured testing replaces casual poking this month. Structured trials against real workflows (drafting, summarizing, code assistance) with the supervision costs measured honestly alongside the savings. The governance talk starts now. The policy questions, which we will sequence next month, because the tools are already in the building, per every shadow-adoption pattern we document. And the watchlist reorganizes around the three files where the impact lands first. Content: the economics, the quality enforcement, the credibility moat. Search: the machine-mediated surface question, now urgent. Development: what the helper tools do to the cost of code.

Sixteen years of covering weeks like this one teach a single consistent lesson about weeks like this one. The notes that age best record capabilities and constraints honestly, predict sparingly, and commit to the systematic work while the discourse is still performing certainty in both directions. Those are these.

The week everything got interesting is ending. The years of finding out what it meant begin now. We opened this question with BERT three years ago. It is now fully awake.