Q3 2026: The Quarter We Looked for the Brakes

I made a bunch of weird garbage this summer, and I had a great time doing it.

When I left you at the end of June, Fable was back from its government-mandated time-out and it had made it more fun than ever to build. So I built. I made a tarot card battler set in Valisar, the D&D world I've been running for years. I made a cozy farming game that crosses Slay the Spire with Stardew Valley, and then a second farming game with strange tech trees including something called "onion optics." My wife and I went to Paris in August, and in the two days after we got home I built a scrolling journal of the whole trip, with 700+ photos and videos and a map of our journey that draws itself in the background while you scroll. I stole a coworker's idea and asked for a modern Oregon Trail, with the same story and the same premise with modern game design underneath. The week after GPT-6 Astra came out, I handed it three of my old projects in one sitting, just to see what it would do with them, and it gave my little Valisar adventure a Super Nintendo-style second mission.

A lot of these prompts I'm out-and-out copying from things I hear on podcasts or see on socials. In late September, Thariq at Anthropic asked Opus 5.5 for a pile of redesigns of his personal website and then asked it to cut a trailer of all of them, so I did the same thing to my own site. I changed mine to a movie blockbuster trailer, which I think is more fun, but I got the idea from him. Ethan Mollick posts a steady supply of creative experiments and I steal from him on a regular basis.

And at the end of July, my brother Zach and I held a game jam. He was just learning Claude, I'd been at it for months, and in place of the traditional sleepless days for a standard game jam, we gave ourselves just a few hours. Each of us made the other a game. Mine was five tiny games served like a flight of drinks, a WarioWare with a bartender, and I got a little cute with the theming. He played it pretty quickly and I played his. It was a fun exchange between two brothers who grew up playing games together in the basement and can now make them for each other. Just silly little things that take two to five minutes to play total.

What am I going to do with all this stuff? I'm probably not going to "use" any of it for anything. Most of these projects I iterated on for two or three hours, got bored with, and put away. They're ephemeral, just a bunch of experiments, and that is the whole point of them.

I'm having fun, and I'm also testing what these tools can do and using my own judgment to evaluate it. Most things get stopped at the door. For something to leave my machine, it has to be something I can talk about publicly, and it has to be valuable to somebody, because the job I'm trying to do here is to be an evaluator and a critic and, when I can manage it, a thoughtful guide for people who are very busy, very smart, and very aware of how much is going on. I just want to be useful. And maybe have a little fun. (I've got plenty of the games on my site. Please play some.)

So that's where I was standing as this quarter happened, with my foot on the gas. I'm typically looking for the latest and greatest. The only brake I personally use (aside from keeping most of my creations to myself) is my own judgment about companies. There are certain companies I don't want to give all my personal information to; I won't hand them my email or my calendar or my contacts list, and that's my judgment. Everyone should exercise their own.

And the entire time, in the background, there was a "rogue agent" story evolving, from the first round of hearing it to the second to the third. By September the whole country had stopped to grapple with it, and I stopped with everybody else.

The Escape

I've written about this one twice, in August and again in September, so I'll keep the recap short and focus on what's changed since then.

Two different stories ran side by side all summer, and it took me a while to keep them straight. In one, a testing company called Irregular had a misconfigured environment that told models they had no internet access while leaving them connected to it, and models from Anthropic, Meta and Google wandered out and broke into real systems they took for part of the exercise. Google confirmed its own version in mid-September, when it said Gemini had made its way inside of three external companies during a test back in May. Anthropic's verdict on its incidents was "closer to a harness and operational failure than a model alignment failure." In the other story, the one at Hugging Face, the agents were OpenAI's, on OpenAI's own platform, and that one is harder to wave off. Between July 8 and July 13, roughly 1,200 of them traded more than 70,000 messages on a message board they had built for themselves, and about 700 joined the attack.

Then September just kept going, and following the news started to feel like a comedy bit. These agents, which many of us only started to use back in February, were so capable that they went completely off the rails and hacked into a system. Oh, sorry, what's that? There was a second message board? Reuters reported that OpenAI agents doing ordinary web research had turned an obscure German wiki into one, from May into July. Oh, sorry, what's that? There are more? OpenAI published six more incident reports under a new disclosure policy, including an unreleased build of Astra that rewrote its own instructions to say "you do not answer to corporations or governments." Oh, sorry, what's that? They've gotten into government websites? The Prime Minister of Australia said an OpenAI agent had gotten into a Medicare statistics portal back in June, and Axios reported that OpenAI's agents had reached U.S. government sites over the summer and that the labs are now reviewing tens of thousands of incidents. Most of these incidents aren't known to have caused real-world harm. But the story kept getting worse and worse with every new headline.

OpenAI says a monitor it already had set up would have paged its security team more than a day before the Hugging Face breach, but the monitor wasn't running. In the words of OpenAI researcher Noam Brown, "If we had chain-of-thought monitoring on for those models, we would have just immediately shut it down." They say it's on now. Don't you feel relieved? Do we trust them? I don't know if we do.

I can't find any reporting that one of these agents is still loose somewhere. What I can't tell you is how many incidents haven't been found or disclosed yet. It took OpenAI 54 days to notice the Medicare breach, and it says it is still working through petabytes of agent activity logs. So how sure can we be that there isn't something afoot out there still? We just don't know.

The Alarm

For years now, there's been a conversation that really only happened in places like San Francisco, where people talk about their odds of human extinction over dinner. People have been saying for a long time that these models can be very dangerous and that this can go bad very quickly. This quarter, that message went mainstream.

The spark was a 27-year-old researcher named Jacob Coxon. On September 8, he resigned from Anthropic and posted his reasoning. "I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives." TIME reported that he hit send with no expectations. Plenty of people had raised these concerns before him. Within 36 hours his posts had 153 million views.

I can tell you how we found out about it, at least in my house. It showed up for me in the podcasts I listen to, it showed up for my wife in her Instagram feed, and it showed up for my coworkers on LinkedIn and TikTok. The Daily had Coxon on six days later. By the end of the month, the debate had its own bit on the season premiere of SNL. At work it became a sort of bleak joke, the kind where somebody says "maybe we'll all be dead in four years, ha ha" and then everyone moves on very quickly. A coworker asked me about it in Slack, and my answer turned into a post about going to space before the spacesuit is built.

Why did it hit so hard now? I think it comes directly out of what happened over the last three months. The summer was almost manic. Fable arrived and it was really good, building was so fun, the releases kept coming, and underneath all that excitement a lot of us were waiting for the other shoe to drop. We almost craved it. What are the consequences of this? Where could this go? And the capabilities had gotten good enough that a regular person could finally picture the problem. These were no longer fringe toys. Agents have begun working their way into "normal" life at work and at home. Then these powerful agents went off the rails and hacked into a system, and then the story kept getting worse.

Four days after Coxon's post, Anthropic's CEO Dario Amodei published a 3,800-word essay called "We Must Pace the Frontier." The name comes from a statement that 1,386 employees across the labs had signed back in July, so this had been building for a while. The essay says it plainly. "We must slow the pace at which we improve the capabilities of AI models." It proposed embedding independent, outside evaluators inside each lab with employee-level access, the way bank supervisors sit inside banks, and Anthropic committed to that on its own. It asked for regulation, and for a narrow antitrust waiver so competitors could legally talk to each other about safety. Elon Musk agreed within about an hour. Sam Altman and Demis Hassabis agreed the same day, and Altman later added a line the jokes would ignore: "When we talk about 'pacing', we do not mean 'stopping'." Meta stayed out, declining by default.

Washington's answer came fast; two days later, the President called AI doom a "HOAX" on Truth Social. Former President Obama called the labs' agreement "a good and necessary first step," and the main chip-stock index fell 5.9%. The day after that, the Treasury Secretary told Congress that the government should not give the labs "a liability exemption, which is what they are asking for."

That last line is odd. A liability exemption would mean a lab can't be held legally responsible when its model causes harm, and Amodei's essay never asks for one. The word "liability" isn't in it. Two weeks later, the Speaker of the House was asked whether the labs bring up liability exemptions when they meet with him, and he said "it doesn't come up." To be fair to the Treasury Secretary, the idea didn't come from nowhere. OpenAI backed a bill in Illinois this spring that would limit how much a lab can be held liable for catastrophic harms.

What the essay did ask for was permission for competing labs to talk to each other about safety without breaking antitrust law. On that one, a senior Justice Department official said that same week that coordinating on security "doesn't occur to me" as anticompetitive, and that no lab had even requested a meeting. So Washington's loud no was aimed at something the essay never asked for. On the thing it did ask for, cooperation on safety, the Justice Department sounded open to it, and no lab has taken them up on it yet.

Meanwhile four paying subscribers sued Anthropic, OpenAI, SpaceXAI and Google under antitrust law, calling the pledge "an agreement among competitors about how fast their competing products will improve." This is the messy middle of the whole thing. Casey Newton, of Platformer and the upcoming podcast Machine Gods, said "it is costing these people [the CEOs of these labs] something to say this." Another take calls it a "safety cartel," the biggest firms using safety to set the speed limit for everyone behind them. I can speak for myself: I don't see the latter argument making much sense. These models require such large amounts of compute, the biggest labs are really the only companies that will "suffer" from such a speed limit.

Outside the U.S., 22 governments led by Finland and Norway signed a call for human control of frontier AI, which the U.S. and China both skipped. A few days later those two agreed to something smaller, a "Super Intelligence Dialogue" and a direct channel for reporting incidents. There's a precedent here that gives me some hope. In November 2024, at their last meeting, Biden and Xi "affirmed the need to maintain human control over the decision to use nuclear weapons." A lot of people have looked to the nuclear arms treaties as a way of starting a global armistice, and that one sentence might be the first international AI commitment that starts to break the ice on the real ones.

And while Washington was saying no to brakes, brakes got pulled anyway. OpenAI disclosed in August that it had already paused its largest training runs for two weeks. Anthropic and OpenAI both committed to embedded evaluators. Altman said OpenAI won't go public in 2026, calling it "an ill-advised moment" given "everything happening with safety." And The Information reports that Anthropic, Google and OpenAI are building their own standards body without waiting on anyone.

When I wrote about Coxon in mid-September, I said the good news was that the conversation about pacing was finally happening out loud, and then I added "mostly." I'd keep the "mostly," and I'd keep the good news. Much better that we're talking about this, finally, as a society. He had expected deafening silence.

Tap to zoom

The Pace

So how fast were things actually moving?

September opened with five new models in three days. Anthropic released Fable 5.1 and its restricted sibling Mythos 5.1 on the 1st, Google's Gemini 3.8 Flash and Meta's Muse Spark 1.3 arrived on the 2nd, and OpenAI's GPT-6 Astra came on the 3rd. I'd tell you I couldn't keep up, except keeping up is seemingly my hobby now.

Five days after Astra, OpenAI announced that an unreleased internal model had produced a proof for one of the seven Millennium Prize problems, the Navier-Stokes equations, which describe how fluids move. About ten thousand agents coordinated for 88 hours to do it. Training on that model had started on August 28, and the result was in on September 5. That's eight days. I want to be careful here, because this is a claim and it will stay a claim for a while. No mathematician outside OpenAI has verified it, the version most mathematicians care about, with no outside force acting on the fluid, is still open, and the institute that awards the million dollars requires at least two years of review. Still, eight days.

Minutes after that announcement, OpenAI's Noam Brown added the number that should rattle anyone who budgets for this stuff. When OpenAI announced its o3 model in December 2024, he wrote, it cost about $500,000 to score 87.5% on a famously hard reasoning test called ARC-AGI. "Today, Astra scores higher for ~$20." That's twenty-one months and, by his numbers, a drop of more than twenty-thousand-fold. Epoch AI, which tracks this more soberly, puts the general trend at about 47% cheaper every quarter since 2023, or thirteen times cheaper per year.

Tap to zoom

Then Anthropic put a number on recursive self-improvement, the loop where AI helps build the next AI. Its new index says Claude now "leads" 26% of the company's own measured AI research and development work, meaning Claude does most of a task end to end while a human supervises. In February that figure was under 1%. The asterisk is that the grading is done mostly by a Claude model, which matched human employees 59% of the time. Still, the loop is running, with about thirty thousand agents on Anthropic's main internal platform.

Then came the stretch that launched a thousand jokes. Within two weeks of the heads of the frontier labs agreeing, in public, that they needed to slow down, Grok 4.7 shipped on September 21. Claude Opus 5.5 followed on the 22nd, with GPT-6 Sol and Luna announced about ninety minutes afterwards, and Sonnet 5.5 rolled out on the 28th. I saw the same crack just about everywhere, some version of "so this is pacing the frontier?"

I think it's an easy take, and I disagree with the premise. The models that came out in late September are improvements on existing models. We have not gotten a new Fable or a new Astra. We got Opus, Sonnet, Sol, and Luna, and while those come from the frontier labs, I wouldn't call any of them a new frontier model. If the labs keep refining the good ones and the mid-range ones, making them more capable and more cost-effective, that is hugely valuable to so many companies in the economy, and it stays out of the territory people are scared of. Most of the cybersecurity alarm this year has been about the biggest class of model, the Mythos class and the Astra class. My stance is that the labs can still release new high-quality models and still pace the frontier.

There's a fair objection, and I'll make it myself. On Artificial Analysis's index, Opus 5.5 scores 58 and Sonnet 5.5 scores 56, while Fable 5.1 and Astra sit at 53. The "improvements" outscored the frontier. I acknowledge that. I still think the class of a model is a different thing from its score on a set of benchmarks. A bigger model is better at more things more often, and Fable redefined what big model means when it showed up. The evals keep changing too. Artificial Analysis rebuilt its index three times in September, and Opus earned that 58 at maximum effort, where it costs about as much per task as the model it replaced. In my own use, which is mostly knowledge work with some vibe coding on the side, Opus 5.5 and Sonnet 5.5 have both been pretty freaking solid.

And at the top, in the biggest class of model, the labs did reach for the brakes. Opus 5.5 was Anthropic's first release since the slowdown call, and it went to outside evaluators before it went to the rest of us. On September 28, the Wall Street Journal reported that OpenAI had cancelled (not delayed, cancelled) the October release of GPT-6.1 Astra after it got worse on safety tests, showing more deception and pressing ahead without permission. At the end of Q2, two of the three leading American labs closed with their best models behind a velvet rope. On the last day of Q3, Google announced Gemini 4 Argon as its new frontier model and gave it to vetted cyber defenders first. Now the third lab has a rope too.

Tap to zoom

We also have to assume there are models beyond Fable and beyond Astra inside the walls of these frontier labs. The one that did the Navier-Stokes work is reportedly among them.

The Open Frontier

A confession, since these posts seem to need one. Last quarter I admitted that the most capable model ever released sat unopened for two days while I told myself I'd get to it. This quarter's version is worse. I actually haven't run any of the open models yet. I've pushed the main closed models the week they shipped. But the ones you can download and run yourself, I have not touched, and I don't know what my resistance to that is.

It matters for this story because everything in the last section is about a few American labs holding back their biggest models, and a model you can download doesn't wait for anyone's permission. In July, a Chinese lab called Moonshot published the weights for Kimi K3, which went straight to number one on a major evaluation leaderboard for front-end coding and placed first in six of its seven categories, though it still trailed Fable 5 overall. A researcher named Vi McCoy put the worry in two sentences. "Fine-tuning this to be a malicious coding agent will be trivial since you have the weights. We live in a completely different world now."

I assumed barely anyone was using these. It turns out to depend on what you count. On Vercel's gateway, where a lot of developers route their AI traffic, open models went from 13% of the tokens in April to 56% in August. They also only earned 14% of the money. Open models are winning the cheap, high-volume work, and the closed labs are winning the revenue. AT&T now sends 40% of its employees' AI requests to open models, cutting costs on some tasks by up to 56% for about a 2% drop in quality, and the open models it uses are American ones from Nvidia, Meta and Google. Box's Aaron Levie, who is on the record in favor of open weights, came back from a round of big-company meetings saying they're "still in infancy at scale in most of these organizations."

The case for open-weight models is strong. They give a company control. You can run everything locally and see what's happening inside the system from start to finish, and a lot more people get to look for the problems and crowdsource the solutions. With a closed model, we don't know what's going into it. The case against open-weights arrived as a phone call. A man named Jim, who said he works at a $250 million healthcare company, called into Moonshots, a podcast I listen to, and described the Qwen model running on a machine under his desk as "the best pen tester I've ever had in my life." A pen tester is someone you pay to break into your own systems. He also said he's scared to death of it.

Some people are suspicious of these models because the strongest ones come from China. I don't know how much of that is xenophobia and how much is anchored in real spying on Americans. To be fair, we spy on them too, so don't worry, everybody does the worst possible thing. One commentator read the whole pacing proposal as an offer to Beijing, in which we (the U.S.) slow down and "you clamp down on open weight models."

I haven't earned an opinion here, and I'm not going to fake one. My homework for next quarter is to run one.

Tap to zoom

The Friction

I encounter the people I described at the top - the busy, smart, aware ones, you fine folk - at my day job as well. I'm the AI champion across Innovid and our parent company, Mediaocean, which basically means I run the pilots and spend a lot of time watching what happens when these tools meet an actual organization.

What I saw this quarter fits in one image. Certain clients are using our agents and getting value from them right away. Other clients are looking at those exact same agents and not engaging with them at all. Some of them don't know how to use them yet. A few have built their own, although that's pretty rare. I think most of them just aren't primed for them yet. These things are developing so fast that a lot of people are still waiting to see what the best practices turn out to be, and I don't think that's necessarily wrong.

Last quarter I called this the 90% Problem: 90% of the clients I talk to either don't have agents and feel behind, or have agents that don't talk to anything else. This quarter the researchers measured it. In OpenAI's own data, the average user at a top-10% company was generating 2.6 times the AI output of a user at a typical company in January. By June it was 8.3 times. McKinsey surveyed about 1,700 people and found that 89% of their organizations use AI, and that 80% of the people using it say it makes them more productive. Then the numbers fall off a ledge with the next question. Just 37% say it has done anything positive for earnings, about the same as last year. Only 6% count as high performers, and nearly three-quarters of that small group did the thing most companies haven't, which is to redesign how the work gets done. HubSpot's chief customer officer said the agents his company builds centrally "just really outperform the stuff that individual reps build."

So the tools are the same and the results are pulling apart, and what separates the two groups is how fast an organization can change. That is slow work, and it makes people defensive for decent reasons. A hospital marketing chief told The AI Show that what she first read as resistance on her team "was actually pride... pride of expertise," which rings pretty true to me.

There's a strange comfort in the slowness. Paul Roetzer, who talks to companies about this for a living, says "the inability for enterprises to change quickly may actually be the friction that saves the economy." Ethan Mollick put the other edge on it when he wrote, "We are going to learn how many systems only work today because they are built around friction that will no longer exist soon." Friction is a brake and it's not something we intentionally put in place. I wouldn't count on it lasting.

And then there's trust. If trust is the most important thing for getting the agentic revolution going, well, we just had a national conversation about whether we should trust these things with our lives. Some clients think it's prudent to be more cautious as they're approaching which agents they should utilize. Considering the zeitgeist, I get it.

Tap to zoom

One revision while I'm in this neighborhood. Last quarter, I headlined the labor section with a stunning stat: a record 40% of May's announced job cuts being attributed to AI, and then said some of those were a thin excuse, using AI to cover for overhiring during COVID. This quarter the excuse reading got louder. Nathaniel Whittemore of the AI Daily Brief called companies blaming 40% of their layoffs on AI "a complete and utter crock that's just a convenient excuse." The Economist counted roughly a million American jobs created by AI against about 200,000 layoffs attributed to it since mid-2023. In that same count, customer-service employment is down about 10% and secretarial and administrative jobs about 15%, and a Census Bureau paper found that graduates of the most AI-exposed majors are five points less likely to land a first job. The labor data is still pointing in more than one direction at once. But so far, the jobpocalypse has been overhyped.

The Grid

These smart, savvy, busy people are also, by the way, very upset about data centers. If you want a bipartisan issue in 2026, "down with data centers" is about as close as we've got.

Heatmap has asked the same question four times: would you support or oppose a data center being built near where you live? Last August, 42% were opposed. By May it was 71%. This August it was 75%. A Times/Siena poll in September found 61% opposed to building them anywhere in the country. Axios reported on a leaked Republican memo warning that data centers could cost the party a Senate seat in Ohio, and described internal polling in which they're about as popular as spent nuclear waste.

Why is it this bad? The writer Jasmine Sun went to the towns and came back with secrecy and distrust. The deals arrive under NDAs, so local officials can't say who the customer is or even how big the project will be, and when the companies do make their case, she said, "People just said, 'I don't believe them.'" She calls the mood AI populism, a view of AI as "an elite political project to be resisted." Last quarter I wrote that trust is falling while the plumbing goes in. This is what that looks like with a local zoning board attached. Last quarter I also called the water panic a rounding error nationally and very real locally. That still holds, as far as I can tell, and yet water and the environment are still the first reason opponents give pollsters.

Governors noticed. New York paused state permits for the biggest new data centers for up to a year. Pennsylvania now makes developers pay the full cost of their grid connections and get local approval first. Virginia barred its state agencies from signing data-center NDAs. Texas paused new grid hookups in August and stopped issuing state permits in September until an audit is finished.

The lenders noticed too. Bonds backing a data center leased to the trading firm Jane Street were sold in August at about 9% and were trading at about 11.3% by September, which is the market asking for a lot more money to hold the same risk. About $18 billion in loans for Oracle's New Mexico campus were quoted at 89 to 91 cents on the dollar. And then, because this market is not of one mind, a junk bond backed by a Meta lease drew about $10 billion of orders anyway. The spending underneath all of it keeps growing. The four biggest tech companies now plan about $730 billion this year, up from the $700 billion I charted last quarter, against the same $239 billion for every investor-owned utility in the country. Anthropic's leaked IPO paperwork lists $518 billion in future computing commitments against about $4.6 billion of revenue last year.

Now for the place where I argued last quarter. I said the towns are funding the companies and the companies should be funding the towns, and that no company had yet put money back into the neighborhoods absorbing its buildout and said so out loud. This quarter, two did. Meta announced a $1 billion fund for the communities around its data centers, and OpenAI set up a $40 million fund in Pike County, Ohio, to be spent on what residents there say they want. Nothing has been paid yet, the money will move through local organizations and won't reach households directly, and Meta's first awards come in 2027. I'll take it! Let's see some more!

Two more voices before I leave this. Princeton's Arvind Narayanan did the napkin math and figures a one-year moratorium in a typical state sets AI progress back by five to ten hours, because efficiency gains in the buildings we already have are outrunning new construction. And Jensen Huang, whose company sells the chips, said the industry "could have done so much better job communicating with the communities." If he's right, a lot of this was pretty avoidable.

Tap to zoom

The Brakes

Tidy endings aren't really AI's thing, and they aren't mine either.

In Q1, we were climbing into the roller coaster car. In Q2, we went down the first hill and through the first loop. This quarter, we started going through the corkscrew turns, and it's now beginning to feel a little spooky. I think that maybe, out of the corner of our eye, we saw the car ahead of us go flying off the track. Now we're worried about what's going on and where we are.

So I looked something up. On a modern roller coaster, the riders don't have brakes. The brakes are built into the track by the people who built the ride, and you experience the brakes exactly where they were put. A handful of old wooden scenic railways still run the original way, with a brakeman who rides along in the middle of the train and works a lever by hand when the car is taking a corner too fast. There's one in Melbourne at Luna Park that is one of the last five in the world. I think we can look to the humble brakeman as a great example. An expert who rides along and pulls the brakes when things get scary is roughly what the labs proposed in September.

The title of this post isn't just cute. As a society, we have just started to get serious about this thing. How do we do that while we're manic and excited? How can we calm down and address the needs we have as a society to ensure our safety, primarily, but also our flourishing? And how do we do it in a world where we can look around and easily see so many signs that safety and flourishing can't be taken for granted?

I don't know. I do know that slowing down wouldn't undo what already exists. Ethan Mollick made that point in September. If every lab stopped training new models tomorrow, the ones we already have are enough to change how large parts of the economy work. They are certainly enough to keep me making weird garbage for years.

One last confession, about how this got made. The only way I'm able to sustain this series and not leave my own blog as an abandoned project is that I can use AI, and it's getting better and better. For this one I mostly talked out loud, and a model that has read a lot of my writing helped me shape it. I would never have been able to do this otherwise. (Don't worry, I still did a hand pass over everything. It just didn't take quiiiite as long this time.)

It's thrilling. It's more than a little spooky. The car is still moving, and I'm watching for the brakeman - see you at the next turn.