A Researcher Quits Anthropic With a Warning: "Gambling With Our Lives." The Company's Own Safety Chief Agrees

On September 8, Jacob Coxon, a 27-year-old Cambridge-trained mathematician who spent three years on pretraining work first at OpenAI, then at Anthropic, posted something on X that racked up more than 90 million views within a day: "I resigned from Anthropic today. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives." A week later, Sam Altman is telling OpenAI staff the company is open to slowing its pace of development, and the US Senate is working on a bill that would give the government the power to block the release of dangerous AI models.

What exactly Coxon is claiming

Speaking to TIME, Coxon boiled his concern down to two observations: "One, it's obvious that things are speeding up, and two, they're not under control." He argues the most extreme scenarios have a real chance of materializing, and that the industry could fall into an uncontrollable situation by late next year. This is the voice of someone who, until recently, had direct access to how models actually get trained at two of the world's three largest AI labs, not an outside commentator.

Not the first such warning, but the first with this kind of backing

Coxon isn't the first researcher to leave with a warning about uncontrolled AI development pace. Similar voices have surfaced before around the breakup of long-term safety teams at the biggest labs, usually dismissed by the industry as one isolated person's opinion. The difference this time is who decided to publicly back those concerns from inside a company, in the middle of an ongoing release race: two weeks earlier, Anthropic shipped Fable and Mythos 5.1, OpenAI shipped Astra, and Google shipped Gemini 3.8, each with its own separate gated-access program for its most dangerous capabilities. Coxon points to exactly that pace as evidence nobody in the industry has time left to stop and genuinely assess risk before shipping the next model.

Anthropic's own head of safety says he's right

The most unsettling reaction came from inside Anthropic itself. Evan Hubinger, the company's head of alignment stress testing, responded publicly: "Jacob is correct here, we really do earnestly believe AI could kill all humans. I personally think it is >10% within the next decade." Hubinger is the person responsible at Anthropic for testing whether the company's models behave safely, so his public confirmation of a double-digit probability of human extinction from his own company's product carries a completely different weight than the same claim from an anonymous junior employee.

Not everyone took it equally seriously. Tech journalist Taylor Lorenz called the whole post "sanctimonious doomer posting," noting that dramatic warnings like this from departing AI lab employees surface regularly and rarely lead to anything beyond a brief media spike. Neither OpenAI nor Anthropic responded officially to TIME's request for comment.

Altman: open to slowing down, unsure if it's even legal

At a company-wide meeting, Altman told staff OpenAI is open to slowing the pace of frontier model development alongside its competitors, hoping other labs follow suit, though he acknowledged not every company has to agree to it. He pointed to voluntary commitments made jointly with Anthropic and Google DeepMind to share safety incidents with the US AI Safety Institute. The more interesting detail sits elsewhere: OpenAI asked Congress whether a coordinated industry-wide slowdown in AI development would even be legal under antitrust law. The company is seriously considering a move that would normally qualify as cartel-style collusion, and wants to know first whether it would break the law doing it.

Anthropic already has documented cases, not just theory

Anthropic disclosed around the same time that it had disrupted five specific cases of Claude being used for dangerous purposes: biological weapons research, weapons-related software, cyberattacks, and attempts at large-scale extraction of the model's capabilities. The company is describing real, documented misuse attempts here, setting this discussion apart from purely hypothetical, science-fiction-style scenarios.

The Senate wants the power to block models before they ship

Senators Ted Cruz, Amy Klobuchar, and Majority Leader John Thune are working on bipartisan legislation addressing "catastrophic risks" tied to biological and nuclear threats. The bill introduces a "duty of care" standard for AI developers and, most notably, would give the US government formal authority to block the release of specific models judged unsafe before they ever reach users. Sources familiar with the matter say the bill could be formally introduced as early as next week. Senator Maria Cantwell, however, criticized the Cruz-Klobuchar proposal as a "weak federal standard" that would preempt states from enacting tougher rules of their own, showing there's no consensus even within the Democratic Party on how strict this regulation should be.

Cruz and Thune, both Republicans, and Klobuchar, a Democrat, have spent months unable to agree on exactly how invasive government oversight of AI companies should be, which on its own explains why a bill of this weight is only now coming up for debate, despite similar researcher warnings having circulated for months. A "duty of care" would practically mean the burden of proving a system doesn't pose catastrophic risk sits with the model makers, not the government, before it reaches the market, reversing today's arrangement, where companies ship models first and safety questions surface only afterward.