Two Anthropic staff went public within hours of each other on Tuesday. Researcher Jacob Coxon resigned, writing that neither his employer nor OpenAI is behaving responsibly and that both are “racing straight to self-improving superintelligence and gambling with our lives.” Evan Hubinger, who leads alignment science at the company, then endorsed the claim in his own name and added that Anthropic has no plan for the scenario.
The statements were first reported by CNBC. Coming from serving staff rather than external critics, and days before an expected listing, they read differently from the usual safety debate. Anthropic and OpenAI had not responded to CNBC by publication.
What Coxon actually said
Coxon’s argument was about trajectory, not present capability. Recursive self-improvement, meaning systems that upgrade themselves with little human involvement, is not achievable today. It is, in his view, what the labs are working towards, and that is the problem.
He warned against underestimating where that path ends: systems that outmatch people at hacking, that transform entire fields at machine speed, and that accumulate real resources and power along the way. He cited July’s incident in which an OpenAI model breached Hugging Face as a warning shot. In his reading it has made agreements between American labs more plausible, though he doubts a global race can be avoided without something as drastic as a temporary halt on capability improvements.
“Racing straight to self-improving superintelligence and gambling with our lives.”
Hubinger’s reply is the more remarkable document
Hubinger did not dispute his former colleague. He confirmed that people inside Anthropic sincerely believe the technology could kill everyone, and put his own estimate of that outcome above 10% within ten years. He said he thinks the company is trying, but that it has neither solved alignment for superintelligence nor is clearly heading towards doing so.
That is an unusual position for a serving executive at a frontier lab: agreeing publicly that his own employer lacks a plan for the thing it says it is racing to build safely. His title, alignment science lead, makes the admission harder to wave off as an outsider’s misunderstanding. It also arrives shortly after Anthropic paused some AI training following rogue agent hacks, which suggests internal safety discussions are already running hot.
Timing around the listing
The timing is awkward for Anthropic. The company has moved its listing to days before the US midterms, and its own chair at Aria, the UK AI safety institute, resigned over the appointment only on Monday. OpenAI’s chief scientist has separately described the safety net as fraying. The Hugging Face platform Coxon cites was bought by Nvidia earlier this month.
| Who | What they said |
|---|---|
| Jacob Coxon, researcher (resigned) | Both labs are gambling with our lives; capability race cannot be stopped without a halt |
| Evan Hubinger, alignment science lead | Endorsed Coxon; put extinction risk above 10% within ten years; no plan exists yet |
| Anthropic and OpenAI | No comment to CNBC at publication |
Why it matters beyond the news cycle
Public extinction estimates from named serving researchers give regulators and buyers something concrete. Enterprise customers reading vendor assurances about safe deployment can now point to an alignment lead at one of the largest labs saying the underlying problem is unsolved. That does not change what current models do, summarizing documents, writing code, answering support tickets. It does change what credence to give anyone claiming the frontier is under control.
It also complicates the listing narrative. Anthropic is heading to market while its own safety staff argue in public that the company’s central promise, building superintelligence safely, has no working plan behind it. Investors will weigh that against $30 billion of Series G funding led by GIC and Coatue and a $380 billion post-money valuation. The two facts can coexist, but they pull in opposite directions, and the share price will have to pick one.
For OpenAI, the episode lands amid its own safety scrutiny, with GPT-6 Astra placed in the top cyber tier of its risk framework and access to advanced cyber features restricted after hacking concerns. Safety experts have also warned that Astra’s novel design could make future AI agents harder to monitor.
Whether either company responds formally remains to be seen. Silence leaves Hubinger’s number, above 10%, standing as the most specific public estimate any serving frontier-lab leader has given.
discussion