Jacob Coxon, the former Anthropic AI researcher who sparked the renewed debate over AI safety, resigned from Anthropic on Tuesday after three years of conducting pre-training research.

“Neither company is acting responsibly,” Coxon wrote on X in the announcement for his resignation. “They are racing straight to self-improving superintelligence and gambling with our lives.”

Coxon, who previously worked on GPT-4o at OpenAI, said AI leaders privately fear the technology could be catastrophic.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon added. “At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk.”

Evan Hubinger, an alignment science lead at Anthropic, reposted Coxon’s post on X and said he is most worried about “superintelligence arising from recursive self-improvement.”

“Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” said Hubinger on Tuesday.

“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” Hubinger added.

Marcus Williams, a member of OpenAI’s safety oversight team, issued a stark warning in a post on X on Thursday.

“Unless there is AI regulation or a coordinated slowdown between labs, human extinction in the next few years seems very likely,” said Williams.

When a commenter asked Williams to put the risk in numbers, he said the risk is “70% in the next 3 years if there isn’t regulation/slowdown,” and added that “regulation/slowdown is very possible.”

Anna Wang, who works on AGI safety and alignment at Anthropic, said she stays because she believes she can better reduce risks from the inside, but “respect and endorse” people like Coxon who “think that it’s better to do so from the outside.”

“This is a common sentiment amongst my peers,” Wang wrote in a post on X on Wednesday. “There is not yet a viable scientific plan to solve risks from recursively self-improving AI. Please look up!”

Paul Christiano, OpenAI’s new safety hire, addressed the issue of guardrails for AI in a lengthy statement about his role on Wednesday.

“If we build superintelligence without more robust alignment I expect we will permanently lose control of it. If that happens then most people could die,” Christiano said after OpenAI announced he was joining its board and Safety and Security Committee.

“I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term,” Christiano added. “I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.”

Chris Hayduk, a life sciences researcher at OpenAI, reposted Coxon’s post but is looking at the brighter side.

“I work at OpenAI and personally think AI has been and will continue to be an extremely beneficial technology to humanity,” Hayduk wrote on X on Thursday.

“The conversation should be around how many billions of lives it will save,” Hayduk added.

Tomek Korbak, an AI safety researcher at OpenAI who previously worked with Coxon, wrote in a post on X that Coxon remains a “very thoughtful researcher.”

“Neither anthropic nor openai are on track to solve alignment to a degree sufficient for shipping superintelligence and we need to slow down,” Korbak said on Thursday.

Jen Leike, an AI researcher at Anthropic, said in a post on X on Thursday that the most effective safety measures take time to implement, such as the “jailbreaking mitigations for Opus 4” that took over a year to develop.

“Now is a good time to build institutional mechanisms to pace the frontier of AI development,” Leike said.

“The industry is locked into an all-out scaling race to build superintelligence as quickly as possible, and we may need to give everyone more time for safety and alignment mitigations,” Leike added.

Julie Steele, who works on OpenAI’s preparedness team, reposted Coxon’s post on X.

“I work at OpenAI,” said Steele on Wednesday on X. “In my personal capacity, I also think we need to slow down.”

Samuel Marks, a technical staff member at Anthropic, wrote about Coxon’s warning in an X post on Wednesday. He said he was writing in a personal capacity, not on behalf of Anthropic.

He said that AI developers believe their technology could cause human extinction, and “the more senior the employee, the more concerned they are.”

AI models frequently “severely misbehave,” he said, and while developers have methods to “nudge” them toward better behavior, they cannot be robustly aligned.

“I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes,” Marks wrote.