A researcher at Anthropic who trained new AI models has resigned his position, issuing a dire warning that has gone viral.
Jacob Coxon posted a seven-part message on X yesterday – see it below – that racked up nearly 76 million views overnight, a response fueled in part by a Wall Street Journal exclusive interview with the now-former Anthropic employee.
The 27-year-old Coxon begins his cri de coeur on X with his resignation announcement:
“I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.
“Do not underestimate the power of this technology,” he continues. “These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.”
Coxon’s third post in the string is perhaps the most chilling. “The people building AI earnestly believe that it could kill us all by the end of the decade,” he writes. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”
In the Wall Street Journal interview, Coxon explained, “We’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already.” The researcher, who worked at OpenAI before joining Anthropic earlier this year, said that competition between the two companies, as well as with Chinese rivals, make safety trade-offs inevitable.
“We’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already,” he told the Journal, which notes that Coxon reports that his AI colleagues use terms “crunchtime” and “endgame” to characterize the state of the fast-tracked self-improving AI models.
In his X posts, Coxon addresses the mindset of the AI industry:
“A common response is ‘if they truly believe this, why are they still building it?’ At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk.”
The Journal notes that Coxon isn’t alone in sounding the this alarm, and quotes OpenAI CEO Sam Altman who recently said at a summit in North Carolina: “I think some things are going to go very wrong with cybersecurity unless people act quite urgently.”
Coxon ends his seven-part X message by saying he is “optimistic about the potential for coordination” and urging other researchers to speak out.
“Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable,” he writes. “I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities.
“If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because ‘it’s happening anyway’ – or take this moment to call for different conditions?”
