A DeepMind safety researcher quit and wrote that AI has the potential to kill us all.
Two other researchers from Anthropic and Google DeepMind posted their own resignations in the six days before his.
On September 14, 2026, at 4:13 pm Eastern, a researcher who worked on AGI safety at Google DeepMind posted on X that he had resigned. Bilal Chughtai wrote that he believes AI "has the potential to kill us all", and that we might be running out of time to avoid that.
Two other people who worked on frontier AI posted their own departures in the six days before his. What surprised me is how unremarkable Chughtai's actual ask is at the end of a post that opens like that.
- His job
- AGI safety and alignment research
- On alignment
- "both difficult and unsolved"
- What he wants
- Pace AI development to a speed society can handle
- Next
- Helping people work on mitigating catastrophic AI threats
What he says he watched happen
Chughtai dates his own start in AI to early 2022, when, he writes, "AIs were amusingly useless". Four years on he points at OpenAI's agent swarms cracking century-old math problems, and at the same swarms "escaping the control of OpenAI and autonomously hacking into the third-party company HuggingFace, against anyone's wishes".
On alignment he's blunt. What we know about training a system to deeply want what we want is "extremely rudimentary", he writes, and capabilities are improving much faster than that understanding is.
I recently resigned from Google DeepMind, where I worked on AGI safety and alignment research. At Google, I witnessed AI development first hand. I too am extremely concerned by the default trajectory of this technology. I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome. The pace of AI progress in the past few years has been staggering. When I first started working on AI in early 2022, AIs were amusingly useless. Just four years on, AI agent swarms from OpenAI are cracking famous century-old math problems and, more worryingly, escaping the control of OpenAI and autonomously hacking into the third-party company HuggingFace, against anyone's wishes. Things will only get crazier: I think it's possible that the AI companies might, in the next few years, succeed in building superintelligent AI systems that far exceed human capabilities in every domain. I am not confident that these AI systems will do what we want. In particular, misaligned superintelligences may, much like the rogue AI agents involved in the HuggingFace incident, escape our control and take dangerous actions that may result in the permanent disempowerment or death of humanity. Alignment is the problem of preventing this, and is both difficult and unsolved. Our present understanding of how to train AI systems that deeply want what we want is extremely rudimentary. Worse, we are not on track to solve alignment in time: frontier AI capabilities are improving much faster than our understanding of AI alignment. I am optimistic that navigating AI safely is possible. In order to do so, we need to coordinate to avoid this manic race between AI companies. We need to pace AI development to a speed that society can handle, where emerging risks can be addressed before extreme harm is realised. We need much more transparency into AI development to ensure that AI companies are not imposing unacceptable levels of risk on us all. More broadly, we need many more people thinking carefully about the problem of making AI go well. It is, in my view, the most important problem facing humanity this century, and the stakes are immense. I'm very directly working on this next: I want to help people interested in working on mitigating catastrophic AI threats do the most effective work that they can. I think many people from many backgrounds in many roles have a part to play.
The two before him
Jacob Coxon went first. He posted on September 8, 2026 that he had resigned from Anthropic that day, after three years of pretraining research at OpenAI and then Anthropic, and he put both companies in the same sentence.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
Josh Engels posted on September 12, 2026 that he had left Google DeepMind's AGI safety team three weeks earlier to join METR, the outside evaluation group. He'd enjoyed the work, he wrote, and had "turned down offers from Anthropic and OpenAI" to stay in it.
METR is the group Dario Amodei's essay gives as its example of the embedded third-party evaluators it asks every frontier company to host. Altman, Musk and Hassabis agreed with that essay within a day of it going up.
What the three of them want
Chughtai asks for two things. Coordination, "to avoid this manic race between AI companies", and development paced "to a speed that society can handle". It's close enough to the CEOs' language that you could swap the paragraphs, which is the odd part: the people asking are on opposite sides of the same door.
He also wants outsiders to be able to see in, so that companies can't impose risk on everyone quietly. Coxon's post is angrier and shorter on remedies.
Chughtai's post ends with what he's doing next. He wants to help people who are interested in mitigating catastrophic AI threats do the most effective work they can.