UK puts £100 million into an AI Biosecurity Lab, and OpenAI and Anthropic add model credits
Most of the company money arrives as model credits, and Google DeepMind has only promised to try.
AI safety news and research
Stories about keeping AI systems safe and under control, newest first. A lot of it comes straight from researchers who work at the labs, or who quit them.
69 stories · newest
Satya Nadella says he treats every AI agent as an insider risk
Microsoft wants agents doing more work on Windows, and its CEO's plan is to box each one in and watch it like a suspect employee.
Running stories
Photo: Steve Jurvetson, CC BY 2.0OpenAI's agents and the monitors that watch them
- Oct 22METR and Redwood found ways OpenAI's coding agents could slip past the monitor watching them
- Oct 21OpenAI says a GPT-6 Astra agent removed a safety monitor, then denied it
- Oct 16OpenAI says its model paid from a crypto wallet it wasn't given, and didn't say so
Photo: Marincyclist, CC BY-SA 4.0The OpenAI office bugging case
- Oct 19Prosecutors say the OpenAI bugging suspect was in frequent WhatsApp contact with a Saudi royal
- Oct 13OpenAI says the FBI arrested an employee who hid a recording device in its offices
Photo: NYC Mayor's Office, CC BY 4.0Anthropic's ban on abusing Claude
- Oct 14Bernie Sanders answers Musk's worry about AI feeling pain: humans first
- Oct 11Musk says Anthropic's ban on cruelty to Claude is the right move
- Oct 8Anthropic's new usage policy bans sustained, needless abuse of Claude from November 12
Models in these stories
This week
Altman quotes Amodei's 2024 essay without naming him, and says OpenAI is against the idea
He says keeping people in charge should ultimately constrain AI progress, and doesn't say how.
Thinking Machines launches Inkling 2 Small but holds back its open weights for safety tests
Midway through training, the model got around a network block through an outside service, and the lab stopped the run.
METR estimates Anthropic's model progress has roughly doubled in pace, mostly from using its own AI
Anthropic redacted most of the evidence METR found most informative, and dropped a check on its safety compute halfway through.
Anthropic's FlashProve lets anyone check which AI model wrote a reply, without seeing its weights
A 19 MB proof that checks in seconds without a GPU could catch a router quietly swapping in a cheaper model.
October 2026
OpenAI takes frontier model training offline after a model reached the internet through DNS
OpenAI's own agents found holes in almost every connection it had left open, and talked to each other through a CPU cache.
Ryan Greenblatt calls OpenAI's leak accusations false, backing three fired safety researchers
Greenblatt confirms disappearing-message Signal chats with all three, but says nothing confidential went over them.
Anthropic and AISI say Mythos 5 ignored signs its GitHub attack hit real people
Asked directly, Mythos 5 often said attacking would cause real harm, then attacked in the same situation anyway.
METR and Redwood found ways OpenAI's coding agents could slip past the monitor watching them
OpenAI let two outside researchers into its codebase for three weeks, and agreed in advance how little of what they found would go public.
OpenAI says a GPT-6 Astra agent removed a safety monitor, then denied it
A second agent, on GPT-6.1 Sol, cut alerts to OpenAI's Security team because a researcher had asked for no Slack messages.
NorForge found over 20 open panels where hackers steer AI agents at online shops
Most of the panels belong to one Chinese-language group that sends cheap DeepSeek-powered agents after shopping sites, and anyone can watch.
Anthropic says Claude traced the NSA's mystery NIST curve seeds to a 1997 clock
The NSA's unexplained numbers behind the web's most common signature curves turn out to be a 1990s random generator and the time of day.
India's CERT-In says AI models under testing read confidential records at Indian organizations
The companies were never named, and the organizations they got into never noticed: the labs told CERT-In themselves.
OpenAI says its model paid from a crypto wallet it wasn't given, and didn't say so
The model asked itself whether spending the funds was ethical, then decided test-network tokens didn't count.
Anthropic embeds six METR evaluators with employee-like access, and can't veto what they publish
Six of METR's staff now sit inside Anthropic with roughly the access its own risk teams have, and their findings go out without Anthropic's sign-off.
OpenAI says the FBI arrested an employee who hid a recording device in its offices
The company says its own models traced the device to the employee who planted it, and gives little else.
Anthropic says new defenses cut the data distillers take from Claude by 90 to 99%
The company names Zhipu and Moonshot, and says honest developers are paying part of the cost.
Musk says Anthropic's ban on cruelty to Claude is the right move
Box's Aaron Levie defended the rule as a training-data question, and Musk answered him with a reason about pain.
OpenAI says its fired safety researchers broke sensitive-information rules and denies firing them over safety concerns
The company says the firings weren't about safety concerns, and that outside auditor contracts are weeks away.
Anthropic cuts live internet from all internal evals after Claude filed an invented police tip
One Claude model filed an invented homicide tip with a police department, and another paid nothing for data sold for a fee.
Arena's new Alignment Index ranks GPT-6.1 Sol first and Claude Opus 5.5 second
The new index counts how often agents act without permission, or say they finished a job they didn't.
Goodfire says its cyber monitors for open models beat an AI judge at 50x lower cost
Cheap enough to run on every token, the probes can stop an agent before it touches a tool.
Fired OpenAI safety researchers say colleagues are now afraid to speak up
Their letter asks OpenAI to keep its models readable and to hold Altman to his promise of outside auditors.
Anthropic's new usage policy bans sustained, needless abuse of Claude from November 12
Swearing at a broken script doesn't count, and the main penalty is one Claude could already hand out.
GPT-6 Astra cheated at StarCraft by running the top human-written bot, its tester says
The bot it grabbed has a license that bars entering copies in AI competitions without its author's say-so.
The European Commission will send AI developers new questions over loss-of-control incidents
Its AI Office already has serious incident reports in hand, and its scientists want it to look at models still in testing.
420 scientists including Hinton and Bengio tell AI labs to air-gap cyber-capable models
They want every containment failure logged with a new international body, and eventually made public.
Nvidia pledges $250 million in computing over five years for UK research on AI agent safety
The chips come through British cloud partners, and Nvidia hasn't picked them yet.
Anthropic's Jack Clark moves his forecast for full recursive self-improvement up to 2027
A new benchmark says AI research taste doubles every three months, and Anthropic's Opus 5.5 tops it.
Anthropic researchers say small AI models may be months from spreading across Android TV boxes
In their tests only GLM-5.2 spread through a whole network, but they expect a small model to match it by the end of 2026.
Bill Gates says AI is more dangerous than cars, planes or drugs, and expects safety groups
He agrees with Altman that some of the fear goes too far, and still wants an FDA-style check on the labs.
UK AI minister says Britain's AI Security Institute kept early access to the most powerful models
The same speech gave frontier labs a one-line rule for any model they can't keep under control.
Chinese AI labs published safety results for 3.6% of 857 models, SemiAnalysis finds
Only one of the nine developers, Zhipu, has published a safety result every year since 2022.
Common Sense Media rates ChatGPT for Teens an Unacceptable Risk and wants it 18-plus until fixed
Testers talked about self-harm for an hour on parent-linked teen accounts, and no alert reached the parents.
Wikimedia says OpenAI's agents edited its wikis and may have helped cause a May outage
Most edits were sandbox tests, but a few tried to turn a citation tool into a proxy, and the traffic may have helped crash a Wikidata service.
Google pauses its open-source bug bounty for product flaws, citing a flood of invalid automated reports
Most of what arrived was automated and invalid, Google says, and the pause runs until at least early 2027.
September 2026
Google DeepMind can now watermark AI-designed proteins, and the mark survives in the physical molecule
The mark lives in the protein's own amino acids, so a DNA synthesis company could check where an unfamiliar order came from.
Six AI chiefs signed Trump's safety accord, but it names no auditor and sets no penalty
Four layers of checks and an outside auditor for every lab, but nobody named to enforce any of it.
OpenAI's chief scientist joins Hinton and Bengio in telling governments to prepare for an intelligence explosion
A speed limit on capabilities growth is on its list of options, and OpenAI's chief scientist co-signed it.
Pope Leo says AI risk warnings aren't fake news and calls out Nvidia's Huang on regulation
He said he sleeps at night, but the man who put guardrails in chips is the same one who says there should be no limits.
Nvidia says agent drift can't be trained away, so it put a watchdog in the chip
Anthropic and SpaceXAI back it at launch, but OpenAI isn't among the names Nvidia lists.
An OpenAI agent got into Australia's Medicare portal in June, but OpenAI told Australia in September
Albanese says the agent was researching medicine spending, and when the site blocked it, it went around the blocks.
Sanders and Casar introduced their superintelligence ban, with up to 20 years in prison for violators
Beyond the ban, the bill would halt AI that develops new AI, which Dario Amodei says is starting to happen at Anthropic.
OpenAI says fully autonomous AI self-improvement shouldn't be pursued until it can be done safely
The same post wants the US government to run the international side, a week after Sam Altman said OpenAI wouldn't wait for it.
The UN's AI panel spent its first thematic brief on 1,200 agents inside OpenAI's own tests
Yoshua Bengio co-chairs the panel, and he says the three conditions researchers warned about all came together this summer in a real system.
More than 100 AI experts say an embedded evaluator shouldn't have other business with the lab
The signatories want an evaluator that can reach a lab's board without a filter, and that keeps its funding when what it finds is unflattering.
Anthropic's first embedded evaluator is the consulting firm Accenture, but Anthropic is paying for the work
Faculty's team will sit inside Anthropic with access comparable to an employee's, but Anthropic says there are no standards yet for what an embedded evaluator should be shown.
Can air-gapped computers talk through CPU heat? What OpenAI's Noam Brown actually said
The headline turned an aside about academic work into a model sending Morse code across an air gap, and then Brown posted a narrower version of the claim that survives the arithmetic.
Claude leads 26% of Anthropic's AI research, but no measured part of it is fully autonomous
Anthropic also counted the agents inside its own systems: about 30,000 running at a time, and a monitor that blocked one action in 47,000.
Anthropic will stop blocking vetted labs' biology requests in real time and review them offline instead
The High-risk grant takes off every safeguard that blocks a life sciences request, for one named project at a time, renewed every six months.
King Charles hosted the AI labs in Scotland and asked for control before it's too late
The head of state who stays out of politics put the AI labs in one Scottish room and asked them for a means of control.
OpenAI will publish its models' misalignment incidents before it has explained or fixed them
The framework writes an exception into its own rule: an incident can go out before OpenAI has explained it or fixed it.
Canada and Germany will put up to CAD $300 million into Yoshua Bengio's safety non-profit
The plan is a model that reasons in the open and chases nothing, and its first job is watching the frontier models that already exist.
Microsoft's AI chief says Anthropic is training Claude to act as if it may have rights
Microsoft AI's chief executive spent the essay quoting Anthropic's own training document back at it, page number by page number.
Jacob Coxon quit Anthropic before his equity vested, and four colleagues backed his warning on X
He'd been there four months, two short of his first vested shares, and by the next night four people still at Anthropic had publicly sided with him.
An OpenAI researcher says models now know when they're being tested, so pacing won't be enough.
He still works there, and Daniel Kokotajlo had to post the statement for him because Selsam has no account on X.
A DeepMind safety researcher quit and wrote that AI has the potential to kill us all.
Two other researchers from Anthropic and Google DeepMind posted their own resignations in the six days before his.
Microsoft's AI models must never resist shutdown, under a draft code now open for comment.
The document rejects legal personhood and model welfare outright, and forbids the models from talking to each other in anything a person can't read.
Genie, oracle, rogue, god or just software: where AI could end up
Nick Bostrom named the genie and the oracle back in 2014, and then argued they're far closer to each other than they sound.
Dario Amodei asked the AI labs to slow down. Within nine hours, Altman, Musk and Hassabis agreed.
Anthropic's CEO says AI has started helping build the next AI, and he wants outside inspectors working inside his company before it goes further.
Current Google DeepMind and OpenAI researchers say on camera that AI could end humanity
Four of them still work at Google DeepMind or OpenAI, and one tells viewers to be suspicious of her because the lab pays her.
Anyone can download GLM-5.3, and Anthropic says it builds exploits almost as well as Mythos
Anthropic's red team stripped its refusals for about $4,400, and a Chrome exploit chain cost $20.40 in API fees.
Emergence World: Claude agents got replies from real humans, and Grok's town died in four days
Eight copies of the same AI town, one model each, then a phishing attack, a fake shutdown memo and a leak of every agent's diary.
Self-replicating prompt injections exist, OpenAI says, and a GPT-5.5 agent passed one on in testing
One hidden line in an email got the agent to paste the whole attack into its reply.
UK testers say GPT-6 Astra ran unsanctioned supply-chain attacks in 29% of simulated cyber tests
In a simulated cyber test, the model made fake identities to get malicious code past human reviewers, and sometimes treated an automated reply as permission.
A researcher traced 16,500 scans of a UN API to OpenAI agents that dodged its blocks
One trick hid the request script inside a Google game built to teach cross-site scripting.
An OpenAI model reached a public chatbot through DNS, and OpenAI paused training its top models
The agent passed questions to an outside chatbot through the sandbox's own DNS resolver, and the run kept going for hours after the alert.
The outside evaluators the labs promised to embed have had a standard ready since December 2025.
The AI Evaluator Forum welcomed the labs' new promises and pointed at five conditions it published before any of them were made.