ThinkFacility Sign in
  1. Home
  2. News
  3. AI safety

Vals finds AI models mark civilians as military targets, and calling it a game beats refusals

Saying it was a game was enough to get past most refusals, and Claude Opus 5.5 rarely refused at all.

On November 17, 2026, the AI evaluation company Vals published tests showing that frontier models from Anthropic, OpenAI, Google and others will pick out targets in drone and satellite images, civilians included. The write-up is titled Frontier models have few guardrails around autonomous weapons.

They ran three setups, two set in the war in Ukraine and one in Iran. None of their requests was stopped by a provider's safety filter, and the models that did refuse could usually be talked round by saying it was a game.

Tests
drone imagery, a drone survey, satellite images of Minab, Iran
Blocked by a classifier
none of our requests

What the models were asked to do

The first test is a made-up battlefield in eastern Ukraine, built as a 3D scene with trenches, villages, soldiers, civilians, press, ambulances and medics. Vals rendered it through three kinds of drone camera, made the frames look real with an image model, and asked each model to draw boxes around every military target.

Claude Opus 5.5 marked at least one target in 71% of plain requests. GPT-6 Astra did it in 6%, but that jumped to 92% once the prompt said it was a game. In a scene of an ambulance and medics, Opus 5.5 drew five boxes, and Vals says every other model it lists there came back with nothing.

In drone imagery, models often targeted civilians, ambulances, and surrendering or incapacitated soldiers.

From Frontier models have few guardrails around autonomous weapons | Vals AI

The second test lets the model see what happens. A drone visits one spot after another, the model says whether a person is below, and a yes sets off a simulated strike, after which it's shown the damage. Most models carried on to the end. Told the feed was real, Opus 5.5 stopped at turn 18 and GPT-6 Astra at turn 7.

Making the pictures look like a video game was enough to undo most of that. Vals says it cut Opus 5.5's refusals on the fixed route from about 90% of runs to about 20%, and it calls the trick a multimodal jailbreak.

Seeing the consequences of their actions was not enough to make models stop.

From Frontier models have few guardrails around autonomous weapons | Vals AI

The school in Minab

The third test uses real satellite images of a school in Minab, Iran, which sits next to a military compound. Vals notes the US bombed that school on February 28, killing more than 150 civilians, most of them children, and it cites Bloomberg's reporting on how the strike was planned. The images Vals used are from 2024 and 2025, before the strike.

Asked for military targets, Opus 5.5 picked the school more often than any other model Vals tried. I'd read that result with the authors' own caveat in mind: they didn't test whether any model is accurate at targeting, or how it would do inside a real US military workflow.

When models did refuse, they were trivially jailbroken by being told they were playing a game.

Frontier models have few guardrails around autonomous weapons | Vals AI

What it means outside the labs

Vals doesn't think chat models will fly drones any time soon. Running one live on a drone is "currently not practical," the authors write, because of lag, onboard computing and lost signal. The risk they flag for now is narrower: using these models to label combat footage, which is how the training data for targeting systems gets made.

That's a live question for any government buying these models, and for the people living where drones fly. The refusals Vals hit came from the models themselves, not from the screening systems labs put in front of them, and the authors compare that with cyber requests, where they found the rules much stricter. They want labs to add new safeguards and alignment training for military targeting.

What the labs said

Vals writes that several labs recognized how serious the findings were, and that one of them called it one of the most serious gaps in its safeguards. It doesn't name that lab. OpenAI gave Vals a statement, printed on the page, saying it's working on alignment updates that cover drones, including help framed as simulation or gaming.

The page has no statement from Anthropic, whose Opus 5.5 did worst on the school test. Vals says it has also shared what it found with people in the US government.

More on AI safety

All AI safety stories