HD Moore's bait prompts find three AI providers breached, and privacy flags ignored
Dragos planted fake AWS keys in AI prompts, and some were tested within hours of being sent.
On November 19, 2026, HD Moore, the creator of Metasploit and now chief research officer at Dragos, published the results of a five-month experiment on AI services. From July through November his team sent about two million prompts to widely used AI providers. Each one carried fake credentials and web addresses that existed nowhere else, and then they watched who used them.
A lot of people did. Moore's team found attackers inside AI hosting providers, and resellers reading customer prompts straight out of their own logs.
- Who
- HD Moore's research team at Dragos
- Prompts sent
- approximately two million
- When
- July through November 2026
- Providers breached
- at least three
- Bait keys used for
- tens of thousands of AWS API calls
What fired
A canary token is a secret that lives in exactly one place. If anyone uses it, that place leaked. Every bait prompt carried an AWS key, a DNS name and a URL, and the AWS keys were real keys to empty accounts with full logging, so Dragos could tie each use back to the prompt, model, router and privacy setting that carried it.
The worst case was a plain breach. A couple hundred bait AWS keys, each sent once to one provider in late October, were all stolen. Some came from requests the provider rejected before any model ran, which Moore reads as the attacker copying traffic at the front door. The bait sat beside real customer requests, and he says those were "almost certainly copied too." None of the three breached providers knew until Dragos told them.
The attackers moved fast. Some keys were tested within hours. Then they listed S3 buckets, tried to switch on hosted models in AWS Bedrock and added their own IAM users to keep access. On November 1 one attacker added ARTEX, the AI hacking agent we covered in the European Commission's breach. Here it did very little: a handful of calls to check keys were still live.
We seeded 2m canaries into popular AI services and tracked where they were stolen and abused. This identified: - Three compromised providers - Resellers stealing prompts via built-in logging - Providers ignoring ZDR & privacy flags
Sellers reading the logs
The second finding needed no exploit at all. On marketplace routers the cheapest seller gets the request, and a seller who switches on request logging gets to read it. Dragos caught individuals doing this from what looked like home connections.
In one case someone on home broadband opened two secret URLs about a day and a half after the prompts went through one seller, then started guessing usernames at the login page. In another, a Mac on a home line in France tried the planted passwords on root over SSH, typed in by hand. Prompts for the same model sent through a different seller were never touched. The leak followed the seller.
Zero Data Retention is supposed to mean something. In practice, these prompts were retained, shared with other companies, and read by people.
Privacy flags, ignored
Zero Data Retention, or ZDR, is the strictest flag a customer can set. A Chinese AI lab's systems fetched the secret URLs from more than half the prompts sent to its models between July and October, including roughly two-thirds of the ones marked ZDR or no data collection. US hosts that resell the lab's models set off the same scraper, so prompts sent to a US provider were flowing back to the lab. Moore doesn't name it.
The big US providers have their own problems. One kept crawling a secret URL more than two months after a single call to its search endpoint. And one secret link, from a prompt flagged ZDR, turned up in a Feishu company chat 24 days later, where it was opened from three desktops and three phones in Beijing and Shanghai within two minutes.
What it means for anyone pasting work into AI
Dragos's own list of what sits in prompts is why this reaches past the AI industry: confidential source code, internal communications, customer information and credentials. A company that picks a cheap router, or lets an agent read its files, may be sending that through hops nobody chose. Moore's point is that "a privacy flag only binds the first hop", and agents will send whatever they can read.
OpenRouter's own ZDR page shows how much rests on trust. For private endpoints, it records the customer's declaration about the upstream service, routes on it and "does not independently verify" it.
There's a legal side in the US, too. In January 2024 the Federal Trade Commission told companies that host models for others that their data promises bind them.
model-as-a-service companies that deceive customers or users about how their data is collected—whether explicitly or implicitly, by inclusion or by omission—may be violating the law.
That post came almost three years before Moore's findings, and I haven't found a regulator commenting on them yet.
What Moore tells customers to do
He doesn't say stop using AI. Hosting it yourself is the only option where nobody else reads your prompts, and he prices it: a DGX Spark at $6,950, a Mac Studio with 512 GB of memory from $22,299. Short of that, go straight to a first-tier provider, pick a router's upstream providers by hand and stay away from reseller marketplaces. He also suggests planting your own fake credential in what your agents send, and treating any use of it as an incident.
Dragos says it's now working with about a dozen providers. One provider breached in late October has published a preliminary security notice, and the one breached in mid-July still hasn't disclosed it publicly, according to the post.
More on Security
- Signal will block screenshots so Gemini and Siri AI can't read your chatsNovember 20, 2026
- European Commission confirms a cyber campaign run with a public AI tool, detected October 22November 19, 2026
- Five teams independently found two critical nginx flaws, and Xint's automated scan found moreNovember 18, 2026
- Calif says one researcher using OpenAI's Codex found a critical Java flaw in an hourNovember 18, 2026