More than 100 AI experts say an embedded evaluator shouldn't have other business with the lab
The signatories want an evaluator that can reach a lab's board without a filter, and that keeps its funding when what it finds is unflattering.
More than 100 people signed a public letter on September 18, 2026. Signatories set out what an embedded evaluator has to be given before its findings are worth anything, and the AI Evaluator Forum organized it. Geoffrey Hinton and Stuart Russell are the first two names.
Anthropic named its own first embedded evaluator the same day. The letter doesn't mention that choice, or any company by name.
- Published
- September 18, 2026
- Signatories
- 100+, all signing in a personal capacity
- Standard cited
- AEF-1, already in early adoption
- Named first
- Geoffrey Hinton and Stuart Russell
- Asks for
- Access equivalent to highly privileged employees
What the letter asks for
Five conditions, which the signatories call a minimum rather than a full list.
- Independence. Evaluators keep full editorial control, and they aren't owned or governed by the companies they assess.
- Differing viewpoints. Several evaluation organizations across a range of priority risk areas, each free to say where its conclusions differ from another's.
- Transparency. Methods and findings published, with non-disclosure agreements kept narrow.
- Protection from retaliation. Cover against retaliatory litigation, and funding that survives an unflattering finding.
- Access. Equivalent to the company's own highly privileged employees, with candid one-on-one conversations with staff.
Independence is where it gets specific, and it turns out to go well past who owns whom.
should not have other significant commercial business with them, and should not accept any form of payment or other reward contingent on the evaluator's findings
Anthropic is funding Accenture's evaluation work directly. Both have also run a joint business since December 9, 2025 (the Accenture Anthropic Business Group, with roughly 30,000 Accenture staff being trained on Claude), so the commercial relationship is about nine months older than the evaluation one. Granted, this letter sets a condition and doesn't say who anyone should have picked.
Transparency here goes well past publishing a report. Signatories want an evaluator talking straight to a company's board and its other oversight bodies, unfiltered, and publishing the evidence behind what it says. Redactions would be time-limited and held to narrow interests (customers' sensitive information among them).
One condition I didn't expect is about money. Signatories want funding arrangements that leave an evaluator confident of getting paid even when its conclusions embarrass the company it's sitting inside (retaliatory litigation gets its own clause too).
Where this came from
AEF-1, the forum's written standard, gets a mention as one example of terms like these being set down, and far more work will be necessary, the letter says. We covered the standard itself on September 15, 2026.
Embedded evaluation should be treated as "a complement to, rather than a replacement for" wider external oversight, the letter says, which means more public transparency and broader access for independent researchers as well.
What happens next
Signatures are still open. A new signer gives a title or role and an organization, and the name and title go up publicly.