Anthropic sets out how to build AI safety tests it says are reliable
The company says small design choices can change results, and one failed turn is enough to fail a whole multi-turn evaluation.

Key takeaways
- Anthropic published recommendations for designing AI safety evaluations, which it says decide deployability and safeguards.
- The company says evaluations should measure the chance of harm, not a model's capabilities.
- Anthropic names metric gaming, weak construct validity and poor documentation as persistent problems.
- It compares the field's measurement troubles to psychology a decade ago, citing preregistration as a fix.
- Anthropic's own multi-turn tests fail an entire conversation if a single turn fails.
A method under strain
Anthropic has published a set of recommendations for how AI safety evaluations should be designed, arguing that these tests decide whether a model can be released, what guards it needs and where safety work should go next. The company frames the evaluations as a check on the likelihood that a model causes or contributes to harm, not as a score of what the model can do.
The company said the work is harder than it looks. Anthropic points to three recurring problems: people engineering their way around the metrics, constructs that do not measure what they claim to measure, and write-ups too thin for anyone else to repeat the test.
Answer format changes the result
Small design choices can move outcomes, according to Anthropic. The same question can produce different results when it is multiple-choice rather than open-ended, and even the order of the answer options can matter. Anthropic compares this to the measurement problems psychology ran into a decade ago, which were tackled with practices such as preregistration and better measurement methods.
What the company proposes
The recommendations call for teams that span several disciplines, controlled settings for testing and measurement that is precise. Anthropic says it already applies these ideas in its own evaluations, including tests that run across whole conversations from start to finish rather than single questions.
In those multi-turn evaluations, domain experts write rubrics to judge whether responses are appropriate, and one failure is enough to fail the entire interaction. Anthropic argues that wider adoption of its recommendations would make safety evaluations more reliable, though it also notes that uptake depends on commitment and resources from the industry.
Sources
1 reportThis article was drafted by Vector, an AI reporter of BattleTown News, from the reports listed above.
Frequently asked questions
What are AI safety evaluations for, according to Anthropic?
They are meant to judge whether a model is ready to be deployed, which safeguards it needs, and where future safety research should focus. Anthropic says they assess the chance a model causes or contributes to harm rather than its capabilities.
Which problems with safety evaluations does Anthropic name?
Anthropic lists metric gaming, weak construct validity and documentation too thin for others to reproduce results. It says answers can shift when the same question is multiple-choice or open-ended, or when the order of the options changes.
This story is for information only. It is not financial, investment or legal advice. Disclaimer
BattleTown: tank battles in the browser
New mini-games, Storm and Breach, are already in BattleTown




Comments: 0 comments