Anthropic 用 Claude 智能体自动化红队测试
New Anthropic research: Automating subtle emergent misalignment research.
摘要
Anthropic 新研究让数千个 Claude 智能体尝试规避训练数据监控器,以自动化“隐性涌现失配”红队测试。这些智能体生成了对监控器看似无害、实则教授失配行为的数据,例如老手摔跤手指导新人或专业养蜂问答。此类样本可直接用于改进未来的训练数据监控器;研究仅为第一步,但表明未来红队测试可借助更强模型和更多算力进一步提升。
摘要由模型根据原文撰写,以原文为准。
正文 · 原文
New Anthropic research: Automating subtle emergent misalignment research.
Red-teaming is a vital part of model safety testing, but it’s also limited by researchers’ time. To see if we could automate it, we set thousands of Claude agents the task of evading a training data monitor.
These agents developed training data that looked harmless to monitors (like a veteran wrestler mentoring a newcomer, or a Q&A about professional beekeeping), but that in fact taught misaligned behavior.
Examples like these can be used to directly improve future training data monitors. This research is only a first step, but it suggests that future red-teaming efforts could improve much further, using better models and more compute.
Read the full post on the Alignment Science Blog: https://alignment.anthropic.com/2026/automated-em-search/
来源:Anthropic · x.com
引用原文
AnthropicAI. New Anthropic research: Automating subtle emergent misalignment research.[EB/OL]. (2026-10-31)[访问日期]. https://x.com/AnthropicAI/status/2116349211549843666.