Agents are demonstrating increasingly strong capabilities in recreating games and applications. One likely reason is their growing abilit...
摘要
Arena 研究提出 Interaction-as-a-Judge 评估范式，让计算机使用智能体直接操作游戏和应用来改进评估。该范式提升了与用户偏好的一致性、模型检测游戏错误的能力以及下游生成，并随计算机使用能力增强而扩展。团队还开源了 ArenaWebdevPreferences 和 ArenaWebdevCorrectness 两个智能体轨迹数据集。
摘要由模型根据原文撰写，以原文为准。
正文 · 原文
Agents are demonstrating increasingly strong capabilities in recreating games and applications. One likely reason is their growing ability to directly interact with these applications.
We studied Interaction-as-a-Judge, a growing evaluation paradigm where computer-use agents directly interact with games and applications to improve evaluation.
Interaction-as-a-Judge improves:
- alignment with user preferences.
- a model’s ability to detect errors in games.
- downstream generation. We see improvements in inference-time scaling and the ability for interaction-only reinforcement learning.
It also scales with computer-use capabilities; the better the computer-use model and method, the better the judgments.
We’re also open-sourcing two agent trace datasets, ArenaWebdevPreferences and ArenaWebdevCorrectness, to help support future research. Dive into the thread to learn more.
Arena research intern and PhD candidate at Carnegie Mellon University, Wayne Chi, explains Interaction-as-a-Judge: letting computer-use agents play games and use apps to better improve evaluation and downstream generation. A computer-use agent navigates an application, producing interaction traces for evaluation by an LLM judge or a trained reward model. The talk covers preference evaluation using real Code Arena battles and correctness evaluation using requirements mined from real user complaints in Agent Arena. Watch the full talk: https://www.youtube.com/live/lu--lrbznOk在 X 查看被引用的帖子
来源：Arena.ai · x.com
引用原文
arena. Agents are demonstrating increasingly strong capabilities in recreating games and applications. One likely reason is their growing abilit...[EB/OL]. (2026-11-13)[访问日期]. https://x.com/arena/status/2120992949169446994.