GPT-6-Astra 在 TheoPhysEval 上解出 50 道理论物理题中的 49 道，单题成本不足 3 美元

团队测试了市售的 AI 智能体，且未给它们任何人工提示。在 OpenAI 的 Codex 编码环境中运行的 GPT-6-Astra 表现最佳。它在 50 道题中答对了 49 道，正确率 98%，平均每道题成本不到 3 美元，整个基准测试的成本略低于 116 美元。

The team tested commercially available AI agents and gave them no human hints. GPT-6-Astra, running in OpenAI's Codex coding environment, came out on top. It found the right answer for 49 of 50 problems, or 98 percent, at an average cost of under $3 per problem and just under $116 for the whole benchmark.

该基准测试考察的是常规研究工作，而非重大难题

The benchmark tests routine research work, not grand challenges

该题集有意略去了重大开放问题，聚焦于作者所称的“按部就班”型问题，即研究人员将已知方法应用于稍作修改的问题。出题者往往只是改动自己近期某篇论文中计算里的几个数值，使新结果无法从旧结果中猜出。据研究人员称，这类问题常被交给博士生，作为进入科研的第一步。

The collection deliberately leaves out major open questions and focuses on what the authors call "turn-the-crank" problems, where researchers apply known methods to a slightly modified question. Contributors often tweaked a few values in a calculation from one of their own recent papers so the new result couldn't be guessed from the old one. According to the researchers, problems like these are often handed to PhD students as a first step into research.

一个来自黑洞研究的例子显示了这些问题仍然颇具难度。要解决它，需要找到一篇较早的论文，将其方程从五维时空移植到四维时空，然后用自定义程序计算结果。协调人 Suvrat Raju 是那篇旧论文的合著者，他花了大约一天时间。作者估计，在现代智能体出现之前，只有少数专家同时具备物理学知识和这些计算方法的经验。即便是他们，也需要好几天。

An example from black hole research shows how demanding these problems still are. Solving it means finding an older paper, carrying its equations over from a five-dimensional to a four-dimensional spacetime, and then computing the result with a custom program. Coordinator Suvrat Raju, a co-author of the older paper, needed about a day. The authors estimate that before modern agents arrived, only a handful of experts had both the physics knowledge and the experience with these computational methods. Even they would have needed several days.

这些问题横跨理论物理的十一个子领域，从流体流动、生物过程到粒子物理和宇宙学。每个智能体都运行了两次测试，一次是在有网络搜索的现实条件下，一次是在无互联网访问的隔离环境中。

The problems span eleven subfields of theoretical physics, from fluid flows and biological processes to particle physics and cosmology. Each agent ran the test twice, once under realistic conditions with web search and once in isolation without internet access.

在 Claude Code 中运行的 Anthropic Opus 5.5 以 92% 位居第二。谷歌的 Gemini 3.8 Flash 得分 68%，而开放权重模型 DeepSeek-V4.1-Flash 和 GLM 5.3 分别达到 54% 和 42%。较弱的模型有时会犯专家一眼就能看出的错误。在黑洞问题上，Gemini 和 GLM 在网络搜索运行中照搬了旧论文的公式，未作改动。那些公式适用于五维，而非所需的四维。

Anthropic's Opus 5.5, running in Claude Code, came in second at 92 percent. Google's Gemini 3.8 Flash scored 68 percent, while the open-weight models DeepSeek-V4.1-Flash and GLM 5.3 reached 54 and 42 percent. The weaker models sometimes made mistakes an expert would have caught right away. On the black hole problem, Gemini and GLM copied the formulas from the older paper unchanged during the web search run. Those formulas apply to five dimensions, not the required four.

互联网访问带来的差异小得惊人。在没有网络搜索的情况下，Astra 和 Opus 各得 92%。许多问题依赖于研究论文中的公式，而模型通常并未获得这些论文。作者将此视为一个迹象，表明领先的语言模型实际上已经记住了理论物理文献的很大一部分。

Internet access made surprisingly little difference. Without web search, Astra and Opus each scored 92 percent. Many problems depend on formulas from research papers that the models usually didn't receive. The authors take this as a sign that leading language models have effectively memorized large parts of the theoretical physics literature.

在有网络搜索的情况下，Astra 解出了所有问题，只有一个问题难住了所有模型。该问题要求给出一个能产生给定曲线的方程。专家的解法依赖于特殊的数学形式，这些形式只有在更宽泛的意义上才算作所需的微分方程。作者承认这个问题可能显得“不公平”，但他们认为它是现实的，因为研究人员常常不得不从不精确的数据中推断底层物理。

With web search, Astra solved everything except one problem that stumped every model. It asked for an equation that produces a given curve. The experts' solution relies on special mathematical forms that only count as the required differential equation in a broader sense. The authors admit the problem might seem "unfair," but they consider it realistic because researchers often have to infer the underlying physics from imprecise data.

若干 AI 的失误其实是答案键的缺陷

Several AI misses turned out to be flaws in the answer key

对于若干问题，与专家的后续核对揭示了其参考解法中微小、难以察觉的错误。一旦修正这些错误，Astra 的答案每次都是正确的。

For several problems, follow-ups with the experts revealed small, hard-to-spot errors in their reference solutions. Once those were fixed, Astra's answer was correct every time.

这种模式或许也能解释一个表面上的矛盾。这些智能体在 TheoPhysEval 上的表现优于其他一些物理基准测试，例如 CritPt，尽管作者认为他们自己的题目平均而言更难。一项近期研究发现，一些理论物理基准测试的评分有误，在某些情况下严重低估了模型的能力。

This pattern could also explain an apparent contradiction. The agents do better on TheoPhysEval than on some other physics benchmarks, such as CritPt, even though the authors consider their own problems harder on average. A recent study found that some theoretical physics benchmarks are graded incorrectly and in some cases badly underestimate what the models can do.

任何想要核验结果的人都可以在 Hugging Face 上以数据集的形式下载题目和测试设置。

Anyone who wants to check the results can download the problems and test setup as a dataset on Hugging Face.

为什么强大的计算能力尚未带来物理学突破

Why strong calculations haven't led to physics breakthroughs

作者写道，当前一些最强的模型可能已经在具体计算上超越了最优秀的人类研究者，即便是在专业领域也是如此。他们称这是一个重大里程碑。

The authors write that some of the strongest current models may already have surpassed the best human researchers on concrete calculations, even in specialized fields. They call this a major milestone.

他们指出，这种影响在数学领域更为明显，AI 智能体在那里已经解决了重要的开放问题。据 OpenAI 称，其中包括关于描述流体流动方程的纳维-斯托克斯问题，它是著名的千禧年大奖难题之一。理论物理尚未出现可比的突破。

The impact is even more visible in mathematics, they note, where AI agents have already solved important open problems. According to OpenAI, that includes the Navier-Stokes problem about the equations that describe fluid flow, one of the famous Millennium Prize Problems. Theoretical physics hasn't seen a comparable breakthrough yet.

作者解释说，一个结果可验证的具体计算只出现在研究过程的末端。在此之前，必须有人构建理论，并向它提出“正确”的问题。当一个问题被证明太难时，优秀的研究者会转向一个更简单但同样有趣的相关问题。当前的智能体并不擅长这一点。

The authors explain that a concrete calculation with a verifiable result only comes at the end of the research process. Before that, someone has to build a theory and ask it the "right" questions. When a problem turns out to be too hard, a good researcher switches to a related problem that's simpler and just as interesting. Current agents aren't good at that.

物理学家对什么问题算得上有趣也有一种共同的（尽管远非一致的）感觉。智能体似乎还不具备这种“品味”。当被要求设计新问题时，它们会陷入人类会觉得无聊的细节之中。

Physicists also share a common, if far from unanimous, sense of what makes a problem interesting. Agents don't seem to have this "taste" yet. When asked to design new problems, they get lost in details a human would find boring.

研究人员强调，这些观察无法量化，也不是对 AI 发展方向的预测。他们认为 AI 智能体可能在不久的将来独立开展顶级研究，但这需要的不仅仅是解题能力。

The researchers stress that these observations can't be quantified and aren't a forecast of where AI is headed. They think AI agents could do top-level research on their own in the near future, but that will take more than the ability to solve problems.

AI 已经在改变物理学家的培养方式以及谁能够参与竞争

AI is already changing how physicists train and who can compete

作者已经看到了对培养的影响。直到最近，一些基准题目本可以很好地作为早期职业研究者的首个研究项目。如今这已不再合适，除非目标明确只是把材料过一遍。

The authors already see consequences for training. Until recently, some of the benchmark problems would have worked well as first research projects for early-career researchers. That's no longer appropriate unless the goal is explicitly just to work through the material.

他们还指出了一个结构性问题：从历史上看，理论物理所需的资金远少于实验研究，这使该领域的机会相对平等。有能力的 AI 智能体可能会改变这一点。基准题目求解成本仍然很低，Astra 的完整运行花费约 116 美元。然而对于更难的题目，作者预计资源需求会急剧上升。到那时，研究小组的预算可能对其能取得什么成果拥有更大的话语权，即便是在理论物理领域也是如此。作者表示，这引发了关于该领域参与机会和机会平等的问题。

They also point to a structural problem: Theoretical physics has historically needed far less money than experimental research, which has kept opportunities in the field relatively equal. Capable AI agents could change that. The benchmark problems are still cheap to solve, and Astra's full run cost about $116. For harder problems, though, the authors expect resource needs to climb sharply. At that point, a research group's budget could have more say over what it can accomplish, even in theoretical physics. The authors say this raises questions about access and equal opportunity in the field.

数学领域已经在进行这些讨论，例如围绕《莱顿宣言》。作者主张理论物理也应展开这些讨论，以免 AI 颠覆该领域。至少物理学中的一个流派迄今一直敦促研究者“闭嘴，去计算”。如果品味和远见开始变得更加重要，物理学家或许需要转而“停下来思考”。

Mathematics is already having these debates, for example around the Leiden Declaration. The authors argue theoretical physics should have them too, before AI upends the field. At least one school of thought in physics has so far urged researchers to "shut up and calculate." If taste and vision start to matter more, physicists may need to "stop and think" instead.

注意转载授权 · 阅读原文。中文正文为 AI 翻译，请以原文为准。

注意转载授权 · 阅读原文