To test whether failures came from architecture or pretraining, we trained LLMs directly on StateBench, then evaled them on longer inputs.
摘要
为测试失败是源于架构还是预训练，我们直接在 StateBench 上训练 LLM，然后用更长的输入对它们进行评估。 混合模型 + Gated DeltaNets 在前 3 个级别上近乎完美；transformer 随长度增加而退化，且无法解决一般有限问题。
摘要由模型根据原文撰写，以原文为准。
正文 · 原文
To test whether failures came from architecture or pretraining, we trained LLMs directly on StateBench, then evaled them on longer inputs.
Hybrids + Gated DeltaNets were near-perfect on the first 3 levels; transformers degraded with length & couldn’t solve general finite.
来源：Ai2 · x.com
引用原文
allen_ai. To test whether failures came from architecture or pretraining, we trained LLMs directly on StateBench, then evaled them on longer inputs.[EB/OL]. (2026-11-04)[访问日期]. https://x.com/allen_ai/status/2117670299298599001.