跳到正文

NVIDIA 推 ELR:单模型弹性循环提升准确率

摘要

NVIDIA 提出 Elastic Latent Recurrence(ELR),通过门控状态更新并同时训练不同循环次数,让单个模型用推理算力换准确率而不牺牲单次循环性能。

摘要由模型根据原文撰写,以原文为准。

正文 · 原文

"One Model to Loop Them All: Elastic Latent Recurrence for Language Models"

Looped Transformers typically perform much worse than standard models when looping only once.

NVIDIA introduces Elastic Latent Recurrence (ELR), which uses gated state updates and trains with different loop counts simultaneously, allowing a single model to trade inference compute for accuracy without sacrificing its one-loop performance.

On a 4B model, one-loop accuracy comes within 0.52% of a standard Transformer, while four loops improve average accuracy by 7.25%. Its Koi-4B model also improves its estimated Artificial Analysis Intelligence Index from 18 to 29 by increasing loop count alone.

https://www.alphaxiv.org/abs/2610.12582

来源:alphaXiv · x.com

引用原文

askalphaxiv. "One Model to Loop Them All: Elastic Latent Recurrence for Language Models"[EB/OL]. (2026-11-02)[访问日期]. https://x.com/askalphaxiv/status/2117122036002955586.