"Scaling Laws for Looped Mixture of Experts"
摘要
Meta 研究者提出 Loop Scaling Laws，可在给定算力和内存预算下预测循环 Transformer 的最优循环次数与专家数量。在万亿 token 规模上，1.3B 循环 MoE 以相近训练算力达到 2.9B 非循环 MoE 的推理性能，参数效率约提升 2 倍。
摘要由模型根据原文撰写，以原文为准。
正文 · 原文
"Scaling Laws for Looped Mixture of Experts"
Looped Transformers eventually hit diminishing returns when you loop too much, but MoE sparsity lets models benefit from more loops before performance plateaus.
So Meta researchers introduce Loop Scaling Laws to predict the optimal number of loops and experts for a given compute and memory budget.
And at trillion-token scale, a 1.3B looped MoE matches the reasoning performance of a 2.9B non-looped MoE at similar training compute, achieving roughly 2x parameter efficiency.
来源：alphaXiv · x.com
引用原文
askalphaxiv. "Scaling Laws for Looped Mixture of Experts"[EB/OL]. (2026-10-28)[访问日期]. https://x.com/askalphaxiv/status/2115406473656451216.