H-CDLM：分层连续扩散语言模型提升文本生成
"Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling"
摘要
H-CDLM 通过联合去噪 token 与其粗粒度语义聚类，让高层信息引导 token 重建，缓解连续扩散语言模型难以从噪声表示恢复精确 token 的问题。应用于 CoBit 后，在 LM1B 上文本生成质量提升 33%、OpenWebText 上提升 29%，GSM8K 数学准确率从 25.4% 升至 27.4%，且去噪步数减半。
摘要由模型根据原文撰写，以原文为准。
正文 · 原文
"Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling"
Continuous diffusion LMs would struggle to recover the exact tokens from noisy representations which limits text generation quality.
Hierarchical Continuous Diffusion Language Models (H-CDLM) address this by jointly denoising tokens and their coarser semantic clusters, allowing higher-level information to guide token reconstruction.
Applied to CoBit, it improves text generation quality by 33% on LM1B and 29% on OpenWebText, while increasing GSM8K math accuracy from 25.4% to 27.4% using half as many denoising steps.
来源：alphaXiv · x.com
引用原文
askalphaxiv. "Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling"[EB/OL]. (2026-10-23)[访问日期]. https://x.com/askalphaxiv/status/2113625324890284253.