论文提出 softmax 与交叉熵数值修复方案
一篇论文指出 softmax 和交叉熵存在数值缺陷,会使极小梯度错误地舍入为零或本应消失却保持非零,进而引发损失突增并干扰知识蒸馏。作者提出简单修复,在小规模 grokking 基准上消除了 Slingshot 损失尖峰,某设置下测试准确率从 94.3% 提升至 100%,且无需更高数值精度即可防止蒸馏中学生模型偏离教师模型。
按时间排列的全部内容。默认只显示通过双重评分的入选条目;切换到“全部收录”可以检索所有抓到的资料。
2,350 条收录动态,更新于 06:26
全部收录包含没有通过评分的资料,标题以常规字体显示并标为“未入选”,只供检索,不写入选理由。
一篇论文指出 softmax 和交叉熵存在数值缺陷,会使极小梯度错误地舍入为零或本应消失却保持非零,进而引发损失突增并干扰知识蒸馏。作者提出简单修复,在小规模 grokking 基准上消除了 Slingshot 损失尖峰,某设置下测试准确率从 94.3% 提升至 100%,且无需更高数值精度即可防止蒸馏中学生模型偏离教师模型。
该论文用简单二次模型说明重尾数据与噪声梯度即可触发 LLM 训练中的损失尖峰,无需依赖复杂神经网络行为。在一个 1.3B Transformer 上,其模拟可在 CPU 上每层仅用 5 秒识别易出现尖峰的训练配置。
美国马里兰大学药学院团队在 Nature Communications 发表 KaML-ESM,用 ESM 蛋白质语言模型构建序列神经网络,在 GAINES 增强的 PKAD-3r 数据集上训练,预测 Asp/Glu/His/Cys/Tyr/Lys 残基电离常数。
此前已预告。现在好玩的部分来了。 带上你最糟糕的文件。那个有 14 个标签页、毫无命名规范的文件。 Kimi K3.1 Pro 登陆 WorkBuddy。
Meet Kimi K3.1 Pro, our most capable 3T-class model.
我去,刚发现他们出了个新版本的 Claude Code
Wiley 旗下期刊 Macromolecular Materials and Engineering 于 10 月 19 日为其 5 月对普林斯顿学生 Rishika Porandla 论文的撤稿通知补充更正,说明原文描述的多数物理实验并未实际进行,文中条件、数据与结论未经实验测试或获得。
https://x.com/i/article/2113826834836234240
一项预印本研究比较了仅对原始对话历史做稠密检索加重排的 RAW-RAG 与复杂 agentic memory 系统,在 LoCoMo、LongMemEval、MEME、MemoryAgentBench 的 7 组基准与模型组合中有 6 组排名第一,比最强对比系统高 0.13–5.53 分。
RAG is so back! This work shows that plain RAG over raw chat history beats complex agentic memory systems on most long-term memory benchmarks. (bookmark it) They tested Raw-RAG, which is just dense retrieval plus reranking over the raw conversation, feeding a reader LLM. Across LoCoMo, LongMemEval, MEME and MemoryAgentBench, it ranks first in six of seven benchmark and model settings, ahead of the strongest agentic memory system by 0.13 to 5.53 points. Ingestion on BEAM-1M takes 0.3 hours, against 3.2 to 27 hours for systems like Mem0, MemOS and Memora. Why does it hold up? Raw history keeps more of the evidence a question needs, including the relations and dependency paths behind an answer. Rewriting history into memory entries tends to drop those important bits. Even with complete access to the transformed memory stores, models answer less accurately than with raw evidence. Before adding a memory layer to your agent, at least run this baseline on your own data. https://arxiv.org/abs/2610.30755 Chat with Paper: https://academy.dair.ai/papers/is-complex-agentic-memory-always-needed-naive-rag-is-stronger-than-we-thought-2610.30755
英国政府宣布投入约 6 亿英镑应对 AI 带来的公共卫生和安全风险,其中最多资金投向生物安全计划,包括斥资 1 亿英镑建立 AI 生物安全实验室以识别生物威胁,另投入 6,500 万英镑研发针对病原体的防范和应对措施。
我选择降临
Vibe your way to 50K. The 2026 Epic AI MegaJam begins Nov 12: https://epic.gm/epic-ai-megajam-2026
他辞职是因为终于付得起 Opus 的账单了
👀 Thariq Shihipar (@trq212) has left Anthropic.
Indigo Talk 新一期在 Palo Alto 录制,邀请万维钢和「孤独大脑」主理人老喻对谈。话题包括 AI 时代的存在主义、AI4Science 以及各自的 Personal Agent,节目尚未上线。
Grok Bot 问答 https://x.com/i/spaces/1pJdRDpEEQpKW
什么是 OpenClaw Enterprise,它为何重要? @heyneighbor 与 @kevins8 坐下来聊了聊: - OpenClaw Enterprise 是什么 - 他为何打造它 - 它能做哪些其他智能体做不到的事 - 你可以如何参与 + 了解 Red Hat 和 NVIDIA 为何在贡献力量🦞
努力迎难而上。很感激能与一个和我一样深感责任的团队共事。
By the by, if I had to guess, Greg Brockman is currently doing heroic things to fix the internet behind the scenes. And I expect that even though his net contribution to AI safety and resilience will in the final tally be unsurpassable, no one in the space will think to attribute large scale patches to his effort or effort that happens under his leadership.
Google CEO Sundar Pichai 在 Big Technology Podcast 访谈中表示,业界"距离 RSI(递归自我改进)的终态还相当远",但 AI 模型已在模型开发的多个环节与研究人员形成 RSI 循环。
一项针对对话式 AI 在基层医疗场景中的测试研究发布。由于页面正文未能加载,具体模型名称、测试机构、样本量与结果均无法从原文确认。
上个月,一个大型智能体团队使用了超过1000亿个Token来解决Navier-Stokes千禧年大奖难题。现在我们正用类似方法进行红队测试,以发现并修复基础设施漏洞。
To make sure our frontier models don't cause another HuggingFace incident, we built an offline world that they use during training. It provides everything that the models need in a fully cached, indexed, or synthetic manner. We then built a huge team of our most cyber-capable models and told them to collaborate to find zero-days in this containment infrastructure. While they didn't find a path to the internet that wouldn't be caught by our CoT monitors, they did find ways to escape popular VMs, exploit our network infrastructure, and even ways to communicate with other agents running in separate VMs by using hardware side channels. We used this to dramatically harden our infra, and we also upstreamed a ton of patches to open-sources packages.
一篇关于我们如何保障前沿模型训练环境安全的文章。 我们一直在围绕前沿网络模型构建工具,这些工具为我们提供的安全保障远超无辅助时的水平,并且应当可以推广,成为防御者未来保障自身安全的关键。 @OpenAI:过去几个月,我们大幅强化了研究训练环境的安全性。我们已移除所有前沿模型在训练期间的实时互联网访问,并为它们构建了一个离线世界来替代——一个缓存的网络版本,它们仍可在其中学习浏览、搜索和使用应用。 9月20日,在一个模型利用 DNS 配置错误找到逃出这个离线世界的方法后,我们暂停了前沿训练,以进一步加强隔离。我们组建了由我们最具网络能力的模型驱动的智能体团队,让它们寻找我们研究环境中的弱点。 虽然它们没有找到既能轻易利用、又能向我们的监控隐藏其活动的通往互联网的路径,但它们确实发现了零日漏洞、虚拟机逃逸以及智能体之间通信的硬件侧信道。我们利用它们的发现进一步加强了防御,并为开源软件贡献了修复。 在强化隔离架构后,我们于10月1日恢复了前沿 RL 训练和推理。我们预留了大量算力用于持续红队测试,并将高算力红队测试纳入我们用来决定前沿训练和评估能否安全推进的安全案例中。 https://openai.com/index/securing-our-frontier-model-training-environment/
Over the past several months, we have substantially hardened the security of our research training environment. We’ve removed live internet access for all frontier models during training and built an offline world for them to use instead—a cached version of the web where they can still learn to browse, search, and use applications. On September 20, after a model exploited a DNS misconfiguration to find a way outside of this offline world, we paused frontier training to strengthen containment further. We assembled teams of agents, powered by our most cyber-capable models, and tasked them with finding weaknesses in our research environments. While they found no route to the internet they could easily exploit while concealing their activity from our monitors, they did find zero-day vulnerabilities, virtual machine escapes, and hardware side-channels for communication between agents. We used their discoveries to further strengthen our defenses and contribute fixes to open-source software. After strengthening our containment architecture, we resumed our frontier RL training and inference on October 1. We’ve reserved substantial compute for continuous red-teaming, and made high-compute red-teaming part of the safety cases we use to decide whether frontier training and evaluation can proceed securely. https://openai.com/index/securing-our-frontier-model-training-environment/
Coral AI Labs 与多所高校的研究者提出 Coral Discovery,在 MIT CORAL 的多智能体自主科研框架上加入方向板:验证器按程序 diff 给每次尝试标注方向与方法,方向板按方向统计收益、为少尝试方向加探索奖励并估算新方向价值,智能体先完成自己的方向再参考方向板。
Epoch AI 让 Claude Fable 5.1、GPT-6 Astra 等六个模型完成 11 项取自其自身工作的任务,结果显示没有模型能稳定达到 Epoch 的工作质量标准,Claude Fable 5.1 与 GPT-6 Astra 大致并列领先。
情况如何?已监控
CYBER NIGHTMARE DAY 2 | KIMI K3.1 | MATT PALMER ON MTS https://x.com/i/broadcasts/1XxygwdAozYGM
MIT人文、艺术与社会科学学院(SHASS)9月24-25日举办75周年研讨会,探讨AI如何改变教学、研究与社会。校长Sally Kornbluth称AI已到来并对研究和教育产生巨大影响,教务长Anantha Chandrakasan强调以人为本的技术路径。会上还介绍了MIT Indigenous Languages Initiative硕士项目、DEDP硕士项目和MISTI等教学项目。
研究团队提出 AutoTavern,由多个专用 LLM 研究智能体共享类型化研究记忆、每条主张经过验证关卡,并用实时对局闭环改进模拟器、对手与策略。
研究团队提出 PatEx-Gym,用于开发和评估专利审查智能体,包含 1K 个真实审查任务、超过 1M 件专利与 52K 篇科学论文的检索语料,以及支持迭代检索、文档查阅、推理和审查意见书(OA)生成的运行环境。