全部动态

It was quite nostalgic to attend Emerging Science, Technology and Innovation Conclave @estic_2026 at Bharat Mandapam. Last year around the same time, in the first week of November, I was involved in organizing the first edition -ESTIC 2025 and had spent several sleepless nights. Today morning, I chaired the Session on Plenary talk by Dr Pushmeet Kohli, Chief Scientist Google Clouds and VP at Google DeepMind. Dr Pushmeet delivered a wonderful talk on possibilities that AI has opened in biology to materials science. He also talked about breakthroughs such as AlphaFold. Most importantly, he underscored the fact that AI can certainly help all scientists in tasks such as data collection and curation, literature survey and even new hypothesis generation or experimental investigation planning. It is then upto a scientist to use this to perform her research. There was an equally stimulating session on Physical AI with an interesting perspective from industry by Jalaj Dani, Chairman of @Addverb_Tech. I also released DST’s in-house Dream magazine with special feature on Physical AI. @DaniJalaj @pushmeet @IndiaDST

Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers. Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale. New to the Intelligence Index: ➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change ➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment Upgraded: ➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions ➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised ➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0 Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1 Key results: ➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50 ➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36) ➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04

Our latest open-source research has a pretty interesting name, inspired by the story of the Tower of Babel. What interests us about that story is how language barriers can get in the way of people working toward the same goal And when it comes to agents, there are a few more places for that understanding to break, like tool arguments, database entries, and search queries One example from BabelArena (yes that's the name, surprise) stuck with me: Qwen-3.8-Max completed the same shopping task in English and Telugu, getting the same four products. But it took 25 tool calls in English and 201 in Telugu, with repeated empty searches against the localized catalog. Even though both runs passed, we’d still want to see that difference before shipping. Sorry for the long story, hope you guys are still here. What I want to share is that we’ve open-sourced BabelArena to study these workflows across 23 languages. Code, datasets, and paper in the comment