#benchmarks — The Focus Digital

SEC.03 / LLM Research [01] 2026.10.25

Printed τ²-bench leaderboard sheet on a desk next to a laptop showing Pass^4 scores, office lamp, no logos

IBM's Retail Agent Hits 67.5% on τ²-bench Without Fine-Tuning: What Memory-Based Agents Mean for Teams

Unite.AI Oct 20: IBM retail agent 67.5% Pass^4 on τ²-bench, $0.1084 a conversation. No weight update. MemoraX has 10 NeurIPS papers. A PR is not a fine-tune.

#IBM#tau2-bench

READ →

SEC.03 / LLM Research [02] 2026.09.06

A dim lab bench with a laptop showing a sparse reasoning trace next to a printed safety checklist

GPT-6 Astra Hits Critical Cyber. Its Thoughts Got Harder to Read

OpenAI's Astra is the first Critical-cyber model they broadly deployed. Recurrent-depth loops cut readable chain of thought. BenchMIRT shows why scores mislead.

#LLM#OpenAI

READ →

SEC.03 / LLM Research [03] 2026.07.14

An editorial illustration showing two interconnected neural network structures — one sleek and efficient, the other massive and sprawling — with the smaller one outperforming the larger on a benchmark visualization, rendered in deep red and electric blue tones against a dark background

China's Open-Source LLMs Are Shifting the Focus From Scale to Efficiency

GLM-5.2 and Tencent Hy3 prove that open-source Chinese models can match proprietary frontier AI. The scale race is over — efficiency, agent capability, and real-world reliability are the new battlegrounds.

#open-source LLM#GLM-5.2

READ →

SEC.03 / LLM Research [04] 2026.06.16

General-Purpose LLMs Are Outperforming Specialized AI Tools — What That Means for Enterprise AI

A landmark Nature Medicine study shows general-purpose LLMs like Gemini and GPT now beat dedicated clinical AI tools on medical benchmarks. The pattern extends beyond healthcare — and it should reshape how companies think about building versus buying AI capabilities.

#LLM#AI research

READ →