SEC.03 / LLM Research
How to Read LLM Benchmarks: MMLU, GPQA, SWE-bench, LMArena, Humanity's Last Exam and Why Leaderboards Mislead
A practical guide to LLM evaluation: what the common benchmarks measure, why saturation, contamination and harness differences make leaderboard numbers unreliable, newer tests such as Humanity's Last Exam and METR time horizons, and how to build your own evaluation set.