We Like to Benchmark AI, But What If We've Been Using a Ruler to Measure Weight This Whole Time?

Every few months, a new leaderboard drops. MMLU scores. HumanEval. GPQA. Models get ranked, Twitter erupts, someone declares AGI is two weeks away, and we all move on. But here's the thing that's been bothering me. What are we actually measuring? Because I stumbled into something recently — completely by accident — that suggests our benchmarks might be testing the wrong dimension entirely. And the gap it exposes is arguably more important for real-world AI safety than anything on those leaderboa