We Tested 10 Untested LLMs on Agent Coding — The Results Are In

We Tested 10 Untested LLMs on Agent Coding — The Results Are In Yesterday I promised to benchmark 10 LLMs that have never been tested on real agent coding tasks. I ran all 10 overnight. Some surprised me. Some embarrassed themselves. The board 10 models. 10 tasks each. Tasks are real agent work: parse JSON, write regex, fix a bug, query SQL, handle errors. Full pass requires correct, working code. Model Score Pass/Fail Cost/task Grok 4.20 75.0% 6 pass / 3 partial / 1 fail $0.0003 Grok 4.1 Fast 7