I built the first open benchmark for federal contracting AI. Here's what it shows about frontier LLMs.

If you ask GPT-4o or Claude to extract Federal Acquisition Regulation clause numbers from a federal solicitation, a non-trivial fraction of the time they will hand you a number that does not exist. There is no FAR 52.999-99 . The model just made it up. For a federal contractor staffing a proposal, that is the difference between a clean compliance matrix and a rejected bid. I went looking for a benchmark that measured this. There isn't one. Commercial tools in the space — Capture2Proposal, GovTri