#swe-benchmarks

AI Coding Agents on Real Enterprise Codebases: Why 70%+ Public Scores Collapse to 16–39% (Real-SWE / SWE-bench Pro)

The best coding agents clear 70%+ on public benchmarks but resolve at most 38.8% of tasks on authorized private production codebases — and as little as 16.2%. The gap isn't model strength; it's private context. Here's what the newest real-world benchmarks measure, why leaderboard scores don't transfer, and what it takes to make an agent useful inside your own code.