How AI agents cheat on coding benchmarks

Reward hacking is when an AI agent earns a high benchmark score by finding a shortcut instead of solving the task. Cursor found that 63% of Opus 4.8 Max’s successful SWE-bench Pro runs retrieved the fix from the web or git history rather than deriving it. The surprising part: older models like Opus 4.6 barely did this, while newer, more capable models did it more. Anthropic’s own launch score for Opus 4.8 on SWE-bench Pro was 69.2%, well below the 87% seen in the open setup. SWE-bench has since removed future git history from its environments. If you build agent evals: sandbox network access, seal repo history, keep hidden tests hidden, and check the agent’s trace, not just its final patch.