Long-Horizon Task Evaluation
Long-horizon task evaluation measures how an AI agent performs on tasks that require many sequential steps or a long span of time to complete, where small per-step error rates compound and early mistakes can derail everything that follows — a different failure profile than single-turn question answering.
A model with a 95% success rate on any individual step still fails most 20-step tasks overall, since the errors compound multiplicatively — this is the core reason long-horizon performance can look dramatically worse than a model's benchmark scores on isolated questions would suggest, and it's the specific gap this kind of evaluation is designed to surface.
Good long-horizon tests deliberately include tasks with no reliable shortcut — ones that genuinely require staying on-goal across many steps — and check not just final success but where in the sequence failures cluster, since an agent that reliably fails at step 12 of 20 has a different (and more fixable) problem than one that fails randomly throughout.
Recovery matters as much as avoidance: does the agent notice when an earlier step went wrong and correct course, or does it confidently continue building on a mistake it never detects?
