Backtest Your Agent Against Yesterday's Web — Ryan Ahern, Exa

How do you know whether your agent would have made the right call? Ryan Ahern of Exa shows how to use version-controlled search to see the web exactly as it was on a past date, then backtest an agent's predictions against what actually happened. The hands-on example predicts company acquisitions from signals like leadership turnover, then tunes the system without leaking future information. Chapters: - 00:00 Why search matters for agents - 04:15 Exa's search benchmark - 06:17 Versioning the whole web - 10:34 Avoiding look-ahead bias - 14:43 Backtesting and tuning a predictor - 18:49 Hands-on: predicting acquisitions - 22:54 Results and overfitting Ryan Ahern — https://exa.ai Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!