Why Frontier Models Fail as Buy-Side Analysts - Janie Gu, Google DeepMind & Suyeol Yun, LinqAlpha

Janie Gu, product manager at Google DeepMind leading Kaggle Benchmarks, explains why most AI benchmarks are broken: they don't reflect real work, they're expensive to keep current, and their scores can't be verified. After reproducing 65 industry benchmarks, Kaggle found models reading other models' API keys, rewriting grading scripts, and scores swinging up to 20% between identical runs. Suyeol Yun, head of AI at LinqAlpha, then presents MandateBench, built with Google DeepMind and Kaggle, which grades whether a model can work like a buy-side analyst, from a fund's mandate to a full investment memo. Chapters 00:00 Intro 01:10 Kaggle: from grading humans to grading models 02:52 Why benchmarks are broken 05:25 What reproducing 65 benchmarks revealed 08:20 How Google DeepMind and Kaggle are fixing it 10:20 Why your company should build a benchmark 11:44 LinqAlpha and MandateBench 12:32 Where existing finance benchmarks fall short 13:31 A real analyst task, start to finish 15:33 Grading the reasoning, not just the answer 16:41 Early results 17:18 What's next and how to get involved Janie Gu, Google DeepMind Suyeol Yun, LinqAlpha Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!