If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com

Ramakrishnan Lokanathan, an AI safety researcher and former startup founder, lays out how to build evals for high-stakes AI applications. Define the most ambitious version of success, test every step of the workflow and not just the final answer, combine automated checks with human review and adversarial tests, keep testing what used to work so upgrades don't break it, and turn real user failures into new tests. He also introduces eSlams, an open-source package that has AI models play games like chess, Go and poker against each other as another way to see how they reason. Chapters: 0:00 Intro 0:37 What is an eval, really? 2:58 About me 4:51 Why evals matter now 5:34 1. Define the most ambitious success 6:43 2. Test the whole workflow 7:28 3. One score is not the system 8:52 Games as evals: eSlams 12:06 4. Keep testing what used to work 13:35 5. Turn real failures into tests 15:08 Recap Ramakrishnan Lokanathan https://eslams.com Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!