We Kept Rebuilding Our AI Agent. Here's What Survived - Fuad Ali, Arize AI

Arize's in-product agent Alex went from a 2023 prototype that was "50% hallucination" to a single agent with planning, memory and on-demand skills, all built with custom harness code instead of a framework. AI PM Fuad Ali covers each decision: moving the plan into the harness, making "finish" an explicit gate, treating large tool results like files, why one 8,000-character memory file per user beat vector stores and knowledge graphs, merging 10 specialists into one agent, and using production sessions as tests. Chapters: 0:00 Intro 0:53 Agents all the way down 2:56 What Alex does 4:15 Timeline: 2023 to 2026 6:19 Who owns the plan? 8:33 What ends a turn? 9:44 Memory: treat big results like files 12:10 One small file beat a knowledge graph 13:33 Specialists vs. one agent 14:13 The newest model isn't always best 15:28 Production sessions as tests 17:46 What still needs a human 18:31 What survived every rebuild Fuad Ali https://arize.com Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!