Stop Fine-Tuning to Fix Retrieval Problems — Anant Srivastava

Most AI teams never decide where their knowledge lives. Six months of normal product work decides for them. Anant Srivastava, principal technologist for data and AI platforms at Oracle, argues that prompt, memory and weights aren't a ladder you climb when answers go wrong. They're three tools for three jobs. Through a support assistant that kept inventing product names after a fine-tune, he shows how architecture builds up by accident. He then gives a simple diagnostic for each: the prompt is for small, stable behavior; memory is for knowledge that's current, large, citable or access-controlled; weights are only for reflexes that have stopped changing. He closes with a circulating architecture where the agent gets better by doing its job. In this talk: • How every prompt edit, indexed doc and training sample is an architecture decision • The diagnostic question for prompt, memory and weights • Why you fine-tune reflexes, not facts • Moving knowledge between context, memory and weights over time SPEAKER Anant Srivastava, Principal Technologist, Data and AI Platforms, Oracle LinkedIn: https://www.linkedin.com/in/anantds/ CHAPTERS 0:00 Where should knowledge live? 0:15 The real engineering problem 0:50 Three tools, not a ladder 1:20 How teams decide by accident 2:00 Every edit is an architecture decision 2:40 A support assistant goes wrong 4:34 Accumulated, not designed 5:14 Prompt: behavior, not facts 5:59 Example: a support agent prompt 6:59 Memory: current, large, citable 8:09 The wrong job for memory 8:49 Access control belongs in memory 9:14 Example: a code assistant 10:08 Chunking and metadata for code RAG 11:23 Weights: what has stopped changing 12:28 The fine-tuning mistake 13:53 When fine-tuning works 15:08 Medical coding: fine-tune the reflex 16:13 Capability or cost? 16:58 The decision table 17:33 A circulating architecture 19:27 The model is the easy part Recorded at the AI Engineer World's Fair 2026 in San Francisco. Subscribe for more talks from the engineers building with AI. AI Engineer: https://ai.engineer YouTube: https://www.youtube.com/@aiDotEngineer X: https://x.com/aiDotEngineer LinkedIn: https://www.linkedin.com/company/aidotengineer/ #RAG #FineTuning #AIEngineer