Building AI Agents You Can Trust & Evaluate | Day 2 Tracing & Observability for Agentic RAG

Most AI workshops end when the demo works. This series starts there. In Day 1, we built a working Agentic RAG support assistant with retrieval, tool calling, order lookup, ticket creation, OpenAI/Ollama support, ChromaDB, and multi-turn conversations. 📺 Missed Day 1? Watch here: https://youtube.com/live/_44x_yL7Q-s?feature=share In Day 2, we open the black box. We'll instrument the agent with tracing and observability so we can see exactly what happens between a user question and the final answer. Topics we'll cover: ✅ Agent tracing fundamentals ✅ Retrieval visibility ✅ Tool-call tracing ✅ Prompt and response inspection ✅ Session and conversation tracing ✅ Latency and execution tracking ✅ Structured observability for AI systems ✅ Foundation for evaluation-driven development The goal is simple: Build behavior first. Then make behavior visible. Because if we cannot see behavior, we cannot evaluate it. And if we cannot evaluate it, we cannot trust it. Prerequisites Completed Day 1 or familiar with the Day 1 repository Python 3.11+ VS Code (or any Python IDE) Git OpenAI API key or Ollama GitHub Repository https://github.com/toni-ramchandani/Agentic-RAG-and-Evaluation Series Roadmap Build → Observe → Analyze → Measure → Judge → Gate → Monitor Day 2 Focus: Observe Community Partners QABash https://qabash.com/ Indian Data Club https://www.indiandataclub.com/ What to Expect Live coding Real engineering discussions Interactive Q&A Recording available after the session ☕ Coffee. Code. Learn.