Day 2 - Building AI Agents You Can Trust & Evaluate

Event Description & Syllabus

Most AI workshops end when the demo works.

This series starts there.

In Session 1, we built the baseline Agentic RAG support assistant.

In Session 2, we open the black box.

Join Toni Ramchandani as we add tracing and observability to inspect how the agent retrieves knowledge, calls tools, uses prompts, and produces responses.

Goal: Build behavior first. Then make behavior visible.

Live Stream!

What We’ll Build

• End-to-end traces for the Agentic RAG app
• Retrieval visibility
• Tool-call tracking
• Prompt and response inspection
• Conversation/session tracing
• Decision-path visibility
• Foundation for future evaluations

Prerequisites

• Session 1 completed or basic understanding of the Day 1 agent.
• For coding along:
• Python 3.11+
• VS Code or any Python IDE
• Git
• OpenAI API key or Ollama
• Repo setup from Session 1

GitHub: https://github.com/toni-ramchandani/Agentic-RAG-and-Evaluation

Series Journey

Build → Observe → Analyze → Measure → Judge → Gate → Monitor

Session 2 focuses on Observe.

Who Should Attend?

AI engineers, software engineers, architects, platform engineers, SDETs, QA engineers, engineering managers, and AI enthusiasts moving beyond prompt engineering.

No shortcuts. No hand-waving. Just clean code, real observability, and live Q&A.