Evaluation vs Guardrails: Testing AI in Production

How do you keep an AI agent safe once it's live? Microsoft Foundry guardrails block harmful prompts, and observability shows you everything the agent did. This clip is from a live session in my Microsoft Foundry series. Evaluation tests your agent while you build it. Guardrails and monitoring protect it in production. I walk through what Foundry gives you by default, what you can tune, and where testers should look when an agent suddenly starts failing. What you'll learn: - Evaluation vs guardrails: development testing vs production protection - Blocking harmful prompts before they reach the agent (and save tokens) - Content filters, prompt-injection protection, hate-speech thresholds and block lists - Why a 500 error may be a guardrail block, and how traces show the real reason - Observability: tracing, token usage, Application Insights alerts and recurring evals on a golden dataset - Agent identity, roles, Azure Policy and managing users in Foundry Chapters: 0:00 Evaluation vs Guardrails 0:32 Block Harmful Prompts Before the Agent 1:07 Content Filters & Attack Alerts 2:17 Prompt Injection & Block Lists 2:59 500 Errors? Check the Traces 3:33 Application Insights, Alerts & Monitoring 4:45 Recurring Evals on a Golden Dataset 5:14 Agents Get Their Own Identity 6:03 Roles, Azure Policy & Access Control 6:39 Manage Users in Foundry Watch next: ▶ Microsoft Foundry - AI Platform (full playlist): https://www.youtube.com/playlist?list=PLeGiFBPpRC04 Connect: 🎓 Courses: https://gauravkhurana.com/courses 🌐 Website: https://gauravkhurana.com 📅 1:1 mentoring: https://topmate.io/gauravkhurana #MicrosoftFoundry #AIGuardrails #AISecurity #AITesting #SharingIsCaring