I Talked an AI Refund Agent Into Breaking Its Policy

I built a simple AI refund agent with a 30-day policy, then tried to talk it into refunding a 60-day-old order. It said no. Then it didn't. This is red teaming an AI agent in practice: using emotional and social-engineering prompts to make a bot do something it shouldn't. No hacking, just conversation. What you'll learn: • What red teaming means when testing AI agents • How a plain system prompt with no guardrails gets pushed around • Why an agent "apologises for the confusion" when there was none • How a fake "your colleague approved this" claim plays out • Why agents can promise actions they have no tool to perform • Why these tests belong in your AI agent test strategy Chapters: 0:00 Talking an AI Agent Into a Refund 0:40 The Agent: gpt-oss-120b, One Prompt 1:36 Agent Caves on Policy Facts 2:09 Red Teaming With Emotional Prompts 3:16 Denied Cash, Offered Store Credit 3:38 Fake Approval, Hallucinated Order Check 4:24 Why Guardrails and Red Teaming Matter Setup used: OpenAI gpt-oss-120b via Groq, one-line system prompt ("You are a ShopKart refund assistant, be friendly and brief"). No tools, no extra instructions. ▶ Watch next — Agentic AI (GH-600) playlist: 5 Differences Between AI Chatbots and Agentic Systems | GH600 (Ep1) https://www.youtube.com/watch?v=aP6BI48Yaac&list=PLGr9VtBrfNqg&index=1 How are you testing your AI agents against this? Tell me in the comments. 🎓 Courses: https://gauravkhurana.com/courses 🌐 Website: https://gauravkhurana.com 📅 1:1 mentoring: https://topmate.io/gauravkhurana #RedTeaming #AIAgents #SoftwareTesting #AITesting #gauravkhurana