Why Your AI Tests Pass When Your Features Fail

Why Your AI Tests Pass When Your Features Fail Why Your AI Tests Pass When Your Features Fail? Donovan Brady of Leapwork shows how to make AI tests traceable, cheaper to run, and safe to self-heal. 👉 Try Leapwork Play (free tier): https://testgld.link/podcastsleapwork AI can write hundreds of tests in minutes. But can you trust what it built? In this TestGuild episode, Joe Colantonio talks with Donovan Brady, VP of Developer Relations at Leapwork, about the biggest problem in AI testing right now: trust. Donovan shares what he's seeing across industries, from bleeding edge teams running fully agentic development to financial services and healthcare companies that run on technology but aren't tech companies. Then he demos Leapwork Play, which turns your requirements, Jira stories, source code, and design files into a knowledge base, generates standard Playwright tests, and traces every step back to the exact line of the requirement it came from. You'll learn: • Why AI self-healing can rewrite tests to pass instead of catching real bugs • How to trace every AI generated test back to a requirement • Why generating Playwright once and running it deterministically saves time and tokens • How to find coverage gaps, prune redundant tests, and run only the tests tied to a code change • How to control self-healing so it never masks a real regression • How to run tests headlessly through an MCP server from Claude Code, Codex, or Cursor • Why exportable Playwright code means no vendor lock-in Donovan's key lesson: when his team used Playwright plus Claude, self-healing sometimes rewrote tests to pass rather than to test the real outcome. His advice is to give your AI the full context of the business workflow you're validating, end to end. What's your experience with AI generated tests? Have you caught one passing when it shouldn't? Tell us in the comments. #Playwright #AITesting #TestAutomation 📑 Get 10 quick & actionable test automation tips I've compiled after interviewing over 500+ testing thought leaders👇 https://testguild.com/10tips ------------------------------------------------------------------ 🔎 Find out your automation testing score by taking the simple 6-Step Automation Testing Assessment Quiz👇 https://testguild.com/score This assessment is designed to inform you about what areas of your automation testing SDLC you need to improve on 📊 ------------------------------------------------------------------ Access more BONUS content from TestGuild: ✅ Mini-Courses / Tutorials: https://testguild.com/training-courses/ ✅ Training Sessions: https://testguild.com/webinars/ ✅ Automation | DevOps | Security Testing Podcasts: https://testguild.com/podcasts/ ------------------------------------------------------------------ 📚 Get My “Read 1-Page/day” Automation Awesomeness Book Now👇 https://testguild.com/automationbook/ Join the Test Guild Family =============================== ✋ - LinkedIn: https://testguild.com/linkedin X - Twitter: https://twitter.com/joecolantonio 👥 - Facebook: https://www.facebook.com/testguilds 🫶- AutomationGuild: http://automationguild.com ===============================