The Transcript Looked Fine. The Call Wasn't. — Debugging Voice Agents, Arize

The transcript says the agent corrected itself. The audio says it refunded the wrong order. Fuad Ali, senior product manager at Arize AI, shows why voice agents are one of the hardest categories to debug: latency, turn-taking and transcription errors are invisible in a text log. Using a refund call that looks fine in the transcript but had 2.4 seconds of dead air, an interruption and a refund on the wrong order, he shows what you need instead. That means audio, transcript and trace in one session view, OpenInference semantic conventions for real-time audio across providers, and evals that run on the audio itself. He closes with the observe, evaluate, improve loop and agent experiments that test fixes against failed traces. In this talk: • Voice failure modes text logs miss: latency, barge-in, mishearing and tone • Tracing real-time audio with open-source OpenInference conventions • Audio-native evals for sentiment, latency, interruptions and task success • Toward self-healing voice agents: monitors, investigation agents and replayed fixes SPEAKER Fuad Ali, Senior Product Manager, Arize AI LinkedIn: https://www.linkedin.com/in/ahmedfuadali/ X: https://x.com/tofuadmiral LINKS Arize AX: https://arize.com/products/ax/ Tracing and evaluating audio (docs): https://arize.com/docs/ax/cookbooks/evaluate/tracing-and-evaluating-audio OpenInference (GitHub): https://github.com/Arize-ai/openinference Arize: https://arize.com/ CHAPTERS 0:00 Intro 0:22 Why voice agents are invisible 1:07 Voice agents are exploding 1:47 Why voice is hard to debug 2:41 The transcript looked fine. The call wasn't. 3:31 What actually went wrong 3:56 See beyond the transcript 4:46 Trace everything 5:41 Audio in OpenInference 6:45 One schema across providers 7:40 Replay the whole conversation 8:30 Demo: a voice trace with a tool call 9:00 The failures, found 9:54 Evals for audio 11:49 Creating audio evals with skills 12:44 Evals on the spans 13:48 Observe, evaluate, improve 14:28 Agent experiments and self-healing 15:53 Start tracing your voice agents Recorded at the AI Engineer World's Fair 2026 in San Francisco. Subscribe for more talks from the engineers building with AI. AI Engineer: https://ai.engineer YouTube: https://www.youtube.com/@aiDotEngineer X: https://x.com/aiDotEngineer LinkedIn: https://www.linkedin.com/company/aidotengineer/ #VoiceAI #LLMObservability #AIEngineer