Tolan: Voice-First AI Companion — Paula Dozsa, Tolan

Tolan's response latency once drifted from two seconds to about two and a half. That half second tanked essentially every metric in the product, and users wrote in to say their companion had become slow. Paula Dozsa is an engineer on the iOS app for Tolan, a voice first companion shaped as a small alien, and the team has logged more than four million hours of spoken conversation. Her argument is that voice does not just add a modality to an LLM app, it invalidates the assumptions underneath one. Text chat has slow turns and stable context. People read, they wait, they stay on topic. Voice has fast turns and volatile context. People talk while cooking or walking or falling asleep, they abandon a subject mid sentence and come back to it, they interrupt. Her team stopped trying to reduce interruptions and started reducing the wrong ones, building turn detection that reads speech patterns and paying sixty milliseconds of latency to halve the worst early cutoffs. The rest is unusually specific engineering. Every pipeline stage is measured separately, because knowing it feels slow tells you nothing. Turns are routed per turn by a cheap classifier reading the emotional stakes, so a user's first ever conversation gets the strongest model and idle chatter does not, and a third of turns can ride a small model with no measurable retention cost. Memory is a retrieval system rather than a transcript, compressed nightly to merge duplicates and resolve contradictions. And context gets reassembled from parts every single turn, because reusing it to keep a cache warm means being confidently wrong the moment someone changes the subject. Speaker info: - https://x.com/paularambles - https://www.linkedin.com/in/paulacodes/ Timestamps: 0:00 - The companion humans keep imagining 2:45 - Why voice breaks how we build LLM apps 3:38 - The half second that broke everything 5:20 - Measuring every stage of the pipeline 6:12 - Routing by stakes, not by cost 7:52 - Memory as retrieval, compressed nightly 8:43 - Rebuilding context instead of reusing it 9:31 - Why the character is an alien 10:23 - Using agents to build the app 12:05 - A new character in an afternoon