Latency is the whole game in voice AI

Ask someone who has spoken to an AI phone agent what gave it away, and they will almost never say the voice. Synthetic speech crossed the believability threshold some time ago. What they notice is the pause.

In human conversation, the gap between one person finishing and the next starting averages around 200 milliseconds. It varies by language and culture, but the tolerance is narrow. Past roughly a second, the silence stops being a pause and starts being a problem — the caller assumes they were not heard, and talks again.

Where the time goes

A voice turn has four stages: detecting that the caller stopped, transcribing what they said, deciding what to say back, and producing audio. Done naively, in sequence, each stage waits for the previous one to finish and you are well past two seconds before the agent makes a sound.

The fix is that none of these stages should wait. Transcription streams as the caller speaks. Inference begins on a partial transcript. Speech synthesis starts on the first clause of the response rather than the complete sentence. The agent starts talking while it is still deciding how the sentence ends — which is, incidentally, exactly what humans do.

Interruption is not an edge case

Insurance calls are frequently made by people having a bad day. They interrupt. A system that keeps talking over an upset caller is worse than useless, and one that stops but loses its place is not much better.

Barge-in has to stop output immediately, discard the unsaid remainder, keep everything already established in the conversation, and fold the interruption into context. Get this wrong and every difficult call — which is to say, every call that matters — falls apart.

What we target

Under 1.5 seconds to first word, measured from the caller finishing speaking, at the 95th percentile rather than the median. Medians hide the calls that go wrong, and the calls that go wrong are the ones people remember.