12 minute read

“Your TTS vendor’s latency number is a lie. Here’s how to read the fine print.”

TL;DR

Several TTS systems now claim first audio under 100 milliseconds: Cartesia Sonic Turbo at 40ms, VoXtream2 at 74ms, ElevenLabs Flash v2.5 at about 75ms, Qwen3-TTS at 97ms (0.6B) and 101ms (1.7B), and Inworld TTS Mini and Rime Mist v2 under 100ms by their vendors’ figures. Hume AI’s TADA is fast on throughput but has no published first-packet number. Each trades off differently across latency, quality, cost, and openness. The real story: when TTS stops being the bottleneck, the rest of your voice agent pipeline becomes the constraint. Most architectures were designed around slow TTS. They need rethinking.

A precision digital stopwatch frozen at 00:00.097 sitting next to a tiny speaker driver with visible sound waves eman...

Why does sub-100ms TTS matter now?

Because voice agents live or die on perceived responsiveness, and the math is unforgiving.

Humans expect conversational turn gaps of 200-400ms. The cross-linguistic mean is about 200ms, but the range is wide: 7ms mean in Japanese, 469ms mean in Danish (Stivers et al., 2009, PNAS). Past 800ms, callers notice. Past 1.5 seconds, the conversation degrades. Past two seconds, it feels broken.

A voice agent’s round-trip includes three serial steps: speech-to-text (50-200ms), LLM inference (200-800ms), and text-to-speech. If TTS takes 300-500ms, you have already blown past the 800ms threshold before accounting for network latency, queue time, or audio encoding. The agent sounds hesitant. Users talk over it. Completion rates drop.

Voice agents only work commercially if they feel conversational, not robotic. Sub-100ms TTS helps by giving the LLM step breathing room within the perceptual budget.

Until recently, only proprietary APIs claimed sub-100ms first audio. By early 2026, open-weight models compete. That changes the economics and the architecture.

How do you actually measure TTS speed?

Three numbers circulate in TTS marketing, and they measure different things.

Metric What it measures Example
RTF (real-time factor) Total inference time / audio duration. Below 1.0 = faster than playback. RTF 0.1 = 10x faster than real time
TTFB (first-packet latency) Time from input to first playable audio chunk. Milliseconds. VoXtream2: 74ms on an RTX 3090
End-to-end latency TTFB + network + queue + encoding. What the user actually experiences. Inworld TTS Mini: <130ms P90

RTF is a throughput metric. A system with 0.09 RTF generates 100 seconds of audio in 9 seconds of compute, but it might buffer for 200ms before the first chunk arrives. TTFB is a responsiveness metric. A system with 74ms TTFB starts playing audio quickly, even if it generates the full clip slower overall.

For voice agents, TTFB matters more than RTF. Users hear the first syllable, not the last. A system that starts fast and streams continuously sounds responsive even if its total generation time is higher.

End-to-end latency is what callers actually experience. It includes everything TTFB does not: network round-trips, request queuing, audio codec overhead, and WebSocket framing. A 40ms TTFB in a benchmark can easily become 120ms or more in production. Always measure end-to-end.

sequenceDiagram
    participant User
    participant STT
    participant LLM
    participant TTS
    participant Speaker

    User->>STT: Speech input
    Note over STT: 50-200ms
    STT->>LLM: Transcript
    Note over LLM: 200-800ms (streaming)
    LLM->>TTS: First text chunk
    Note over TTS: TTFB: 40-100ms
    TTS->>Speaker: First audio packet
    Note over Speaker: User hears response
    Note over User,Speaker: Total round-trip target: < 800ms

How does TADA get its speed?

Most TTS systems suffer from a modality mismatch: one text token maps to a variable number of acoustic frames. A single word like “extraordinary” might span dozens of mel-spectrogram frames, while “cat” spans three. Traditional models handle this with attention mechanisms that occasionally skip words or hallucinate, inserting sounds that have no corresponding text.

TADA (Text-Acoustic Dual Alignment) from Hume AI eliminates this by enforcing one-to-one alignment between continuous acoustic vectors and text tokens. Each autoregressive step covers exactly one text token. The model dynamically determines duration and prosody per token, but it cannot skip or insert because the alignment is structural, not learned.

The results: Hume reports zero transcription hallucinations (no skipped or invented words) across more than 1,000 test samples, and about 700 seconds of audio context. The arithmetic explains the context gain: at about 3 text tokens per second of speech, a 2,048-token window holds roughly 11 minutes, while a codec at 25-50 frames per second fills the same window in 40-80 seconds. One step per text token is also why TADA is fast: there are far fewer autoregressive steps per second of audio.

The architecture builds on Llama 3.2, available in 1B and 3B parameter variants. The code is MIT-licensed; the model weights use the Llama 3.2 Community License. Released March 10, 2026 on GitHub with weights on Hugging Face.

Hume’s repo cites a real-time factor of about 0.12 on an H100 with cached prompts (some reviews quote 0.09), and Hume says TADA is more than five times faster than comparable LLM-based TTS. What Hume has not published is a first-packet (TTFB) measurement. RTF and TTFB can diverge significantly depending on buffering strategy, so treat the RTF as a throughput figure, not a responsiveness guarantee.

What makes VoXtream2’s 74ms first-packet possible?

VoXtream2 (arXiv 2603.13518) attacks latency at every pipeline stage simultaneously.

The architecture has three transformers and a streaming codec, each designed to minimize waiting:

  1. Incremental Phoneme Transformer: processes text as it arrives, with a look-ahead of up to 25 phonemes (at least 3 are needed for intelligible output). It starts generating audio before the full sentence has arrived.

  2. Temporal Transformer: autoregressively predicts semantic tokens and duration tokens. Duration is predicted alongside content, not in a separate pass.

  3. Depth Transformer: predicts the remaining acoustic tokens for each frame. It is frozen and based on Sesame CSM, and it supports zero-shot voice cloning from a reference clip.

  4. Mimi codec: decodes the 12.5 Hz audio tokens to a 24 kHz waveform in a streaming fashion.

The combined result: 74ms first packet on an RTX 3090 with CUDA Graphs and an RTF of 0.256, about 4x faster than real time. With torch.compile, the paper reports 63ms and an RTF of 0.173. The model is English-only, trained on about 40,000 hours (30K from Emilia, 10K from HiFiTTS-2). It supports speaking-rate changes mid-utterance, so you can slow down for emphasis or speed up for lists without regenerating.

The original VoXtream (arXiv 2509.15969) is close on latency: in VoXtream2’s own comparison it reaches 78ms with torch.compile. The headline addition in VoXtream2 is dynamic speaking-rate control. Code, pretrained weights and audio samples are linked from herimor.github.io/voxtream2.

Where does open-source Chatterbox fit?

Not in the sub-100ms race. Resemble AI publishes no first-chunk latency for the open Chatterbox models; its “sub 200ms” claim refers to its paid hosted service. Its value is elsewhere.

Chatterbox is a family of MIT-licensed models: the original English model and Chatterbox-Multilingual (both 500M parameters, the latter covering 23 languages), and Chatterbox-Turbo (350M, English). Resemble reported that 63.75% of evaluators preferred the original Chatterbox over ElevenLabs in a blind test it ran on the Podonos platform; that is a vendor-run evaluation.

The standout features are emotion control (the original model has an exaggeration setting) and paralinguistic tags in Turbo ([cough], [laugh], [chuckle]). Every output carries Resemble’s Perth watermark for provenance tracking. A community-built Chatterbox-TTS-Server adds a Web UI and an OpenAI-compatible API for self-hosting.

One change since this post was written: as of August 2026, Resemble lists Chatterbox as retired research that is no longer maintained. The weights remain usable under MIT, but plan for no upstream fixes.

For voice agents where latency is the binding constraint, Chatterbox is the wrong tool. For applications where multilingual coverage, emotion expressiveness, and self-hosting economics matter more than first-packet speed (IVR prompts, audiobook generation, content localization), it remains a solid open option if you can maintain it yourself. See self-hosting TTS production economics for the cost analysis.

How do the sub-100ms systems compare?

Every number below comes from the vendor’s own published benchmarks or paper. “Open” means weights are publicly available. Hardware column shows what the latency was measured on, where stated.

System TTFB RTF Open Params Hardware Best for
Cartesia Sonic Turbo 40ms n/a No n/a Cloud (SSM) Absolute minimum latency
Rime Mist v2 <100ms n/a No n/a On-premise Enterprise, pronunciation accuracy
VoXtream2 74ms 0.26 Yes n/a RTX 3090 Open weights + low latency (English)
ElevenLabs Flash v2.5 ~75ms n/a No n/a Cloud 32 languages, reliability
TADA (Hume AI) n/a ~0.12 Yes 1B/3B H100 No skipped words, long-form
Qwen3-TTS 97ms (0.6B), 101ms (1.7B) 0.29-0.31 Yes 0.6B/1.7B not named Multilingual, voice cloning
Inworld TTS Mini <100ms (median) n/a No n/a Cloud Gaming, interactive apps
Chatterbox Turbo not published n/a Yes 350M n/a English, paralinguistic tags; retired Aug 2026

Two architectural patterns dominate the sub-100ms tier:

State Space Models (Cartesia): an SSM keeps a fixed-size recurrent state, so the cost of each generation step does not grow with the sequence the way attention does. Cartesia credits this for its 40ms figure. Transformer-based systems are not far behind (74-101ms above), so serving stack and hardware matter as much as the architecture.

Incremental, aligned generation with streaming codecs (VoXtream2, TADA): VoXtream2 starts speaking after a few phonemes of look-ahead instead of waiting for the full sentence, and TADA ties each generation step to one text token. Paired with a codec that decodes audio frame by frame (Mimi in VoXtream2’s case), the pipeline delay collapses.

What breaks in your pipeline when TTS gets this fast?

When TTS took 300-500ms, voice agent architectures could afford sloppy handoffs. Audio buffers absorbed jitter. LLM streaming could stall for 200ms without anyone noticing. Those margins disappear at sub-100ms.

The LLM becomes the bottleneck. If TTS starts generating 40-74ms after receiving text, but the LLM takes 400ms to produce the first token, your user is waiting for the LLM, not the TTS. Time to first token becomes the number to optimize: smaller or faster models, prompt caching, and sending the first sentence fragment to TTS as soon as it exists. See streaming speech pipelines for pipeline design.

Audio buffer strategies need rethinking. Many voice agents buffer 500ms-1s of TTS audio before playback to smooth out jitter. With sub-100ms TTS, that buffer adds more latency than the TTS itself. You need either smaller buffers (50-100ms, accepting occasional underruns) or adaptive buffering that shrinks as TTS proves reliable.

Barge-in detection becomes time-critical. When the agent responds in under 200ms total, the window for detecting user interruption shrinks proportionally. VAD and barge-in systems designed for 500ms+ response times will either cut the agent off too early or too late. The VAD architecture needs tighter coupling with the TTS output stream.

Transport and packetization matter. At 40ms TTFB, buffering 20ms of audio per packet adds half the TTS time again before the first packet leaves. Small audio packets, binary WebSocket frames, pre-negotiated codecs, and warm connections shift from nice-to-have to required.

Key takeaways

  • Measure TTFB, not RTF, when evaluating TTS for voice agents. RTF tells you throughput; TTFB tells you how fast the user hears the first syllable.
  • Sub-100ms first audio is real, at least by published figures, spanning proprietary (Cartesia 40ms, ElevenLabs ~75ms) and open-weight systems (VoXtream2 74ms, Qwen3-TTS 97-101ms).
  • Chatterbox wins on breadth, not speed. 23 languages in the multilingual model, emotion control, MIT license, but no published first-chunk latency, and Resemble retired the project in August 2026.
  • TADA’s alignment design ties each generation step to one text token, which rules out skipped or inserted tokens by construction; Hume reports zero transcription hallucinations across 1,000+ samples.
  • When TTS gets fast, everything else becomes the bottleneck. LLM inference, audio buffering, barge-in detection, and WebSocket framing all need tightening.
  • Open weights crossed the threshold. VoXtream2 at 74ms and Qwen3-TTS at 97-101ms mean you no longer need a proprietary API for sub-100ms TTS, if you can run the GPUs.

Further reading

Related posts