Voxtral TTS: what Mistral’s 3-second voice cloning delivers, and when to use it
Zero-shot voice cloning is not new. Microsoft’s VALL-E demonstrated cloning from a 3-second recording in January 2023, and open models such as XTTS followed. What has changed is quality. Voxtral’s paper (arXiv:2603.25551, March 2026) reports that Mistral’s TTS model, cloning from a short reference, beat ElevenLabs Flash v2.5 in 68.4% of blind human comparisons, and it ships with open (non-commercial) weights.
This post isn’t a paper summary. It’s a practitioner decision guide: what Voxtral actually delivers, where it falls short, and how to choose between it, ElevenLabs, Cartesia, and Qwen3-TTS for production work.
TL;DR
Voxtral TTS (arXiv:2603.25551, March 2026) is a roughly 4B-parameter model for 9 languages. It clones a voice from as little as 3 seconds of reference audio and won 68.4% of blind comparisons against ElevenLabs Flash v2.5. Mistral reports 70 ms latency on one H200 at concurrency 1. The API costs $0.016 per 1,000 characters, and the open weights are CC BY-NC. For the full agent audio stack this integrates into, see voice agent architecture.

How Voxtral actually works
Voxtral has three parts. The Hugging Face release is named Voxtral-4B-TTS-2603, so the total is about 4B parameters:
- a decoder backbone based on Ministral 3B, which generates one semantic token per 80 ms frame;
- a flow-matching transformer, a small 3-layer bidirectional model that generates the frame’s acoustic values from the backbone’s hidden state;
- the Voxtral Codec (about 300M parameters), which encodes reference audio into tokens and decodes generated tokens back to 24 kHz audio.
The codec runs at 12.5 frames per second. Each frame has one semantic token from an 8,192-entry VQ codebook, distilled from a frozen Whisper model, plus 36 acoustic dimensions quantized with FSQ (finite scalar quantization). That works out to about 2.14 kbps.
Cloning happens in context. The reference clip is encoded into codec tokens, and the decoder’s input is those voice tokens followed by the text tokens. The model continues in the voice it was shown, with no fine-tuning and no separate speaker-embedding model.
Reference audio (3 s minimum; 10 s in the benchmarks)
│
▼
┌───────────────────────────────────┐
│ Voxtral Codec encoder │ ← audio → codec tokens
└───────────┬───────────────────────┘
│ voice tokens, then text tokens
▼
┌───────────────────────────────────┐
│ Decoder backbone (Ministral 3B) │ ← one semantic token per frame
└───────────┬───────────────────────┘
▼ hidden state per frame
┌───────────────────────────────────┐
│ Flow-matching transformer │ ← 36 acoustic values per frame
│ (3 layers) │
└───────────┬───────────────────────┘
▼
┌───────────────────────────────────┐
│ Voxtral Codec decoder (~300M │ ← tokens → 24 kHz waveform
│ params, whole codec) │
└───────────────────────────────────┘
A few seconds of audio are enough because the cues that identify a voice (pitch range, formant structure, timbre, speaking rate) show up in every syllable. Longer references give the model more prosody and emotional range to copy.
The benchmark numbers: where Voxtral wins and where it doesn’t
The Voxtral paper reports blind pairwise comparisons by native speakers. For zero-shot cloning, it used two recognized speakers per language. The headline result is a 68.4% win rate over ElevenLabs Flash v2.5, but the per-language numbers matter more:
| Language | Voxtral win rate vs Flash v2.5 |
|---|---|
| Spanish | 87.8% |
| Hindi | 79.8% |
| Portuguese | 74.4% |
| Arabic | 72.9% |
| German | 72.0% |
| English | 60.8% |
| Italian | 57.1% |
| French | 54.4% |
| Dutch | 49.4% |
Spanish, Hindi, Portuguese, Arabic and German show clear advantages. English, Italian and French are modest wins. Dutch is effectively a tie.
Against ElevenLabs v3 (the premium tier), on flagship voices rated by three annotators per language, Voxtral won 51.0% of comparisons with explicit emotion steering and 55.4% with implicit steering. That is roughly even.
On automatic metrics (SEED-TTS), Voxtral has 1.23% WER and 0.628 speaker similarity. ElevenLabs v3 has 1.26% and 0.392, and Flash v2.5 has 0.86% and 0.413. Voxtral’s lead in speaker similarity (about 60% higher than v3) is the strongest automatic signal in the paper. Flash v2.5 has the lowest WER of the three.
The cost and latency picture
This is where the practitioner decision often lands:
| Model | Pricing | Latency (vendor-stated) | Languages | License |
|---|---|---|---|---|
| Voxtral API | $0.016 per 1k chars | 70 ms (H200, concurrency 1) | 9 | Commercial via API |
| Voxtral weights | Free | same model | 9 | CC BY-NC 4.0 |
| ElevenLabs Flash v2.5 | $0.04 per 1k chars | ~75 ms model latency | 32 | Proprietary |
| ElevenLabs v3 | $0.08 per 1k chars | not a low-latency model | 70+ | Proprietary |
| Cartesia Sonic Turbo | Plan-based | 40 ms | see vendor | Proprietary |
| Qwen3-TTS 1.7B | Free (self-hosted) | 101 ms first packet | 10 | Apache 2.0 |
ElevenLabs rates are the per-1,000-character figures on its API pricing page as checked in November 2026; they change, so check before you budget. At those rates, Voxtral’s API costs 60% less than Flash v2.5 and 80% less than v3. It wins on quality against Flash in 8 of its 9 languages.
On latency, the numbers are not like-for-like. Mistral measured 70 ms on a single NVIDIA H200 with a 500-character input, a 10-second reference and one request at a time; at 16 concurrent requests it was 331 ms, and at 32 it was 552 ms. ElevenLabs and Cartesia quote model latency for their hosted services. Measure end to end from your own region before you choose on latency.
The commercial open-source gap
The most significant limitation is the license. Voxtral weights are CC BY-NC 4.0, which means research, internal tools and personal use are fine, but any commercial application needs the Mistral API or a separate agreement with Mistral.
For teams that need open weights with commercial rights, Qwen3-TTS (Apache 2.0, 0.6B and 1.7B) is the strongest alternative. Its paper reports the best WER in 6 of 10 languages in a comparison against MiniMax and ElevenLabs, and 3-second voice cloning. Neither paper compares Voxtral and Qwen3-TTS head to head, so test both on your own voices and languages.
Fish Speech 1.5 is sometimes suggested here, but its weights are CC BY-NC-SA 4.0, so it has the same commercial restriction as Voxtral.
For the full cost model of self-hosting, see self-hosted TTS economics.
When to use which model
flowchart TD
A[Voice cloning for production] --> B{Commercial use?}
B -->|No / research only| C[Voxtral weights - free, CC BY-NC]
B -->|Yes| D{Budget sensitive?}
D -->|Cost matters| E{Self-hosting OK?}
D -->|Cost flexible| F{Languages needed?}
E -->|Yes| G[Qwen3-TTS - Apache 2.0, 10 languages]
E -->|No| H[Voxtral API - $0.016 per 1k chars]
F -->|Within Voxtral's 9| H
F -->|70+ languages| I[ElevenLabs v3]
F -->|Lowest latency critical| J[Cartesia Sonic Turbo]
H --> K{Dutch, or a language where Voxtral's win is small?}
K -->|Yes| I
K -->|No| L[Voxtral API - best quality per dollar in its languages]
The decision is mostly about license and language coverage. For English, Spanish, German, Portuguese, Hindi or Arabic voice applications, the Voxtral API offered the best quality per dollar at launch. For commercial products that need open weights: Qwen3-TTS. For 70+ language deployments: ElevenLabs v3. For the lowest vendor-quoted latency: Cartesia Sonic Turbo.
What 3-second cloning means in practice
The data requirement shift changes what is feasible to build.
Older workflows meant recording sessions with voice talent, multi-minute audio files and a fine-tuning pipeline. That friction shaped product design: you collected reference audio once, guarded it carefully, and reused it across your application.
With good zero-shot cloning from a few seconds, a reading app can clone the user’s own voice from a short sample recorded at onboarding, and a dubbing pipeline can work from existing source footage without separate recording sessions.
The same capability is a security and consent problem. A few seconds of anyone’s speech, such as a voicemail greeting or the first seconds of a phone call, is now enough to imitate them convincingly. Cloning a voice should require that person’s explicit consent, and dubbing needs the rights to the voices in the footage. It also means voice alone should not authenticate anyone: treat voice biometrics as one weak signal and require possession-based confirmation for anything that moves money (see security for voicebots).
The technical question “how much audio do I need?” now has a short answer. The product questions are what you are allowed to clone, and what you are willing to ask users to record.
For the full agent-audio pipeline this integrates into (turn-taking, interruption handling, VAD configuration), see voice agent frameworks (LiveKit & Pipecat). For streaming delivery, see streaming speech processing pipeline.
FAQ
How does Voxtral voice cloning work with only 3 seconds of audio? Voxtral encodes the reference clip with its own codec and places those audio tokens in the decoder’s context, followed by the text to speak. The model continues in the voice it was shown. There is no fine-tuning and no separate speaker-embedding model. The paper’s human evaluations and Mistral’s latency benchmark used 10-second references; 3 seconds is the minimum Mistral cites.
How does Voxtral compare to ElevenLabs for production use? In Mistral’s blind tests, Voxtral won 68.4% of zero-shot cloning comparisons against ElevenLabs Flash v2.5, and 51.0% to 55.4% of flagship-voice comparisons against ElevenLabs v3, which is roughly even. On automatic metrics, Flash v2.5 has the lower WER (0.86% vs 1.23%) and Voxtral the higher speaker similarity (0.628 vs 0.413). The Voxtral API costs $0.016 per 1,000 characters, 60% below the $0.04 ElevenLabs lists for Flash on its API pricing page. The open weights are CC BY-NC.
Can I run Voxtral locally for commercial applications? No. Voxtral’s weights are released under CC BY-NC 4.0, which prohibits commercial use. Commercial use goes through the Mistral API ($0.016 per 1,000 characters). For self-hosted commercial use, Qwen3-TTS (Apache 2.0, 10 languages, voice cloning) is the main alternative. Fish Speech 1.5 is not one: its weights are CC BY-NC-SA.
What languages does Voxtral support? Nine: English, French, German, Spanish, Dutch, Portuguese, Italian, Hindi and Arabic. Win rates against ElevenLabs Flash v2.5 vary by language, from 87.8% in Spanish and 79.8% in Hindi to 49.4% in Dutch. For languages outside these nine, ElevenLabs v3 (70+ languages) is the broad commercial option.
What is Voxtral’s architecture? A decoder backbone based on Ministral 3B generates one semantic token per 80 ms frame. A small bidirectional flow-matching transformer (3 layers) then produces that frame’s acoustic values from the backbone’s hidden state. The Voxtral Codec (about 300M parameters) turns the tokens back into 24 kHz audio. The codec uses a hybrid scheme: a VQ semantic token distilled from Whisper plus 36 FSQ acoustic dimensions per frame, at 12.5 frames per second and about 2.14 kbps. Released March 26, 2026, with weights under CC BY-NC 4.0.