Speech Technology

A voice agent has about a second to reply before the caller starts to wonder if anyone is there. In that second it has to notice the caller has stopped talking, turn the audio into text, decide what to do and start speaking, often over a bad phone line and in an accent no benchmark included.

I have worked on this for more than ten years: text-to-speech for 13 Indian languages at IIT Madras, speech recognition and synthesis that shipped on more than 200 million Galaxy devices at Samsung, and voice agents that answer customer calls. Here I write about what holds up on real calls: picking ASR and TTS models, why benchmark accuracy falls apart in production, what self-hosting really costs, and how voice agents get attacked.

Start here

Voice agents and conversational AI

Speech recognition (ASR)

Text-to-speech (TTS)

Audio front end and signal processing

Speakers, keywords and classification

Production systems, cost and privacy

Training, evaluation and MLOps

Planning a voice agent, or choosing between ASR, TTS and speech-to-speech stacks? Arun runs voice agent architecture reviews covering model choice, latency and cost. If your agent takes actions on calls, the agent security review covers how it can be abused. To build it with you, he takes on contract engineering, and for speech research problems, contract research and collaborations.

Recently added


Posts on new research summarize other people’s papers and link to the originals. Some posts are drafted with AI assistance and edited by Arun. If you spot an error, and it gets fixed.