Methods
Ten ways a machine learned to speak.
Each plate is a technical approach, not a vendor. Play the examples. The architecture is audible once you know what to listen for.
- 01Open
Concatenative & formant
1960s–2000s · Before neural nets, machines either impersonated a vocal tract with filters, or they cut recorded speech into pieces and glued the pieces back together.
- 02Open
HMM / statistical parametric
1990s–2016 · Instead of gluing recordings, predict a compact description of speech and run it through a vocoder. The muffling is the method.
- 03Open
Autoregressive seq2seq
2017– · Tacotron reads characters the way a translator reads a sentence: one spectrogram frame at a time, each frame looking at the last.
- 04Open
Parallel / non-autoregressive
2019– · Predict how long each phone lasts, then generate every mel frame at once. Stability and speed, with a new kind of flatness to police.
- 05Open
End-to-end VAE (VITS)
2021– · One training stage. Text in, waveform out. A variational autoencoder, a GAN, and a stochastic duration model sharing a loss.
- 06Open
Neural codec language models
2023– · Quantize speech into tokens with a neural codec. Then train a language model on those tokens, as if speech were text.
- 07Open
Flow matching & diffusion
2022– · Treat a spectrogram, or a latent, as a sample from a distribution. Denoise until it is speech. Duration models optional.
- 08Open
Zero-shot cloning
2023– · A few seconds of a person you have never trained on, then their voice on new text. The capability leaked out of VALL-E and became a product checkbox.
- 09Open
LLM / spoken dialogue
2024– · The model is not reading a script. It is talking, with tools, interruptions, and a voice that is part of the language model.
- 10Open
Streaming / low-latency
2023– · Time-to-first-audio is the number voice agents actually feel. Architectures that decode left-to-right, or that scale linearly, win here.