07 · 2022–

Flow matching & diffusion

Treat a spectrogram, or a latent, as a sample from a distribution. Denoise until it is speech. Duration models optional.

Playable examples

F5-TTS zero-shot English from the Seed-TTS prompt. Then StyleTTS 2 on LJSpeech next to VITS — diffusion of style, not of the waveform.

NaturalSpeech put a variational backbone under Microsoft’s human-parity claim. NaturalSpeech 2 moved to latent diffusion and zero-shot. StyleTTS 2 diffused style vectors instead of the waveform, then decoded — human-level MOS on LJSpeech, and the architecture Kokoro later stripped for speed.

E2 TTS showed you could skip duration models entirely: pad text to the length of speech and let a diffusion transformer align them. F5-TTS made that idea converge, with ConvNeXt text refinement and Sway Sampling. Matcha-TTS is the compact conditional-flow-matching acoustic model you can actually explain on a whiteboard.

Tortoise (2022) is the slow ancestor a lot of people ran anyway: autoregressive codes plus a diffusion decoder, cloning from a handful of seconds if you were willing to wait. Flow matching is, among other things, an attempt to keep that quality without the wait.