Microsoft2023
VALL-E
Codec LMresearchcloneemotionvendor demo
Zero-shot cloning as in-context learning.
06 · 2023–
Neural codec language models
Quantize speech into tokens with a neural codec. Then train a language model on those tokens, as if speech were text.
Playable examples
The VALL-E Bill Gates prompt clip is cloning as in-context learning. Bark’s README pizza clip is the other codec-LM habit: speech plus [laughs] from a text prompt.
VALL-E (Microsoft, 2023) made the claim sharp: take EnCodec-style discrete codes, train an autoregressive transformer on 60,000 hours, and cloning becomes in-context learning. Three seconds of a speaker are a prompt, not a fine-tune.
Bark (Suno) is the chaotic cousin: an audio LM that will also emit music, noise, and [laughs]. Fish Speech / Fish Audio S2 and Spark-TTS stay closer to TTS. Orpheus puts a Llama backbone behind the same idea. Qwen3-TTS is the 2026 open default in a lot of local stacks.
The failure modes change. Codec LMs hallucinate words, insert breaths in the wrong place, or lock onto the prompt’s room tone. They also copy emotion and acoustic environment in a way a vocoder stack never did. That is why this family and the spoken-dialogue models start to blur.
On this plate
The VALL-E Bill Gates prompt clip is cloning as in-context learning. Bark’s README pizza clip is the other codec-LM habit: speech plus [laughs] from a text prompt.
Suno2023
Bark
Codec LMopencloneemotionvendor demo
[laughs] as a first-class output, not a failure mode.
SparkAudio2025
Spark-TTS
Codec LMopencloneemotionvendor demo
Open codec-LM TTS without a 7B backbone.
Fish Audio2024
Fish Audio / Fish Speech
Codec LMresearchclonestreamemotionvendor demo
Top open-weight arena rank with a non-permissive license asterisk.
Canopy Labs2025
Orpheus
Codec LMopenclonestreamemotionno clip
LLM-backbone TTS with a permissive license.
Alibaba Qwen2025
Qwen3-TTS
Codec LMopenclonestreamemotionno clip
The open all-rounder a lot of 2026 local agents actually use.