Before Tacotron, before VALL-E, before neural codecs turned audio into tokens, DeepMind's WaveNet was the paper that showed neural networks could generate raw audio waveforms sample by sample and beat every parametric and concatenative TTS system on naturalness. Look at it now, with GPT-shaped glasses on, and the resemblance is uncanny: a fully probabilistic, autoregressive model conditioning each sample on all previous ones — a language model for audio, five years before LLMs were a household term.

The review walks through the architecture that made it work at 16 kHz: dilated causal convolutions to grow the receptive field exponentially, gated activations, residual and skip connections. It also covers what WaveNet demonstrated beyond TTS — speaker conditioning to swap voices, music generation as a byproduct, and even competitive phoneme recognition as a discriminative model. If you want to see where modern speech generative AI actually started, and why the autoregressive-tokens-over-audio paradigm keeps coming back in newer TTS systems, this retrospective is a great grounding.