How far can you push cross-lingual speech pretraining before the well runs dry? Meta's XLS-R answers with 2 billion parameters trained on 436,000 hours of unlabeled audio across 128 languages — an order-of-magnitude jump over XLSR-53 and, at release, the largest public multilingual speech model by a wide margin.

The results back up the scale story. XLS-R sets new state of the art on BABEL, CommonVoice, and VoxPopuli speech recognition, and — perhaps more surprising — on speech translation into English, where it beats prior systems built on much larger text-side models. The talk walks through the data curation across those 128 languages, how the wav2vec 2.0 architecture holds up at 2B parameters, and what breaks when you try to fine-tune something this large on low-resource data. If you're working on any speech task in a language HuggingFace doesn't already cover, XLS-R is probably the first checkpoint you should try — the deep dive explains why.