How much of your language model's "generalization" is actually just regurgitation? Google Research put a number on it: over 1% of unprompted LM output is copied verbatim from training data, and standard corpora like C4 contain sentences repeated tens of thousands of times. Their paper introduces two efficient deduplication tools that strip near-duplicates and long repeated substrings from web-scale datasets, with striking downstream effects.

Models trained on the cleaned data memorize ten times less, converge in fewer steps, and produce more trustworthy evaluation numbers because train-test overlap drops sharply too. For voice AI teams building LLM back-ends for conversational agents, or training speech-language models on scraped transcripts, this is a reminder that dataset hygiene often beats architectural cleverness. This short review covers the deduplication algorithms, the memorization measurements, and the released code. Worth five minutes if data quality is anywhere near your next sprint.