Self-supervised speech models tend to learn phonetic content beautifully and speaker identity as an afterthought — which is fine for ASR and terrible for speaker verification, diarization, or voice cloning. UniSpeech-SAT, from Microsoft, patches this gap by injecting explicit speaker-aware objectives into HuBERT-style pretraining: an utterance-wise contrastive loss and utterance mixing that force the encoder to represent who is speaking, not just what is being said.

The payoff shows up across the SUPERB benchmark, where UniSpeech-SAT posts especially strong numbers on speaker identification, verification, and diarization while holding its own on ASR and phoneme recognition. This session unpacks how the utterance mixing is constructed, why the extra contrastive term doesn't hurt content learning, and how the design paved the way for WavLM shortly after. If you're building anything that needs to distinguish voices — from meeting transcription to biometric auth — this is worth 10 minutes of your time.