Ask Whisper for word timestamps and you'll get numbers back — but if you've ever tried to line them up with the actual audio for karaoke, captioning, or dubbing, you know they drift, snap to wrong boundaries, and occasionally hallucinate word breaks that don't exist. This talk explains why that happens at an architectural level and what you can realistically do about it.

The short version: end-to-end Transformer decoders trained with cross-entropy are optimized to predict the next token, not to align it in time. Cross-attention weights and DTW-based tricks are heuristics on top of a model that never learned alignment as a first-class objective. The walkthrough then surveys the fixes on offer — heuristic postprocessing tools (mostly unbenchmarked and best avoided), phone-based forced alignment (better but requires a pronunciation dictionary and extra compute), and hybrid approaches that pair Whisper's transcript with a CTC or forced-alignment model. If you're shipping any product where timing matters — subtitles, dubbing, voice editing, speech analytics — this is essential context before you trust Whisper's timestamps in production.