Framed with a wink at Nathan Chen's quad flip scoring, this talk unpacks Switch Transformer, Google's clean and aggressively simplified take on Mixture-of-Experts scaling. Where earlier MoE designs routed each token to two experts, Switch routes to just one, dropping the routing math to its bare minimum and unlocking trillion-parameter training runs that stay stable on real TPU pods.

Part 1 of a two-parter, this review walks through the top-1 gating decision, the load-balancing and capacity-factor tricks that keep experts from starving or overflowing, and the pretraining speedups Switch delivers over dense T5 baselines. For voice AI teams considering sparse architectures for speech LLMs, multilingual ASR, or multi-domain TTS backbones, Switch Transformer is the paper that made trillion-parameter models feel achievable rather than aspirational. Pull it up if you want the intuition before you dive into GLaM in Part 2.