Tutorials
Is Nathan Chen's 4 Flip scored by Mixture-of-Experts? Part 2: GLaM:Efficient Scaling of LMs with MoE
Part 2 of the MoE mini-series turns to GLaM, Google's 1.2-trillion-parameter Mixture-of-Experts language model that matches or beats GPT-3 on zero-, one-, and …