Google fired Blake Lemoine for saying AI bot is sentient? Does LaMDA or ChatGPT think like human?
When Google fired Blake Lemoine for going public with claims that LaMDA had become sentient, the story went viral, but the underlying question stuck around: do…
![[Long Review] CLAS: Deep context: end-to-end contextual speech recognition](/media/video/2022/04/long-review-clas-deep-context-end-to-end-contextual-speech-recognition/preview.jpg)
[Long Review] CLAS: Deep context: end-to-end contextual speech recognition
Anyone who has shipped an ASR product knows the pain of proper nouns: contact names, song titles, medication names, obscure place names.
![[Long Review] Hurdles to Progress in Long Form Question Answering](/media/video/2022/04/long-review-hurdles-to-progress-in-long-form-question-answering/preview.jpg)
[Long Review] Hurdles to Progress in Long Form Question Answering
Long-form question answering is one of those benchmarks that looks impressive on the leaderboard and disturbing under a microscope.
![[Long Review] Finetuned Language Models Are Zero-Shot Learners](/media/video/2022/03/long-review-finetuned-language-models-are-zero-shot-learners/preview.jpg)
[Long Review] Finetuned Language Models Are Zero-Shot Learners
Before InstructGPT made instruction tuning a household phrase, Google's FLAN paper made the case that a modest amount of multi-task fine-tuning on natural-lang…
![[Long Review] 'GShard': Scaling Giant Models with Conditional Computation and Automatic Sharding](/media/video/2022/02/long-review-gshard-scaling-giant-models-with-conditional-computation-and-automatic/preview.jpg)
[Long Review] 'GShard': Scaling Giant Models with Conditional Computation and Automatic Sharding
Scaling a Transformer past a hundred billion parameters gets philosophical fast: do you activate the whole network for every token, or route each token to the …
Triplets-like Russian Figure Skaters: Can Kullback-Leibler Divergence be Used Tell Their Difference?
Speaker adaptation is one of those ASR problems that sounds solved until you actually try to adapt a sequence-to-sequence model without wrecking the encoder's …
Is Nathan Chen's 4 Flip scored by Mixture-of-Experts? Part 1: Switch Transformers: sparse MoE models
Framed with a wink at Nathan Chen's quad flip scoring, this talk unpacks Switch Transformer, Google's clean and aggressively simplified take on Mixture-of-Expe…
Is Nathan Chen's 4 Flip scored by Mixture-of-Experts? Part 2: GLaM:Efficient Scaling of LMs with MoE
Part 2 of the MoE mini-series turns to GLaM, Google's 1.2-trillion-parameter Mixture-of-Experts language model that matches or beats GPT-3 on zero-, one-, and …
BERT Paper Reviewed from a Speech Perspective
BERT hardly needs an introduction in NLP, but reading it through a speech engineer's eyes uncovers a different set of lessons than the standard "masked languag…
Transformer: Attention is All You Need and Listen, Attend and Spell -- from a Speech Perspective
Two papers, one story. "Attention Is All You Need" gave the world the Transformer and reset how sequence modeling gets done, while "Listen, Attend and Spell" (…
From Breaking Bad to Wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
What do Walter White, Jesse Pinkman, and a batch of unlabeled audio have in common?