Boris Johnson’s Rise and Fall - an analysis of the mics
Political news is not usually what voice AI engineers cue up for a Friday afternoon, but Boris Johnson's resignation announcement, delivered in front of a bank…
![[Olewave's Long Review] Xception: Deep Learning with Depthwise Separable Convolutions](/media/video/2022/07/olewaves-long-review-xception-deep-learning-with-depthwise-separable-convolutions/preview.jpg)
[Olewave's Long Review] Xception: Deep Learning with Depthwise Separable Convolutions
Xception took the Inception hypothesis to its logical extreme by pushing cross-channel and spatial correlations into fully separate operations, and the depthwi…
Phased Array Radar on China's Aircraft Carrier Fujian 003 and Its Connection with Speech Beamforming
The phased array radar mounted on China's newest aircraft carrier, Fujian 003, and the microphone arrays that let your smart speaker hear you across a noisy ro…
How Does the All-New Dictation in iOS 16 Work? Reveal Apple's Secret Sauce by a Speech Researcher!
Apple's iOS 16 dictation runs fully on-device, and that single design choice has cascading consequences for latency, privacy, and what kinds of ASR architectur…
![[Long Review] Conformer: Convolution-augmented Transformer for Speech Recognition](/media/video/2022/06/long-review-conformer-convolution-augmented-transformer-for-speech-recognition/preview.jpg)
[Long Review] Conformer: Convolution-augmented Transformer for Speech Recognition
Conformer is the model that quietly took over end-to-end ASR, and its recipe, bolt a convolution module onto every Transformer block, turned out to be one of t…
博士大叔使用计算机作弊降维打击2022高考数学压轴大题 Ph.D. uses computer cheating to solve 2022 college entrance math exam
What happens when you turn a research-grade computer algebra toolkit loose on the hardest problem of the 2022 Chinese gaokao math exam?
![[Long Review] Axial Attention in Multidimensional Transformers](/media/video/2022/05/long-review-axial-attention-in-multidimensional-transformers/preview.jpg)
[Long Review] Axial Attention in Multidimensional Transformers
Full self-attention over an image or a video is prohibitively expensive, so how do you keep the expressiveness of a Transformer without the quadratic blowup?
![[Short Review] Axial Attention in Multidimensional Transformers](/media/video/2022/05/short-review-axial-attention-in-multidimensional-transformers/preview.jpg)
[Short Review] Axial Attention in Multidimensional Transformers
Axial attention is one of those elegant ideas that keeps Transformers tractable on high-dimensional data: instead of attending over every pixel or every time-f…
![[Long Review] Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](/media/video/2022/05/long-review-transfer-learning-from-speaker-verification-to-multispeaker-text-to-speech/preview.jpg)
[Long Review] Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
This is the paper that made zero-shot voice cloning practical: train a speaker encoder on a discriminative verification task with thousands of noisy, transcrip…
![[Short Review] Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](/media/video/2022/05/short-review-transfer-learning-from-speaker-verification-to-multispeaker-text-to-speech/preview.jpg)
[Short Review] Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
The Google paper that kicked off the modern zero-shot voice cloning wave has a beautifully clean architecture: a speaker encoder trained on verification, a Tac…
![[Short Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages](/media/video/2022/05/short-review-wav2seq-pre-training-speech-to-text-encoder-decoder-models-using-pseudo/preview.jpg)
[Short Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages
Self-supervised pretraining like wav2vec 2.0 gave us great speech encoders, but if you want a full encoder-decoder ASR system, the decoder was still starting f…
![[Long Review] Towards Zero-Label Language Learning](/media/video/2022/05/long-review-towards-zero-label-language-learning/preview.jpg)
[Long Review] Towards Zero-Label Language Learning
What if you could train task-specific NLP models without a single human-labeled example? "Towards Zero-Label Language Learning" leans hard into that question.
![[Long Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs](/media/video/2022/04/long-review-fully-sharded-data-parallel-faster-ai-training-with-fewer-gpus/preview.jpg)
[Long Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs
Training a billion-parameter speech or language model on a modest GPU budget used to mean picking your poison across pipeline parallelism, tensor parallelism, …
![[Short Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs](/media/video/2022/04/short-review-fully-sharded-data-parallel-faster-ai-training-with-fewer-gpus/preview.jpg)
[Short Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs
Fully Sharded Data Parallel is Meta's answer to a question every speech and language team eventually asks: how do we train models an order of magnitude bigger …
![[Short Review] Deduplicating Training Data Makes Language Models Better](/media/video/2022/04/short-review-deduplicating-training-data-makes-language-models-better/preview.jpg)
[Short Review] Deduplicating Training Data Makes Language Models Better
How much of your language model's "generalization" is actually just regurgitation?