Olewave - Professional and Trustworthy - Data Services and Solutions

Tutorials

[Short Review] Conformer: Convolution-augmented Transformer for Speech Recognition

Conformer took the ASR world by storm by doing something almost boring: gluing a convolution module into each Transformer block so the model captures both glob…

Jun 11, 2022· 1 min read

Tutorials

[Long Review] Axial Attention in Multidimensional Transformers

Full self-attention over an image or a video is prohibitively expensive, so how do you keep the expressiveness of a Transformer without the quadratic blowup?

May 28, 2022· 1 min read

Tutorials

[Short Review] Axial Attention in Multidimensional Transformers

Axial attention is one of those elegant ideas that keeps Transformers tractable on high-dimensional data: instead of attending over every pixel or every time-f…

May 28, 2022· 1 min read

Tutorials

[Long Review] Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

This is the paper that made zero-shot voice cloning practical: train a speaker encoder on a discriminative verification task with thousands of noisy, transcrip…

May 21, 2022· 1 min read

Tutorials

[Short Review] Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

The Google paper that kicked off the modern zero-shot voice cloning wave has a beautifully clean architecture: a speaker encoder trained on verification, a Tac…

May 21, 2022· 1 min read

Tutorials

[Long Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages

Most speech pretraining has focused on the encoder side, wav2vec, HuBERT, WavLM, but sequence-to-sequence ASR needs a pretrained decoder too.

May 14, 2022· 1 min read

Tutorials

[Short Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages

Self-supervised pretraining like wav2vec 2.0 gave us great speech encoders, but if you want a full encoder-decoder ASR system, the decoder was still starting f…

May 14, 2022· 1 min read

Tutorials

[Long Review] Towards Zero-Label Language Learning

What if you could train task-specific NLP models without a single human-labeled example? "Towards Zero-Label Language Learning" leans hard into that question.

May 7, 2022· 1 min read

Tutorials

[Long Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs

Training a billion-parameter speech or language model on a modest GPU budget used to mean picking your poison across pipeline parallelism, tensor parallelism, …

Apr 23, 2022· 1 min read

Tutorials

[Short Review] Fully Sharded Data Parallel: faster AI training with fewer GPUs

Fully Sharded Data Parallel is Meta's answer to a question every speech and language team eventually asks: how do we train models an order of magnitude bigger …

Apr 23, 2022· 1 min read

Tutorials

[Short Review] Deduplicating Training Data Makes Language Models Better

How much of your language model's "generalization" is actually just regurgitation?

Apr 16, 2022· 1 min read

Tutorials

From Breaking Bad to Wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

What do Walter White, Jesse Pinkman, and a batch of unlabeled audio have in common?

Jan 22, 2022· 1 min read

Tutorials

Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognition

Speech emotion recognition has long been stuck with tiny labeled datasets and hand-crafted acoustic features.

Dec 4, 2021· 1 min read

Tutorials

Improving Speech Recognition Accuracy of Local POI Using Geographical Models

Ask a voice assistant to navigate to a small local restaurant and you'll quickly find where ASR falls apart — long-tail POI names that never showed up in the L…

Nov 19, 2021· 1 min read

Tutorials

BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recog

What happens when you take the semi-supervised ASR recipe that beat the state of the art at 100M parameters and just… keep scaling?

Oct 29, 2021· 1 min read

Tag: asr