Olewave - Professional and Trustworthy - Data Services and Solutions

Tutorials

I-JEPA: Yannn LeCun's First 'World Model' for Computer Vision

Yann LeCun has been arguing for years that generative pretraining is the wrong bet for building machines that understand the world.

Jun 17, 2023· 1 min read

Tutorials

[Long Review] Axial Attention in Multidimensional Transformers

Full self-attention over an image or a video is prohibitively expensive, so how do you keep the expressiveness of a Transformer without the quadratic blowup?

May 28, 2022· 1 min read

Tutorials

[Short Review] Axial Attention in Multidimensional Transformers

Axial attention is one of those elegant ideas that keeps Transformers tractable on high-dimensional data: instead of attending over every pixel or every time-f…

May 28, 2022· 1 min read

Tutorials

[Long Review] Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

This is the paper that made zero-shot voice cloning practical: train a speaker encoder on a discriminative verification task with thousands of noisy, transcrip…

May 21, 2022· 1 min read

Tutorials

[Short Review] Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis

The Google paper that kicked off the modern zero-shot voice cloning wave has a beautifully clean architecture: a speaker encoder trained on verification, a Tac…

May 21, 2022· 1 min read

Tutorials

[Short Review] Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages

Self-supervised pretraining like wav2vec 2.0 gave us great speech encoders, but if you want a full encoder-decoder ASR system, the decoder was still starting f…

May 14, 2022· 1 min read

Tutorials

[Long Review] Towards Zero-Label Language Learning

What if you could train task-specific NLP models without a single human-labeled example? "Towards Zero-Label Language Learning" leans hard into that question.

May 7, 2022· 1 min read

Tutorials

[Short Review] Deduplicating Training Data Makes Language Models Better

How much of your language model's "generalization" is actually just regurgitation?

Apr 16, 2022· 1 min read

Tag: meta