Olewave - Professional and Trustworthy - Data Services and Solutions

Tutorials

A Review of Deepmind's WaveNet for TTS/Audio Synthesis (Does it look like GPT to you?)

Before Tacotron, before VALL-E, before neural codecs turned audio into tokens, DeepMind's WaveNet was the paper that showed neural networks could generate raw …

May 6, 2023· 1 min read

Tutorials

Review of HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

For a long time, neural vocoders forced you to pick a side: autoregressive models like WaveNet gave you gorgeous audio but were painfully slow, while GAN-based…

Apr 22, 2023· 1 min read

Tutorials

In-depth Review of Google's SoundStream: An End-to-End Neural Audio Codec

Neural codecs quietly became the foundation layer of modern generative audio, and SoundStream is one of the papers that got the ball rolling.

Apr 8, 2023· 1 min read

Tutorials

Disclosing OpenAI GPT-4's vision+text model, data, and cost to train (speculated)

OpenAI famously refused to disclose GPT-4's architecture, parameter count, training data, or compute budget, which turned the tech report into a Rorschach test…

Apr 1, 2023· 1 min read

Tutorials

A Review of GPT-4's Technical Report (GPT-4 in a Nutshell)

GPT-4's technical report is famously thin on architecture details and thick on capability claims, which makes a clear-headed review essential reading for anyon…

Mar 25, 2023· 1 min read

Tutorials

[Olewave's Review] AudioLM: a Language Modeling Approach to Audio Generation

AudioLM is the paper that convinced a lot of people audio generation should look more like language modeling than signal processing.

Mar 11, 2023· 1 min read

Tutorials

In-depth review of OpenAI's GPT-3 : Language Models are Few-Shot Learners (Part 3/3: Results&Rest)

The final installment of this three-part GPT-3 deep dive gets to the payoff: what actually happens when you throw 175 billion parameters at a benchmark suite a…

Mar 8, 2023· 1 min read

Tutorials

[10 mins] Explain Why OpenAI's Whisper API Isn't As Good As ChatGPT

Whisper landed with a bang, but it did not reshape ASR the way GPT-3 reshaped NLP, and this ten-minute review argues the reasons are baked into the paper itsel…

Mar 4, 2023· 1 min read

Tutorials

In-depth review of OpenAI's GPT-3 : Language Models are Few-Shot Learners (Part 2/3: Results)

Part two of this GPT-3 deep dive moves past the setup and into the results that actually made people rethink NLP.

Feb 25, 2023· 1 min read

Tutorials

WSJ Made a Mistake in Translating China's Foreign Minister's Speech; What has the Diplomat Said?

Machine and human translation errors are more than a language-nerd curiosity when they land on the front page of the Wall Street Journal and reshape geopolitic…

Feb 22, 2023· 1 min read

Tutorials

In-depth review of OpenAI's GPT-3 : Language Models are Few-Shot Learners (Part 1/3: Intro&Approach)

The opening installment of this three-part GPT-3 review sets up the paper that turned scaling from a research bet into industry gospel.

Feb 11, 2023· 1 min read

Tutorials

In-depth Review of VALL-E: Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers

Zero-shot TTS from a three-second reference clip sounded like a stretch until VALL-E shipped, and this in-depth review unpacks exactly how Microsoft pulled it …

Feb 4, 2023· 1 min read

Tutorials

Understand Microsoft's VALL-E in 3 Minutes (SOTA Zero-shot TTS)

VALL-E is the moment TTS stopped looking like signal regression and started looking like language modeling, and this quick explainer gets you up to speed fast.

Feb 4, 2023· 1 min read

Tutorials

ChatGPT/ChatGPT Plus/InstructGPT:Training language models to follow instructions with human feedback

Behind the ChatGPT product sits the InstructGPT paper, and this detailed review pulls apart the pipeline that turned a raw GPT-3 into something people actually…

Jan 28, 2023· 1 min read

Tutorials

Explain how ChatGPT/ChatGPT Plus works in 3 minutes

ChatGPT looks like magic from the outside, but the recipe is well-documented if you know where to look, and this three-minute explainer condenses the core idea…

Jan 28, 2023· 1 min read

Tutorials

[Olewave's Review] CLIP (3/3): Learning Transferable Visual Models From Natural Language Supervision

The final part of this CLIP review lands on the results section, which is where OpenAI's contrastive image-text model shifted from an interesting idea to a fou…

Jan 14, 2023· 1 min read

Tutorials

[Olewave's Review] CLIP (2/3): Learning Transferable Visual Models From Natural Language Supervision

OpenAI's CLIP flipped the script on computer vision by tossing out fixed label sets and instead training on 400 million (image, text) pairs scraped from the in…

Jan 1, 2023· 1 min read

Tutorials

[Olewave's Review] CLIP (1/3): Learning Transferable Visual Models From Natural Language Supervision

Before there was BLIP, LLaVA, or any speech-LLM worth its salt, there was CLIP, and this is where the story begins.

Dec 17, 2022· 1 min read

Tutorials

[Olewave's Review] Token-level Sequence Labeling for SLU using Compositional E2E Models

End-to-end SLU is having a moment, but treating sequence labeling as sequence prediction quietly throws away decades of well-understood token-level tagging mac…

Dec 3, 2022· 1 min read

Tutorials

[Olewave's Review] Branchformer: Parallel MLP-Attention Architectures, and E-Branchformer

Conformer has been the ASR encoder to beat for years, but Branchformer and its follow-up E-Branchformer make a compelling case that sequential conv-plus-attent…

Nov 19, 2022· 1 min read

Tutorials

Non Collision Mispronunciation Addition (NCMA) for Accented ASR

Accented and non-native speech breaks pronunciation dictionaries in ways that quietly poison your acoustic model alignments, and this Interspeech 2021 work ask…

Nov 3, 2022· 1 min read

Tutorials

[Olewave's Review] OpenAI's Whisper ASR: Robust Speech Recognition via Large-Scale Weak Supervision

When OpenAI dropped Whisper, the ASR community had to reckon with a system trained on 680,000 hours of multilingual, multitask, web-scraped audio that just wor…

Oct 31, 2022· 1 min read

Tutorials

One-Edit-Distance FSA/Network-based (OEDN) in Mispronunciation Detection and Accented ASR

Non-native acoustic modeling has a chicken-and-egg problem: bad phone alignments produce bad models, and bad models produce bad alignments.

Sep 24, 2022· 1 min read

Tutorials

Olewave's most detailed illustration of RNN-T: Sequence Transduction with Recurrent Neural Networks

Alex Graves' RNN-T paper is the quiet ancestor of nearly every streaming ASR system shipping today, from Google's on-device recognizer to countless open-source…

Sep 3, 2022· 1 min read

Tutorials

Google fired Blake Lemoine for saying AI bot is sentient? Does LaMDA or ChatGPT think like human?

When Google fired Blake Lemoine for going public with claims that LaMDA had become sentient, the story went viral, but the underlying question stuck around: do…

Aug 5, 2022· 1 min read

Tutorials

[Olewave's Long Review] Efficient Training of Neural Transducer for Speech Recognition

Neural transducers are the workhorse of streaming ASR, but training them efficiently is famously painful: the RNN-T loss has cubic memory in sequence length, t…

Jul 23, 2022· 1 min read

Tutorials

Boris Johnson’s Rise and Fall - an analysis of the mics

Political news is not usually what voice AI engineers cue up for a Friday afternoon, but Boris Johnson's resignation announcement, delivered in front of a bank…

Jul 9, 2022· 1 min read

Tutorials

[Olewave's Long Review] Xception: Deep Learning with Depthwise Separable Convolutions

Xception took the Inception hypothesis to its logical extreme by pushing cross-channel and spatial correlations into fully separate operations, and the depthwi…

Jul 9, 2022· 1 min read