Olign Ranks First on TIMIT in the Nyra Speech Timing Benchmark

Product

Olign Ranks First on TIMIT in the Nyra Speech Timing Benchmark

Olign v1 ranks first of 22 systems on TIMIT in nyra labs' own table, and Olewave's run of its current release, v1.1.3, has a word timing error of 14.2 ms on TIMIT and 23.6 ms on Buckeye.

Olign Ranks First on TIMIT in the Nyra Speech Timing Benchmark

Olign ยท Nyra Speech Timing Benchmark

Summary

  • First on TIMIT. Olign v1 ranks first of the 22 systems in nyra labs' own TIMIT table. It has the lowest word timing error there, 15.9 ms, and the most words within 50 ms of the hand labels.
  • Closer still today. Olign v1 is now at release v1.1.3. In Olewave's run of nyra's public protocol, it has a word timing error of 14.2 ms on TIMIT and 23.6 ms on conversational speech from Buckeye.
  • Check it yourself. You can rerun both tests with nyra's tool and an Olign API key. We have asked nyra to re-run Olign v1, and their number will be the authoritative one.

The benchmark

Nyra labs published the Nyra Speech Timing Benchmark in October 2026, with a paper and code on GitHub. It measures how close each system's word times come to hand labels, and every system goes through the same protocol. This post covers its read speech and its conversation.

TIMIT is read speech, 1,626 sentences. Buckeye is conversation, 2,008 chunks of up to 20 seconds each, with 80,507 words.

Each word gets a timing error, the average of how far its start and its end land from the hand labels. Word MAE is that error averaged over all the words, in milliseconds. Lower is better. Acc@50 ms is the share of words whose start and end both land within 50 ms of the labels. Higher is better. Nyra treats this share as its headline number.

How we measured

Every row on the two pictures in this post is nyra's own run, except Olign v1, which carries a star. Olign v1 is Olewave's run of release v1.1.3 on the same public protocol, and it holds no place on nyra's leaderboard.

The v1.1.3 numbers were run by Olewave on the released v1.1.3 configuration. We cut nyra's exact chunks from our licensed copies of TIMIT and Buckeye, sent the reference words as the transcript, and scored the output with nyra's public scorer at commit 8acf80f.

Nyra keeps its reference times private, since neither corpus may be redistributed. So we rebuilt them from the corpus files, the TIMIT .WRD files and the Buckeye .words files. Our reference covers 1,625 of nyra's 1,626 TIMIT sentences and 79,871 of its 80,507 Buckeye words.

Nyra's table lists Olign v1 at 15.9 ms on TIMIT and 35.7 ms on Buckeye. Nyra ran an earlier release in October. Scored against our reference, those same predictions give 15.3 ms and 35.3 ms. So our reference reads about half a millisecond below nyra's, and you can read Olign v1's bars with that margin in mind.

Olign v1 at release v1.1.3 on nyra's benchmark
Olewave's runs, TIMIT on November 15, 2026 and Buckeye on November 11, 2026

TIMIT Buckeye
Word MAE 14.2 ms 23.6 ms
Median word timing error 7.9 ms 8.7 ms
Acc@50 ms 86.2 % 86.8 %
Acc@100 ms 96.9 % 94.7 %

Words that come back with no length are scored where they were placed.

TIMIT, where Olign ranks first

In nyra's TIMIT table, Olign v1 ranks first of 22 systems. Its word MAE is 15.9 ms. Nyra's own aligner follows at 21.3 ms, and MFA with a wide beam at 23.2 ms. Olign v1 also has the most words within 25, 50 and 100 ms of the hand labels, and it ties for first at 200 ms.

On release v1.1.3, Olewave's run brings the word MAE to 14.2 ms.

The picture puts that run, starred, next to every other system in nyra's TIMIT table, as nyra published it on November 10, 2026. Systems tagged ASR are timed only on the words they recognized, so nyra asks you to read them together with word recall. You will find it in the table under the picture.

Word MAE on TIMIT in the Nyra Speech Timing Benchmark, lower is better. Olign v1, starred as Olewave's run, 14.2 ms, then nyra-FA 21.3 ms and MFA with a wide beam 23.2 ms. Nineteen more systems follow, up to 88.3 ms.
All the numbers
SystemNyra's nameTypeWord recallWord MAEAcc@50 ms
Olign v1 โ˜… Olewaveforced aligner14.2 ms86.2 %
nyra-FA nyra labsnyra_faforced aligner99.9 %21.3 ms82.2 %
MFA, wide beam โ€ mfa (wide beam)forced aligner99.9 %23.2 ms77.7 %
MFA, default beam โ€ mfa (default beam)forced aligner99.9 %23.5 ms77.7 %
CrisperWhisper 2.0 large nyra labscrisperwhisper_2.0_largeASR system98.8 %29.6 ms70.1 %
Qwen3-FA Qwenqwen_faforced aligner100.0 %30.1 ms70.8 %
Fish Transcribe 1 Pro Fish Audiofish_transcribe_1_proASR system98.9 %30.4 ms70.2 %
Grok STT xAIxai_grok_sttASR system98.3 %37.3 ms51.3 %
MMS-FA Metamms_faforced aligner100.0 %37.3 ms51.3 %
ElevenLabs CueCue (ElevenLabs)forced aligner100.0 %44.6 ms32.5 %
MWA, TIMIT model MLSpeechmwa_timitforced aligner100.0 %47.2 ms53.5 %
CTC segmentationctc_segmentationforced aligner100.0 %49.3 ms38.9 %
Universal 3.5 Pro AssemblyAIassemblyai_u35_proASR system97.9 %51.3 ms33.6 %
Universal 2, disfluencies AssemblyAIassemblyai_u2_disflASR system98.5 %53.0 ms27.5 %
MAI-Transcribe-2 Microsoftmai_transcribe_2ASR system99.3 %54.9 ms27.6 %
MWA, Buckeye model MLSpeechmwa_buckeyeforced aligner100.0 %58.9 ms49.8 %
Nemotron Transcribe NVIDIAnemotron_transcribeASR system98.9 %60.0 ms23.4 %
NeMo-FA NVIDIAnemo_faforced aligner100.0 %60.0 ms19.3 %
Nova-3 Deepgramdeepgram_nova3ASR system98.2 %63.4 ms20.5 %
WhisperXwhisperxASR system98.2 %64.8 ms13.5 %
Canary NVIDIAcanaryASR system97.6 %85.5 ms13.2 %
Ink-Whisper Cartesiacartesia_ink_whisperASR system98.2 %88.3 ms20.7 %

โ˜… Olign v1 is Olewave's run of release v1.1.3 on November 15, 2026, with nyra's public scorer on nyra's chunks. It holds no place on nyra's leaderboard. โ€  MFA 3.3.9 with the english_mfa 3.1.0 model, as nyra runs it.

Buckeye

Buckeye is the harder test. Nyra cuts each interview into chunks of up to 20 seconds, exactly at the first and the last word. Each chunk keeps its laughter and its long pauses, and often the interviewer's voice. Earlier Olign releases made most of their Buckeye error at the chunk edges and next to pauses.

In nyra's Buckeye table, nyra's own aligner has the lowest word MAE, 20.2 ms. On release v1.1.3, Olewave's run of Olign v1 gives 23.6 ms. Chunks of 15 seconds or less score 22.9 ms, and longer chunks score 23.8 ms. The picture shows nyra's Buckeye table the same way, with Olign v1 starred.

Word MAE on Buckeye in the Nyra Speech Timing Benchmark, lower is better. nyra-FA 20.2 ms, then Olign v1, starred as Olewave's run, 23.6 ms, and Fish Transcribe 1 Pro 35.5 ms. Eighteen more systems follow, up to 144.8 ms.
All the numbers
SystemNyra's nameTypeWord recallWord MAEAcc@50 ms
nyra-FA nyra labsnyra_faforced aligner100.0 %20.2 ms88.7 %
Olign v1 โ˜… Olewaveforced aligner23.6 ms86.8 %
Fish Transcribe 1 Pro Fish Audiofish_transcribe_1_proASR system93.2 %35.5 ms75.8 %
MFA, wide beam โ€ mfa (wide beam)forced aligner99.9 %35.7 ms79.1 %
CrisperWhisper 2.0 large nyra labscrisperwhisper_2.0_largeASR system93.2 %38.8 ms66.1 %
Qwen3-FA Qwenqwen_faforced aligner99.9 %39.4 ms75.0 %
Grok STT xAIxai_grok_sttASR system92.9 %48.8 ms50.5 %
MFA, default beam โ€ mfa (default beam)forced aligner98.7 %48.9 ms78.3 %
ElevenLabs CueCue (ElevenLabs)forced aligner100.0 %50.8 ms32.4 %
MMS-FA Metamms_faforced aligner100.0 %52.7 ms49.9 %
MAI-Transcribe-2 Microsoftmai_transcribe_2ASR system91.7 %54.4 ms31.0 %
Nemotron Transcribe NVIDIAnemotron_transcribeASR system93.8 %60.1 ms33.4 %
Universal 3.5 Pro AssemblyAIassemblyai_u35_proASR system88.7 %68.1 ms30.9 %
NeMo-FA NVIDIAnemo_faforced aligner100.0 %72.6 ms20.4 %
CTC segmentationctc_segmentationforced aligner100.0 %81.0 ms48.8 %
Nova-3 Deepgramdeepgram_nova3ASR system92.7 %87.6 ms14.8 %
WhisperXwhisperxASR system88.1 %89.8 ms8.7 %
Canary NVIDIAcanaryASR system88.2 %107.2 ms10.0 %
MWA, Buckeye model MLSpeechmwa_buckeyeforced aligner100.0 %108.7 ms58.9 %
MWA, TIMIT model MLSpeechmwa_timitforced aligner100.0 %143.2 ms51.3 %
Ink-Whisper Cartesiacartesia_ink_whisperASR system88.6 %144.8 ms35.3 %

โ˜… Olign v1 is Olewave's run of release v1.1.3 on November 11, 2026, with nyra's public scorer on nyra's chunks. It holds no place on nyra's leaderboard. โ€  MFA 3.3.9 with the english_mfa 3.1.0 model, as nyra runs it.

What is in release v1.1.3

Since nyra's run, Olign v1 has moved through releases v1.1.0 to v1.1.3. The API and the response format are unchanged.

  • Words outside Olign's dictionary get their pronunciation from a pronunciation model.
  • Recordings longer than 15 seconds, and recordings of any length sent without a transcript, are handled differently at their start and end.
  • A word next to a pause ends or begins in a new place. The word before a pause no longer runs into the pause, so pauses come back closer to their full length.

The last two changes address the chunk edges and the pauses, the two places where Buckeye cost earlier releases the most. A recording of 15 seconds or less sent with a transcript is aligned in one pass.

Check it yourself

You can rerun both tests through the hosted service with nyra's own tool. You need an Olign API key, which comes with an account at tycho.olewave.com, and your own licensed copies of TIMIT and Buckeye. Point --data-root at them if they are not in nyra's default folder. The two runs send about 8.6 hours of audio, 1.4 hours of TIMIT and 7.2 hours of Buckeye.

Run nyra's benchmark on Olign v1

git clone https://github.com/nyrahealth/nyra_speech_timing_benchmark
cd nyra_speech_timing_benchmark
pip install -e .
export OLIGN_KEY=your-olign-key
nyra-timing run --dataset timit --model olign_v1 \
  --out results_api --save-predictions predictions
nyra-timing run --dataset buckeye --model olign_v1 \
  --out results_api --save-predictions predictions

Each run prints its scores and saves the service's raw responses. Every Olign response carries its release in the X-Olign-Version header, so you can see which release answered.

Check which release answered

python - <<'EOF'
import gzip, json
for name in ("timit", "buckeye"):
    raw = json.load(gzip.open(f"predictions/raw/olign_v1_{name}.json.gz"))
    print(name, {v for job in raw.values()
                 for k, v in job.get("_versions", {}).items()
                 if k.lower() == "x-olign-version"})
EOF

It prints {'v1.1.3'} for each test set when v1.1.3 aligned every clip. Your scores use your own copy of the reference, so expect them to differ from ours in the decimals.

Next steps

  • Our request for nyra's re-run
    Our pull request asks nyra to re-run Olign v1 and to record which release answered each hosted row. When nyra publishes its run, that is the number to use, and we will link it here. We have also opened an issue on nyra's MFA rows.

  • Try Olign on your own recording
    The Speech Timestamps Playground aligns up to 30 seconds of English speech, with no account.

  • Start with the Olign API
    Sign up and get free credits to align your own recordings.

  • See FA-Bench
    Olewave's own public benchmark of forced alignment and ASR with timestamps.