Forced Alignment with Local Correction Capability

Inspect forced alignments of (learner) speech on a waveform + spectrogram, drag exactly the span that is wrong, fix its words, and re-align only that span with the Montreal Forced Aligner — with every correction recorded on a provenance tier and exportable as Praat TextGrid or JSON.

---

Contents

• Overview
• Features
• Quick start
• Installation
• Using the app
• Export formats
• Saving corrections
• Configuration
• Project layout
• Troubleshooting
• Changelog

---

Overview

Forced aligners are reliable on fluent read speech and unreliable on hesitant, pause-heavy learner speech: a single mis-transcribed or mis-placed stretch can drag a whole utterance out of place. Re-running the aligner on the entire file with a corrected transcript often reproduces the same drift.

This app takes a detect-then-correct approach. You look at the alignment on the signal, select exactly the region that is wrong, type what is actually said there, and the aligner runs on that clip only — anchored to the boundaries you chose, not to the boundaries the previous (wrong) alignment produced. The result is spliced back into the file's tiers, the correction is logged on an edits tier (old words → new words), and the corrected layer can be saved server-side or downloaded.

It works on a built-in example set (the NIAOpic synthetic-mismatch corpus, when available) or on any wav you upload, optionally with an existing TextGrid, and can transcribe uploads with CrisperWhisper.

Features

Signal view	Waveform + mel spectrogram in one scroll/zoom container (up to 480 px/s); words in the top lane, phones in the bottom lane as clickable regions; drag-to-select with Zoom to span / Play once / Loop.
Local correction	Drag the wrong region → edit its words → ⟳ Segment re-align. MFA aligns only that clip (wide beam), the result is spliced into the shown layer, the main transcript is updated, and the span is recorded on the edits tier.
Whole-file re-align	⟳ MFA re-align from the edited transcript, exp3-style: VAD-segmented, per-segment MFA, speaker-adapted model when available.
Transcription	🎙 Transcribe (CrisperWhisper) for uploaded wavs — verbatim, fillers kept as [UM]/[UH] (→ spn).
Inputs	Built-in examples, or your own wavs: single files, multi-select, or a whole folder; optional wav + TextGrid pairs matched by basename. Runs without the corpus tree (uploads-only mode).
Persistence & export	💾 Save realigned keeps the corrected layer on the server (restored as the file's default view); ⬇ TextGrid (Praat, 3 tiers) and ⬇ JSON downloads.
UI	Light / dark theme (remembered per browser), English labels, unsaved-change guards.

Quick start

git clone git@github.com:exphon/local_realign.git && cd local_realign
pip install -r requirements.txt                    # server python: numpy + torch (CPU is fine)
conda create -n mfa -c conda-forge python=3.10 montreal-forced-aligner=3.3.3
conda run -n mfa mfa model download acoustic english_us_arpa
conda run -n mfa mfa model download dictionary english_us_arpa
./run.sh                                            # builds the frontend, serves UI + API on http://0.0.0.0:8000

Open http://<host>:8000 from any machine on the network. Hot-reload development mode (Vite on :5173 proxying to the backend on :8000): ./run.sh dev.

run.sh uses python when available (including an activated environment), otherwise python3. Python 3.10+ is required. To select an interpreter explicitly, use PYTHON=/path/to/python ./run.sh; install requirements.txt into that interpreter if you need Silero VAD. Set PORT=8001 ./run.sh (or PORT=8001 ./run.sh dev) to use a different backend port.

Installation

Requirements are split by the process that needs them; each file documents its own install.

File	Used by	Install
requirements.txt	the server's Python — numpy, torch (Silero VAD only)	pip install -r requirements.txt — CPU-only torch is enough: add --index-url https://download.pytorch.org/whl/cpu
requirements-mfa.txt	conda env mfa — Montreal Forced Aligner (conda-forge; not pip-installable)	conda create -n mfa -c conda-forge python=3.10 montreal-forced-aligner=3.3.3, then mfa model download acoustic english_us_arpa and … dictionary english_us_arpa
requirements-crisperwhisper.txt	project env .venv-crisperwhisper — optional, only for 🎙 Transcribe	./setup-crisperwhisper.sh; official CrisperWhisper 2.0 with the Transformers backend for macOS, CPU or CUDA
frontend/package.json	Node ≥ 18 + npm	run.sh runs npm install and npm run build

Without torch the server still runs; whole-file re-align then skips the VAD split and aligns each file as one segment. Without the CrisperWhisper env everything except the Transcribe button works.

CrisperWhisper 2.0

./setup-crisperwhisper.sh       # uses uv + Python 3.12 when available, otherwise python3 + venv
./run.sh

Upload a WAV and click 🎙 Transcribe (CrisperWhisper 2.0). The default model is nyralabs/CrisperWhisper2.0_medium, with English verbatim transcription. Fillers, repetitions and false starts are kept. Audio longer than 30 seconds uses the official continuation algorithm; the app no longer splits audio and deletes repeated words itself. The transcript fills the editor for the existing MFA re-alignment workflow.

The backend automatically uses .venv-crisperwhisper/bin/python; conda is not required for transcription. The first transcription downloads model weights from Hugging Face and subsequent runs reuse its cache. HF_TOKEN or a cached Hugging Face login is used when available, but the app does not require a token before trying to load the model. Inference runs locally. Device selection is CUDA → Apple MPS → CPU; MPS and CPU use float32 by default. To force CPU or change model/language:

CRISPER_DEVICE=cpu ./run.sh
CRISPER_MODEL=large CRISPER_LANGUAGE=ko ./run.sh
# A separate interpreter or existing conda environment can also be selected:
CRISPER_PYTHON=/path/to/python ./run.sh
CONDA_BIN=/path/to/conda CRISPER_ENV=crisperWhisper ./run.sh

These environments need the current requirements-crisperwhisper.txt; the old v1 environment with Transformers 4.37.2 is not sufficient. Transcription jobs allow up to 30 minutes for model loading and inference. Direct CLI use is also available:

.venv-crisperwhisper/bin/python backend/transcribe_wrapper.py --f recording.wav

The integration uses crisperwhisper 2.0.3 with Transformers 4.57.6 pinned for long-audio compatibility (Transformers 5.19.0 failed in the upstream attention/cache path during local testing). See the official documentation for model capabilities. The model weights have a separate license from the inference code; see the medium model license.

Using the app

Sidebar

• Example list (top) — the built-in synthetic-mismatch set, filterable by proficiency level. Empty in uploads-only mode.
• My files (uploads) — ⬆ Select files (one or many wavs) or 📁 Folder. 16-bit PCM wav only. If the selection also contains .TextGrid files, each one whose basename matches a wav is attached as that file's uploaded layer. Files with a saved correction show 💾.
• ☀ Light / ☾ Dark — theme toggle (persisted in localStorage; first visit follows the OS preference).

Controls row

• Layer — Original exp3 (whisperx) · Uploaded TextGrid · Realigned (radio; unavailable layers are greyed). Default: the saved realigned layer if any, else uploaded, else exp3.
• ⬇ TextGrid / ⬇ JSON — download the shown layer (see Export formats).
• 💾 Save realigned / 🗑 — persist or delete the corrected layer (see Saving corrections).
• Words / Phones / Spectrogram — overlay toggles. The edits tier is always drawn when present.

Player

• Waveform and spectrogram scroll and zoom together; Zoom slider up to 480 px/s.
• Click a word or phone region to play just that interval.
• Drag on the waveform or spectrogram to select a span; the selection bar offers 🔍 Zoom to span, ▶ Play once, 🔁 Loop, ✕ Clear.

Editors

• Edit transcript → re-align — the whole-file transcript. ⟳ MFA re-align re-aligns the file from it (uploads: 🎙 Transcribe (CrisperWhisper) fills it first).
• Realign selection [t0–t1 s] — appears after a drag, prefilled with the words that overlap the selection. Edit them and press ⟳ Segment re-align.

Typical workflow

1. Pick a file (or upload wavs and transcribe them).
2. Listen and find a mis-aligned stretch on the waveform / spectrogram.
3. Drag from its true start to its true end — ignore the current word boxes; they may be what is wrong — fix the words in Realign selection, ⟳ Segment re-align. Repeat for other spans; each is appended to the edits tier.
4. 💾 Save realigned, and/or download ⬇ TextGrid / ⬇ JSON.

Export formats

Both buttons export the currently shown layer as <file>.<layer>.<ext>.

Button	File	Built by	Content
⬇ TextGrid	<file>.<layer>.TextGrid	server, POST /api/export_textgrid	Praat long format; interval tiers words, phones, and edits when present; gaps filled with empty intervals, overlaps resolved; xmax = wav duration. Opens directly in Praat.
⬇ JSON	<file>.<layer>.json	client, downloadJson()	Labelled intervals only (no gap fillers).

JSON layout:

{
  "format": "local_realign.tiers/1",
  "exported_at": "2026-09-21T09:12:33.120Z",
  "uid": "ESPEAK_2192_IG_STUD_F_CC_09__del31",
  "name": "ESPEAK_2192_IG_STUD_F_CC_09",
  "layer": "realigned",
  "saved_at": "2026-09-21T18:11:02+09:00",
  "xmax": 43.64,
  "tiers": {
    "words":  [{"start": 0.065, "end": 0.153, "text": "will"}, "…"],
    "phones": ["…"],
    "edits":  [{"start": 23.30, "end": 30.94,
                "text": "channels → explain your situation to the receptionist"}]
  }
}

saved_at is set when the exported realigned layer equals the saved one, otherwise null.

Saving corrections

Generated files are automatically written next to the WAV the server processes:

• CrisperWhisper completion writes <wav-stem>.txt (UTF-8, original transcription).
• Whole-file or segment MFA completion writes <wav-stem>.TextGrid. Segment alignment saves the complete merged timeline, including the edits tier, rather than only the clip.
• Repeating an operation replaces its output file. Cached MFA results also recreate the file. Files are replaced atomically; a write failure is reported instead of claiming success.
• For browser uploads the WAV is a server copy, so these files appear in backend/uploads/ beside <uid>.wav. The browser does not provide the original local folder path. The completion message shows the actual output path.

The app's separately saved layer is a snapshot restored when reopening a file:

• 💾 Save realigned posts the current realigned tiers to /api/save_realigned, which writes backend/saved/<uid>.TextGrid (canonical, Praat-openable) and <uid>.json (exact tiers + timestamp), atomically. It also updates the TextGrid beside the WAV.
• Afterwards GET /api/meta/<uid> reports saved / saved_at and lists realigned among the available layers, and GET /api/textgrid/<uid>?which=realigned serves it — for built-in examples and uploads alike. Reloading the page or restarting the server restores the layer as the file's default view; the sidebar marks the file with 💾.
• The button reads 💾 Save realigned ● while the in-memory layer differs from the saved one and 💾 Saved once they match. Switching files or closing the tab with unsaved changes asks for confirmation.
• 🗑 deletes the app's saved layer (with confirmation); the in-memory layer is kept and marked unsaved. Generated files beside the WAV are retained.
• backend/saved/ is git-ignored — back it up with your data.

Project layout

backend/
  server.py                 stdlib HTTP server + API; MFA / CrisperWhisper via `conda run`
  transcribe_wrapper.py     CrisperWhisper 2.0 CLI wrapper (project venv or configured runtime)
  uploads/, saved/          uploaded wavs (+ index.json), saved realigned layers      [git-ignored]
frontend/
  src/App.jsx               sidebar, layers, transcript editor, span editor, save / export
  src/components/Player.jsx wavesurfer viewer: regions, selection, theme colours
  src/api.js, src/styles.css
docs/                       screenshots used in this README
paper_figures/              figures used in the accompanying paper
run.sh                      build the frontend and serve everything on :8000
requirements*.txt           server python / mfa env / crisperWhisper env

Troubleshooting

Symptom	Cause / fix
A button described here is missing	the browser tab holds a stale bundle — hard-reload (Ctrl+Shift+R)
MFA produced no TextGrid for the clip (… words: "…")	the dragged span is shorter than the words entered, or the words are not what is said there; widen the span or fix the words
Result looks "squeezed", status warns > 5 words/s	same as above — the span was too short for the transcript
CrisperWhisper dependencies missing / runtime not found	run ./setup-crisperwhisper.sh; check CRISPER_PYTHON / CRISPER_ENV overrides
CrisperWhisper model download fails	check network access; for a restricted model, verify Hugging Face access and HF_TOKEN
CrisperWhisper MPS error or insufficient memory	restart with CRISPER_DEVICE=cpu ./run.sh or select CRISPER_MODEL=small
MFA / conda executable not found	existing conda environments are detected automatically; for a custom installation use MFA_BIN=/path/to/env/bin/mfa ./run.sh (its bin directory is also added to the MFA subprocess PATH)
Example list is empty	NIAOPIC_ROOT does not point at the corpus tree — expected in uploads-only mode
Re-align aligns each file as one segment	torch is not installed in the server's Python (Silero VAD unavailable)

Changelog

2026-09-22 — README redesign; screenshots in docs/.

2026-09-21 — edits provenance tier for re-aligned spans (drawn on the waveform, included in TextGrid/JSON exports and saved layers); ⬇ JSON export; export names keep the layer suffix; duration in /api/meta.

2026-09-18 — Portability: requirements*.txt; server runs without the corpus tree (NIAOPIC_ROOT, uploads-only mode); Silero VAD torch.hub fallback; bundled transcribe_wrapper.py.

2026-09-15 — 💾 Save realigned / 🗑 (/api/save_realigned, /api/discard_saved), saved layer restored as default view, unsaved-change guards; span edits propagate to the main transcript.

2026-09-14 — Light / dark theme; English UI; title renamed; layer menu reduced to exp3 / uploaded / realigned; GT / ASR overlays and transcript-fill buttons removed from the UI.

2026-09-11 — Span re-align uses the dragged region exactly (no snapping); straddling intervals are clipped; MFA beam 100 / 400 for spans; status reports words / clip seconds and warns above 5 words/s.