Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# LF in every checkout, Windows included: the PTX kernels are embedded with
# include_str!, so CRLF would change what a Windows build ships to the driver
* text=auto eol=lf
*.wav binary
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,10 @@ jobs:
os: ubuntu-24.04
- name: macOS arm64
os: macos-15
# the CUDA backend and the Python bindings' CUDA path, as on Linux; there is
# no GPU here either
- name: Windows x86_64
os: windows-2025
steps:
- uses: actions/checkout@v6

Expand Down
75 changes: 69 additions & 6 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,9 @@ jobs:
- name: Package and build as a dependency
run: just package

# one abi3 wheel per platform with the backend that platform runs; the Linux
# wheel builds without a GPU or CUDA toolkit because CUDA is loaded at run time
# one abi3 wheel per platform with the backend that platform runs; the Linux and
# Windows wheels build without a GPU or CUDA toolkit because CUDA is loaded at run
# time
wheels:
name: Wheel (${{ matrix.name }})
runs-on: ${{ matrix.os }}
Expand All @@ -73,13 +74,36 @@ jobs:
os: ubuntu-24.04
target: x86_64-unknown-linux-gnu
manylinux: "2_28"
rustflags: -D warnings
- name: macOS arm64, Core ML
os: macos-15
target: aarch64-apple-darwin
manylinux: "off"
rustflags: -D warnings
# the C runtime is linked statically, so the extension runs without the
# Visual C++ redistributable whichever Python loads it. RUSTFLAGS replaces the
# per-target flags rather than adding to them, so each target spells out its
# full set
- name: Windows x86_64, CUDA
os: windows-2025
target: x86_64-pc-windows-msvc
manylinux: "off"
rustflags: -D warnings -C target-feature=+crt-static
env:
RUSTFLAGS: ${{ matrix.rustflags }}
steps:
- uses: actions/checkout@v6

# a Windows checkout without symlink support writes the LICENSE symlink as a text
# file holding its target, which maturin would ship in the wheel as the license;
# removing it first keeps cp from writing through a real symlink onto itself
- name: Resolve the LICENSE symlink
if: runner.os == 'Windows'
shell: bash
run: |
rm -f crates/scriptrs-python/LICENSE
cp LICENSE crates/scriptrs-python/LICENSE

- name: Build wheel
uses: PyO3/maturin-action@e83996d129638aa358a18fbd1dfb82f0b0fb5d3b # v1.51.0
with:
Expand All @@ -102,6 +126,29 @@ jobs:
test "$size" -le "$limit"
done

# CUDA must load at run time and the C runtime is static, so the extension may
# import only Python's stable ABI and Windows system DLLs
- name: Check the extension's imports
if: runner.os == 'Windows'
shell: bash
run: |
set -euo pipefail
unpacked="$RUNNER_TEMP/wheel"
mkdir -p "$unpacked"
# 7-Zip ships on the runner image, and a wheel is a zip
7z x -y dist/*.whl -o"$unpacked" > /dev/null
ext="$(find "$unpacked" -name '_native*.pyd' -print -quit)"
test -n "$ext"
rustup component add llvm-tools
readobj="$(cygpath -u "$(rustc --print sysroot)")/lib/rustlib/x86_64-pc-windows-msvc/bin/llvm-readobj.exe"
# a Windows tool's output ends lines with CRLF
imports="$("$readobj" --coff-imports "$ext" | tr -d '\r' | sed -n 's/^ *Name: //p' | sort -u)"
echo "$imports"
if grep -Eiv '^(python3|kernel32|ntdll|advapi32|bcrypt|bcryptprimitives|ws2_32|userenv|secur32|crypt32|ole32|oleaut32|shell32|user32|synchronization|api-ms-win-core-[a-z0-9-]+)\.dll$' <<< "$imports"; then
echo "the extension imports a DLL outside Python and Windows itself" >&2
exit 1
fi

- name: Upload wheel
uses: actions/upload-artifact@v7
with:
Expand Down Expand Up @@ -175,6 +222,10 @@ jobs:
- name: macOS arm64
os: macos-15
target: aarch64-apple-darwin
# no NVIDIA driver either, so the same checks as on Linux
- name: Windows x86_64
os: windows-2025
target: x86_64-pc-windows-msvc
# the one abi3 wheel serves every version, so test the oldest and the newest
python: ["3.11", "3.14"]
env:
Expand All @@ -199,8 +250,14 @@ jobs:
uv pip install --python .venv-wheel dist/*.whl pytest numpy

- name: Run tests
shell: bash
working-directory: crates/scriptrs-python
run: ../../.venv-wheel/bin/python -m pytest -rA
run: |
set -euo pipefail
# Windows venvs keep the interpreter in Scripts, not bin
python=../../.venv-wheel/bin/python
if [ "$RUNNER_OS" = Windows ]; then python=../../.venv-wheel/Scripts/python.exe; fi
"$python" -m pytest -rA

# checks a tag before anything is published: the crate's version must match it, and
# CHANGELOG.md needs its entry, which becomes the GitHub release notes
Expand Down Expand Up @@ -260,7 +317,8 @@ jobs:
set -euo pipefail
version="${GITHUB_REF_NAME#v}"
ls -la dist
test "$(find dist -type f | wc -l)" -eq 3
# the Linux, macOS and Windows wheels and the sdist
test "$(find dist -type f | wc -l)" -eq 4
for file in dist/*; do
case "$(basename "$file")" in
"scriptrs-${version}"-*.whl | "scriptrs-${version}.tar.gz") ;;
Expand All @@ -286,6 +344,8 @@ jobs:
os: ubuntu-24.04
- name: macOS arm64
os: macos-15
- name: Windows x86_64
os: windows-2025
steps:
- uses: actions/checkout@v6

Expand All @@ -309,10 +369,13 @@ jobs:
test "$attempt" -lt 10
sleep 30
done
.venv-release/bin/python -c "import scriptrs; print(scriptrs.__version__, scriptrs.BACKEND)"
# Windows venvs keep their programs in Scripts, not bin
bin=.venv-release/bin
if [ "$RUNNER_OS" = Windows ]; then bin=.venv-release/Scripts; fi
"$bin/python" -c "import scriptrs; print(scriptrs.__version__, scriptrs.BACKEND)"
# only the macOS runner has a backend that can transcribe
if [ "$RUNNER_OS" = "macOS" ]; then
text="$(.venv-release/bin/scriptrs transcribe --model parakeet/v2 crates/scriptrs-python/tests/data/librispeech-1089-134686-0007.wav)"
text="$("$bin/scriptrs" transcribe --model parakeet/v2 crates/scriptrs-python/tests/data/librispeech-1089-134686-0007.wav)"
printf '%s\n' "$text"
grep -qi 'indifference' <<< "$text"
fi
Expand Down
18 changes: 14 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,21 @@ follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions f
[Semantic Versioning](https://semver.org/). The Rust crate and the Python package share
one version.

## [Unreleased]
## [0.5.0] - 2026-11-07

The first release on PyPI. It has breaking changes to the Rust API; the README's
"Breaking changes in 0.5.0" section lists what to write instead.

### Added

- Python package `scriptrs` on PyPI, with one wheel for every Python version from 3.11:
Core ML on macOS arm64 and CUDA on Linux x86_64. It includes the `scriptrs transcribe`
command.
Core ML on macOS arm64 and CUDA on Linux x86_64 and Windows x86_64. It includes the
`scriptrs transcribe` command.
- Windows x86_64 builds of the `cuda` feature. They open `nvcuda.dll` from System32
and cuBLAS 12 or 13 (`cublas64_12.dll` or `cublas64_13.dll` and its cuBLASLt) at
run time, never from the current directory. The Windows wheel installs cuBLAS from
`nvidia-cublas-cu12` and loads it before the first model. CI builds and tests them
on a runner without a GPU; they haven't run on a Windows GPU yet.
- `TranscriptionScheduler` batches windows across concurrent requests on CUDA, for
servers with many short requests. `SchedulerConfig::max_requests` bounds the requests
held at once, and `try_submit` returns `TranscriptionError::SchedulerFull` past it.
Expand All @@ -28,6 +33,11 @@ The first release on PyPI. It has breaking changes to the Rust API; the README's

### Changed

- The CUDA backend uses cudakit 0.3, which loads the CUDA libraries through its cudarc
fork. On Linux it opens versioned sonames such as `libcublas.so.12` before a
toolkit's unversioned `libcublas.so`, so it reuses the cuBLAS the Python wheel
loaded instead of opening a second copy. `low-precision` no longer depends on
upstream `cudarc`.
- `TranscriptionConfig` has only `language`. The frontend settings and the input limit
come from the model bundle. `from_deployment` and `validate_for` are gone.
- `TranscriptionError::Control` wraps `TranscriptionControlError` in place of the three
Expand Down Expand Up @@ -67,5 +77,5 @@ The first release on PyPI. It has breaking changes to the Rust API; the README's
- VAD regions are padded by 0.2 s so soft first words are kept.
- The relative-position attention runs as one fused kernel on CUDA.

[Unreleased]: https://github.com/avencera/scriptrs/compare/v0.4.0...HEAD
[0.5.0]: https://github.com/avencera/scriptrs/compare/v0.4.0...v0.5.0
[0.4.0]: https://github.com/avencera/scriptrs/releases/tag/v0.4.0
11 changes: 5 additions & 6 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 3 additions & 6 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ edition = "2024"
rust-version = "1.96"
exclude = ["fixtures/", "dev-tools/", "crates/"]
authors = ["Praveen Perera"]
description = "Fast NVIDIA Parakeet TDT v2 and v3 speech-to-text in Rust, on Core ML (macOS) and native CUDA (Linux)"
description = "Fast NVIDIA Parakeet TDT v2 and v3 speech-to-text in Rust, on Core ML (macOS) and native CUDA (Linux, Windows)"
license = "Apache-2.0"
repository = "https://github.com/avencera/scriptrs"
homepage = "https://github.com/avencera/scriptrs"
Expand All @@ -29,13 +29,10 @@ vad = ["long-form"]
cuda = ["dep:memmap2", "dep:safetensors", "dep:cudakit", "cudakit/libraries"]
# FP8, INT8 and NVFP4 encoder precisions on CUDA, and the weight files that store
# them; not recommended for most users: slower than FP16 one utterance at a time
low-precision = ["cuda", "dep:cudarc"]
low-precision = ["cuda"]

[dependencies]
cudakit = { version = "0.1", optional = true }
# only to turn on cuBLASLt in cudakit's cudarc, for the FP8 and NVFP4 encoder
# products; cudakit picks the CUDA version and loading
cudarc = { version = "0.19.10", optional = true, default-features = false, features = ["cublaslt"] }
cudakit = { version = "0.3", optional = true }
memmap2 = { version = "0.9", optional = true }
ndarray = "0.17.2"
realfft = "3.5.1"
Expand Down
37 changes: 32 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ Fast [NVIDIA Parakeet TDT](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) s

- Parakeet TDT 0.6B v2 (English) and v3 (25 European languages)
- Core ML on macOS
- Native CUDA on Linux, with no ONNX Runtime or PyTorch (`cuda` feature)
- Native CUDA on Linux, with no ONNX Runtime or PyTorch (`cuda` feature), and a
Windows build that has not run on a GPU yet (see [Windows](#windows))
- Long-form transcription of hour-long recordings, with optional VAD
- Token-level timestamps and confidences

Expand All @@ -28,15 +29,16 @@ The base crate exposes a single-chunk `TranscriptionPipeline`. Fast long-audio c
- Optional VAD-backed long-form region planning
- Scheduler for many inputs or concurrent requests of any length, batched on CUDA
- Native CoreML inference on macOS
- CUDA inference on Linux (`cuda` feature)
- CUDA inference on Linux (`cuda` feature), and on Windows x86_64 untested on a GPU
- Streaming transcription of many live streams at once with Nemotron Speech
Streaming EN 0.6B and Nemotron 3.5 ASR Streaming 0.6B on CUDA
- Hugging Face download support with optional local model loading
- Python package on PyPI, with wheels for macOS arm64 and Linux x86_64
- Python package on PyPI, with wheels for macOS arm64, Linux x86_64 and Windows x86_64

## What it does not do yet

- Windows support
- Tested Windows support: the Windows build passes CI, which has no GPU, and has
not transcribed on a Windows GPU yet
- Other ASR models
- Streaming on Core ML
- Stable public guarantees around model layout or long-form behavior
Expand All @@ -48,7 +50,7 @@ The base crate exposes a single-chunk `TranscriptionPipeline`. Fast long-audio c
scriptrs = "0.5"
```

For native CUDA on Linux:
For native CUDA on Linux or Windows:

```toml
[dependencies]
Expand Down Expand Up @@ -379,6 +381,31 @@ Model downloads use [`hf-fetch`](https://github.com/avencera/hf-fetch), a small
blocking client with rustls and the shared Python Hugging Face cache. The `online`
feature enables downloads; local model loading works without it.

### Windows

The `cuda` feature builds for Windows x86_64 with the MSVC toolchain. CUDA loads
at run time, so building needs no GPU or CUDA toolkit. CI runs clippy and the
tests on a Windows runner and builds and tests the Windows Python wheel there,
but that runner has no GPU, and nobody has transcribed on a Windows GPU yet.
Treat Windows as untested until then, and please open an issue with what you
find.

At run time it needs the NVIDIA driver, which installs `nvcuda.dll`, and cuBLAS
12 or 13: `cublas64_12.dll` or `cublas64_13.dll`, with the matching
`cublasLt64_*.dll` beside it. scriptrs looks for them in the executable's
directory and System32, then each absolute `PATH` entry, then
`%CUDA_PATH%\bin\x64` (where CUDA 13 keeps its DLLs) and `%CUDA_PATH%\bin`
(CUDA 12), so an installed CUDA toolkit is found without changing `PATH`. The
Python wheel installs cuBLAS 12 from the `nvidia-cublas-cu12` package instead.

The driver loads only from System32, in the Python package's driver check too,
and no CUDA library loads from the current directory or a relative `PATH`
entry, so a DLL planted in the folder a program runs from is never picked up.
Each library found on `PATH` or under `CUDA_PATH` loads by full path, and its
own imports come from its directory, so cuBLAS binds the cuBLASLt beside it.
The exception is a process that already loaded a cuBLASLt from another install:
Windows reuses that one, and a mismatched pair can fail its first product.

## Usage

### Short audio
Expand Down
10 changes: 5 additions & 5 deletions crates/scriptrs-python/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions crates/scriptrs-python/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ symphonia = { version = "0.6.1", default-features = false, features = [
] }

# one wheel per platform carries the backend that platform runs: Core ML on macOS
# and native CUDA on Linux x86_64, so a plain `maturin build` produces the release
# wheel. CUDA is loaded at run time, so the Linux wheel builds without a GPU or
# CUDA toolkit
[target.'cfg(all(target_os = "linux", target_arch = "x86_64"))'.dependencies]
# and native CUDA on Linux and Windows x86_64, so a plain `maturin build` produces
# the release wheel. CUDA is loaded at run time, so the Linux and Windows wheels
# build without a GPU or CUDA toolkit
[target.'cfg(all(any(target_os = "linux", target_os = "windows"), target_arch = "x86_64"))'.dependencies]
scriptrs = { path = "../..", features = ["online", "vad", "cuda"] }
Loading
Loading