diff --git a/performance/README.md b/performance/README.md index b58257f..1bdfc89 100644 --- a/performance/README.md +++ b/performance/README.md @@ -30,3 +30,4 @@ - [memo](memo/README.md): memo's own lists — paging past 500 rows, what a brief answer saves, and `people list` with kinds kept in the store. - [funnel](funnel/README.md): wirecat-funnel, the tsnet helper for Tailscale Funnel: whether Tailscale's log upload stays off, and Funnel end to end. +- [beeline](beeline/README.md): Beeline Cloud PBX call recordings, starting with local transcription speed. diff --git a/performance/beeline/README.md b/performance/beeline/README.md new file mode 100644 index 0000000..1f696d4 --- /dev/null +++ b/performance/beeline/README.md @@ -0,0 +1,6 @@ +# Beeline + +Measurements of beeline, the CLI for Beeline Cloud PBX call recordings. + +- [transcription](transcription/README.md): how fast local CPU recognition transcribes stereo calls, one channel at a + time (2026-11-13). diff --git a/performance/beeline/transcription/README.md b/performance/beeline/transcription/README.md new file mode 100644 index 0000000..e2597a7 --- /dev/null +++ b/performance/beeline/transcription/README.md @@ -0,0 +1,56 @@ +# Local call transcription speed in beeline + +Measured 2026-11-13. + +## Why it was run + +The owner wanted to show a potential client a working demo of beeline, and asked how fast it transcribes calls. +beeline recognises speech on the user's own machine, on the CPU, with no GPU and no cloud service. + +The question: is CPU-only local recognition fast enough to keep up with a call center's daily volume? + +The decision it fed: ship local transcription in beeline 0.1.0, without a GPU or a cloud service. + +## Setup + +- **Engine.** `sherpa-onnx` 2.0.0 compiled to WebAssembly, on the CPU, 1 thread. +- **Model.** `gigaam-v3`, Russian with punctuation, about 232 MB. +- **Audio.** Real stereo calls from the owner's account. Channel 0 is the manager and channel 1 is the client. Each + channel is recognised separately. +- **beeline.** A build of beeline-cli 0.1.0 (cli-core 0.647.0) from a local checkout. +- **Machine.** The owner's laptop: AMD Ryzen AI 9 HX 470 (24 threads), 30 GB RAM, Linux 7.0 (Ubuntu), Node + v24.19.0. +- **Run.** + + ```sh + performance/beeline/transcription/run.sh [calls] + ``` + + [run.sh](run.sh) reads the profile's token from the system keyring. It then runs beeline in a throwaway `HOME` + with D-Bus turned off, so the run does not touch the owner's own store or keyring entry. The model folder is + linked in from the real one, so the model is not downloaded again. The script syncs the calls of the last two + days, then times `beeline meetings transcribe --max-meetings --json`. The time is wall time + for the whole command. The audio length comes from the command's own `audio_seconds` total. + +## Results + +| Audio | Wall time | Speed | +|---|---|---| +| 1644 s (27.4 min) | 222 s (3.7 min) | 7.4× faster than real time | + +The run used the owner's real calls, so this page gives totals only. It has no call ids, names, phone numbers or +text. + +## What it showed + +- One CPU thread transcribes about 7 hours of calls in one hour of work. +- The owner took that as fast enough for a call center's daily volume, so beeline 0.1.0 ships local transcription + with no GPU and no cloud service. + +## Not measured + +- Several threads. +- Accuracy. There was no word error rate check. +- macOS and Windows. +- Calls longer than 60 minutes. +- Memory use. diff --git a/performance/beeline/transcription/run.sh b/performance/beeline/transcription/run.sh new file mode 100755 index 0000000..53730f3 --- /dev/null +++ b/performance/beeline/transcription/run.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +# Usage: run.sh [calls] +# Reads the profile's token from the keyring, then runs beeline in a throwaway HOME with no D-Bus, +# so the run never changes the owner's own state or keyring entry. +set -euo pipefail +[ $# -ge 2 ] || { echo "usage: $0 [calls]" >&2; exit 2; } +profile=$1 root=$2 n=${3:-3} +BEELINE_TOKEN=$(secret-tool lookup service beeline-cli username "token:$profile" 2>/dev/null) || true +[ -n "$BEELINE_TOKEN" ] || { echo "no token for profile $profile" >&2; exit 1; } +export BEELINE_TOKEN +sandbox=$(mktemp -d); trap 'rm -rf "$sandbox"' EXIT +mkdir -p "$sandbox/data/WireCat/models" +ln -s "$HOME/.local/share/WireCat/models/audio" "$sandbox/data/WireCat/models/audio" +b() { HOME="$sandbox" XDG_CONFIG_HOME="$sandbox/config" XDG_DATA_HOME="$sandbox/data" XDG_STATE_HOME="$sandbox/state" \ + XDG_CACHE_HOME="$sandbox/cache" MESSAGING_STORE="$sandbox/wirecat.db" DBUS_SESSION_BUS_ADDRESS=disabled: \ + BEELINE_NO_UPDATE_CHECK=1 node "$root/dist/bin/beeline.js" "$profile" "$@"; } +since=$(node -p 'new Date(Date.now() - 2 * 864e5).toISOString().slice(0, 10) + "T00:00:00Z"') +b meetings sync --since-time "$since" --json >/dev/null +start=$(date +%s.%N) +b meetings transcribe --max-meetings "$n" --json 2>/dev/null | node -e ' +let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{const r=JSON.parse(s).item +const wall=(Date.now()/1e3)-Number(process.argv[1]); +console.log(`calls ${r.transcribed}, audio ${Math.round(r.audio_seconds)} s, wall ${Math.round(wall)} s, ` + +`speed ${(r.audio_seconds/wall).toFixed(1)}x real time, failed ${r.failed.length}`)})' "$start"