Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions performance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,4 @@
- [memo](memo/README.md): memo's own lists — paging past 500 rows, what a brief answer saves, and `people list` with kinds kept in the store.
- [funnel](funnel/README.md): wirecat-funnel, the tsnet helper for Tailscale Funnel: whether Tailscale's
log upload stays off, and Funnel end to end.
- [beeline](beeline/README.md): Beeline Cloud PBX call recordings, starting with local transcription speed.
6 changes: 6 additions & 0 deletions performance/beeline/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Beeline

Measurements of beeline, the CLI for Beeline Cloud PBX call recordings.

- [transcription](transcription/README.md): how fast local CPU recognition transcribes stereo calls, one channel at a
time (2026-11-13).
56 changes: 56 additions & 0 deletions performance/beeline/transcription/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Local call transcription speed in beeline

Measured 2026-11-13.

## Why it was run

The owner wanted to show a potential client a working demo of beeline, and asked how fast it transcribes calls.
beeline recognises speech on the user's own machine, on the CPU, with no GPU and no cloud service.

The question: is CPU-only local recognition fast enough to keep up with a call center's daily volume?

The decision it fed: ship local transcription in beeline 0.1.0, without a GPU or a cloud service.

## Setup

- **Engine.** `sherpa-onnx` 2.0.0 compiled to WebAssembly, on the CPU, 1 thread.
- **Model.** `gigaam-v3`, Russian with punctuation, about 232 MB.
- **Audio.** Real stereo calls from the owner's account. Channel 0 is the manager and channel 1 is the client. Each
channel is recognised separately.
- **beeline.** A build of beeline-cli 0.1.0 (cli-core 0.647.0) from a local checkout.
- **Machine.** The owner's laptop: AMD Ryzen AI 9 HX 470 (24 threads), 30 GB RAM, Linux 7.0 (Ubuntu), Node
v24.19.0.
- **Run.**

```sh
performance/beeline/transcription/run.sh <profile> <beeline-cli checkout> [calls]
```

[run.sh](run.sh) reads the profile's token from the system keyring. It then runs beeline in a throwaway `HOME`
with D-Bus turned off, so the run does not touch the owner's own store or keyring entry. The model folder is
linked in from the real one, so the model is not downloaded again. The script syncs the calls of the last two
days, then times `beeline <profile> meetings transcribe --max-meetings <calls> --json`. The time is wall time
for the whole command. The audio length comes from the command's own `audio_seconds` total.

## Results

| Audio | Wall time | Speed |
|---|---|---|
| 1644 s (27.4 min) | 222 s (3.7 min) | 7.4× faster than real time |

The run used the owner's real calls, so this page gives totals only. It has no call ids, names, phone numbers or
text.

## What it showed

- One CPU thread transcribes about 7 hours of calls in one hour of work.
- The owner took that as fast enough for a call center's daily volume, so beeline 0.1.0 ships local transcription
with no GPU and no cloud service.

## Not measured

- Several threads.
- Accuracy. There was no word error rate check.
- macOS and Windows.
- Calls longer than 60 minutes.
- Memory use.
24 changes: 24 additions & 0 deletions performance/beeline/transcription/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
# Usage: run.sh <profile> <beeline-cli checkout> [calls]
# Reads the profile's token from the keyring, then runs beeline in a throwaway HOME with no D-Bus,
# so the run never changes the owner's own state or keyring entry.
set -euo pipefail
[ $# -ge 2 ] || { echo "usage: $0 <profile> <beeline-cli checkout> [calls]" >&2; exit 2; }
profile=$1 root=$2 n=${3:-3}
BEELINE_TOKEN=$(secret-tool lookup service beeline-cli username "token:$profile" 2>/dev/null) || true
[ -n "$BEELINE_TOKEN" ] || { echo "no token for profile $profile" >&2; exit 1; }
export BEELINE_TOKEN
sandbox=$(mktemp -d); trap 'rm -rf "$sandbox"' EXIT
mkdir -p "$sandbox/data/WireCat/models"
ln -s "$HOME/.local/share/WireCat/models/audio" "$sandbox/data/WireCat/models/audio"
b() { HOME="$sandbox" XDG_CONFIG_HOME="$sandbox/config" XDG_DATA_HOME="$sandbox/data" XDG_STATE_HOME="$sandbox/state" \
XDG_CACHE_HOME="$sandbox/cache" MESSAGING_STORE="$sandbox/wirecat.db" DBUS_SESSION_BUS_ADDRESS=disabled: \
BEELINE_NO_UPDATE_CHECK=1 node "$root/dist/bin/beeline.js" "$profile" "$@"; }
since=$(node -p 'new Date(Date.now() - 2 * 864e5).toISOString().slice(0, 10) + "T00:00:00Z"')
b meetings sync --since-time "$since" --json >/dev/null
start=$(date +%s.%N)
b meetings transcribe --max-meetings "$n" --json 2>/dev/null | node -e '
let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{const r=JSON.parse(s).item
const wall=(Date.now()/1e3)-Number(process.argv[1]);
console.log(`calls ${r.transcribed}, audio ${Math.round(r.audio_seconds)} s, wall ${Math.round(wall)} s, ` +
`speed ${(r.audio_seconds/wall).toFixed(1)}x real time, failed ${r.failed.length}`)})' "$start"
Loading