Repository navigation
feat: clef-text-0.6b (distilled from clef-flash) runtime + 1,000-ticket demo - #36
Merged
Merged
Conversation
…-ticket demo ClefTextManager runs FluidInference/clef-text-0.6b-coreml (Qwen3-0.6B + Clef joint head distilled from clef-flash): one multifunction fp16 decoder (L256-L2048) + fp16 head, host embedding gather / RoPE / causal mask, .cpuAndGPU or .cpuAndNeuralEngine. ClefTextModelStore pins @3c61a609 with per-file SHA-256. Swift == PyTorch on the demo tickets (identical tokens, 0/60 flips, ~48 ms GPU / ~77 ms ANE per ticket). ClefFlashDemo: CLEF_MODEL=text runs the 0.6B; fixed 1,000-ticket mock backlog with progress / total time; queue updates without animation (tickets arrive every ~50 ms); bars.py CPU / GPU bar charts over macmon replace the power pane. ClefCompareDemo: 9B (GPU) and 0.6B (ANE) side by side. ClefTextCheck CLI for parity runs.
Alex-Wengg
force-pushed
the
feat/clef-text-demo
branch
from
October 10, 2026 17:15
272ee16 to
b601cc7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #34. Adds the 0.6B student distilled from Cloudflare clef-flash: FluidInference/clef-text-0.6b-coreml @
3c61a609(PyTorch: FluidInference/clef-text-0.6b).ClefTextManager: Qwen3-0.6B + Clef joint head. One multifunction fp16 decoder (L256–L2048) plus an fp16 head; the host does the embedding gather, RoPE and causal mask. Runs on.cpuAndGPU(~48 ms per 3-question ticket) or 100% on the Neural Engine (~77 ms).ClefTextModelStore: pinned revision, per-file SHA-256 (~1.3 GB).ClefFlashDemo:CLEF_MODEL=textruns the 0.6B; fixed 1,000-ticket mock backlog with progress and total time ("1,000 tickets in 57 s");bars.pyCPU/GPU bar charts overmacmon pipe(asitop crashes on M5 Pro, and the CPU/ANE power counters read 0 W on macOS 27).ClefCompareDemo: 9B on the GPU and 0.6B on the ANE side by side (85% of decisions match, 5.6× faster).ClefTextCheck: parity CLI. Swift vs PyTorch on the demo tickets: identical tokens, 0 / 60 flips.Student quality (held-out test splits): 79.1 vs teacher 88.2. Classification is within 0–4 points; knowledge QA is far behind (table on the HF card).
Reviewer notes:
ClefTextTestsneed no model. As in #34, they compile and run on CI only (no local XCTest).🤖 Generated with Claude Code