Repository navigation
feat: clef-flash (Cloudflare 9B decision model) on Core ML + ticket-triage demo - #34
Merged
Merged
Conversation
Cloudflare clef-flash (Qwen3.5-9B decision model) text-only on Core ML: 8 decoder packages (4 layers, 8-bit, one function per bucket) chained on the GPU + the joint schema head; host embedding gather, untied output embeddings for the head's lexical rows, text-only RoPE. ~0.50 s per 3-question ticket at L512 on an M5 Pro; same answers as the Python Core ML pipeline. ClefRecordEncoder now takes the image token ids directly so text-only callers don't need a vision config. ClefFlashDemo: incoming queue -> decided feed (team / urgency / refund + ms), fictional mock tickets, typed tickets jump the queue, Restart. demo.sh opens Ghostty with macmon + the decision log. Bundle comes from model-lab models/clef-flash-coreml (not on Hugging Face).
…lash-coreml @c1b8fdae Downloads the 11 GB bundle (8 decoder parts, head, embedding tables, tokenizer, manifest) with per-file SHA-256. ClefFlashDemo uses it unless CLEF_FLASH_BUNDLE points at a local bundle. Tests: manifest parsing, text-only RoPE tables, store asset list.
Alex-Wengg
added a commit
that referenced
this pull request
Oct 10, 2026
…et demo (#36) Stacked on #34. Adds the 0.6B student distilled from Cloudflare clef-flash: [FluidInference/clef-text-0.6b-coreml](https://huggingface.co/FluidInference/clef-text-0.6b-coreml) @ `3c61a609` (PyTorch: [FluidInference/clef-text-0.6b](https://huggingface.co/FluidInference/clef-text-0.6b)). - `ClefTextManager`: Qwen3-0.6B + Clef joint head. One multifunction fp16 decoder (L256–L2048) plus an fp16 head; the host does the embedding gather, RoPE and causal mask. Runs on `.cpuAndGPU` (~48 ms per 3-question ticket) or 100% on the Neural Engine (~77 ms). - `ClefTextModelStore`: pinned revision, per-file SHA-256 (~1.3 GB). - `ClefFlashDemo`: `CLEF_MODEL=text` runs the 0.6B; fixed 1,000-ticket mock backlog with progress and total time ("1,000 tickets in 57 s"); `bars.py` CPU/GPU bar charts over `macmon pipe` (asitop crashes on M5 Pro, and the CPU/ANE power counters read 0 W on macOS 27). - `ClefCompareDemo`: 9B on the GPU and 0.6B on the ANE side by side (85% of decisions match, 5.6× faster). - `ClefTextCheck`: parity CLI. Swift vs PyTorch on the demo tickets: identical tokens, 0 / 60 flips. Student quality (held-out test splits): 79.1 vs teacher 88.2. Classification is within 0–4 points; knowledge QA is far behind (table on the HF card). Reviewer notes: `ClefTextTests` need no model. As in #34, they compile and run on CI only (no local XCTest). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runs Cloudflare's clef-flash (Qwen3.5-9B decision model, Apache-2.0) on the Mac GPU through Core ML, text only. Bundle: FluidInference/clef-flash-coreml @
c1b8fdae.ClefFlashManager: chains 8 four-layer decoder packages (8-bit, one function per 256–2048 bucket), then the joint schema head. The host does Clef's record rendering, the fp16 embedding gather, text-only RoPE and the untied output embeddings for the head's lexical rows.ClefFlashModelStore: pinned revision, per-file SHA-256 (~11 GB).ClefFlashDemo: SwiftUI support-ticket triage. Incoming queue → decided feed with team / urgency / refund + ms; fictional mock tickets; typed tickets jump the queue; Restart.demo.shalso opens Ghostty with macmon and the decision log.ClefRecordEncodernow takes the image token ids directly, so text-only callers don't needClefVisionConfig. ClefVision behaviour is unchanged.Numbers (M5 Pro): ~0.49 s per 3-question ticket (p95 0.57 s). 8-bit vs a streamed fp32 reference: 1 / 611 answers differ (a near-tie), ARC-E 100 / ARC-C 99.0. The decoder puts 0 ops on the ANE, so this is a GPU-only model.
Reviewer notes: the tests (
ClefFlashTests) need no model, but they couldn't run locally (no XCTest toolchain), so CI is the check. The decoder needs ~7 GB of RAM to itself.🤖 Generated with Claude Code