Skip to content

Add modulo CLI: find, add, verify and run vectors; bundle manifests - #11

Merged
JenD86 merged 3 commits into
mainfrom
feat/modulo-cli
Oct 30, 2026
Merged

JenD86 merged 3 commits into
mainfrom
feat/modulo-cli

Conversation

@JenD86

@JenD86 JenD86 commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Summary

A modulo command (installed by pip install -e ., which setup.sh now does) so agents and people can find, add, check and run steering vectors. It never prompts, every subcommand takes --json, and the exit code is 1 when nothing was done.

  • search: one search over your registries, the community index and modulo_catalog.yaml. Each hit has a status (local / verified / community / catalog) and a next step.
  • list: shows what SteeringAgent would load for a model, and why.
  • add: adds vectors from hf:owner/repo[@rev], a huggingface.co URL or a local directory.
    • It pins the commit, inspects each file (format and layer from shape, metadata or filename) and checks it against the model's config.
    • Vectors are registered unverified, with a ±8 range.
    • In a bundle, it only uses paths that name the model (other models' vectors can share its hidden size) and keeps one layer per concept.
    • --as-index-entry and --as-manifest print drafts instead of registering.
  • extract, verify (strength sweep, --save-range) and run (the demo, which gains --model and --steer NAME=STRENGTH).
  • Bundle manifests (modulo-bundle.yaml): a repo describes its vectors (names, layers, descriptions, sign conventions, test prompts), validated against the index's field rules.
    • Without a manifest, sidecar metadata is read (rotalabs-style metadata.json / layer_N.json, including best_layer).
    • The index keeps holding expanded entries, not pointers to manifests, so all text that reaches a model's prompt shows up in the PR diff.
  • Thinking mode off: Qwen3-style chat templates think by default, which spent every generation's budget inside <think> (agent turns, verify samples, extraction). All prompt rendering now passes enable_thinking=False. Qwen2.5 renders byte-identical.
  • Smaller changes:
    • The loader accepts dict .pt files ({"vector": ...}).
    • registry.yaml keeps its header comments on save.
    • MODULO_MODEL sets the default model.
    • Index and manifest validation reject unfilled TODO placeholders.

Testing

  • CPU: 126 passed, 1 skipped. New tests/test_cli.py covers search, add against a fake hub, manifests, sidecars, local directories and naming.
  • H100: all 107 tests passed at the time, including the GPU steering tests, after the loader and thinking-mode changes.
  • End to end: modulo verify assistant_axis reproduces the known behaviour (+12 coherent, +15 muddled, −15 florid). add → verify → run --steer works on Qwen3-8B with rotalabs vectors.
  • Real repos tried: rotalabs, Butanium assistant-axis, Rachata.

Known limitations

  • verify's suggested range only detects repetition, so it suggests ±15 for assistant_axis, where the real positive cap is +12. Read the samples before trusting a range; the docs say so.
  • The rotalabs refusal vector is weak in Modulo at any allowed strength. Their own setting is about 70% of the residual norm, with the prompt steered as well. Not addressed here.
  • Range calibration is being rethought separately (setting a position within the model's natural range instead of pushing by a fixed amount). Not part of this PR.

🤖 Generated with Claude Code

JenD86 and others added 3 commits October 30, 2026 22:50
- modulo search: one search over your registries, the community index and
  modulo_catalog.yaml, with a status and next step per hit
- modulo add hf:owner/repo[@rev]: pin the commit, download and inspect vector
  files (format/layer from shape, dict .pt metadata or filename), check them
  against the model config, and register them unverified (±8). For bundles,
  keep only paths naming the model (other models can share hidden size) and
  one layer per concept. --dry-run, --as-index-entry for draft index entries
- modulo extract / verify (strength sweep, --save-range) / run (passthrough
  to agent/loop.py, which gains --model and --steer NAME=STRENGTH)
- --json everywhere with progress on stderr; exit 1 when nothing was done
- Sweep code moved to modulo/verify.py; model_dims to modulo/community.py
- Loader accepts dict .pt files ({"vector": ...}); registry.yaml keeps its
  header comments on save; MODULO_MODEL sets the default model
- setup.sh does an editable install; docs and CPU tests for the above

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Qwen3-style chat templates think by default, so every generation spent its
token budget inside <think>: verify samples, agent turns and extraction
activations never came from a real reply. All prompt rendering now goes
through modulo.chat.chat_prompt, which passes enable_thinking=False.
Templates without the variable ignore it (Qwen2.5 renders byte-identical).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- modulo-bundle.yaml (modulo/bundle.py): a repo describes its vectors (name,
  file, layer, description, sign convention, test prompts...), with
  top-level defaults. Validated; maintainer-only fields are rejected. When
  present, add uses only the manifest's vectors for the model.
- Without a manifest, read sidecar metadata (<stem>.json, metadata.json:
  behavior, model_name, layer_index, best_layer, extraction_method). On
  rotalabs this picks their best_layer 14 instead of the middle layer.
- add accepts a local directory; --as-manifest drafts a manifest. Draft
  modes print only YAML on stdout.
- Manifest and index validation reject unfilled TODO placeholders, so a
  draft's text can't reach a model's prompt.
- Index stays expanded entries (no pointer-style bundle entries) so all
  prompt text is reviewed in PR diffs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JenD86
JenD86 merged commit 2a0f696 into main Oct 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant