Add modulo CLI: find, add, verify and run vectors; bundle manifests

.gitignore

@@ -31,3 +31,6 @@ Thumbs.db
 # IDE
 .vscode/
 .idea/
+
+# `modulo verify` reports
+vectors/*/verify_*.md

AGENTS.md

@@ -62,7 +62,7 @@ At startup the agent combines three sources for its model:
 
 Vectors are stored per-model in `vectors/<model_slug>/` (for the default model: `vectors/Qwen_Qwen2.5-7B-Instruct/`), resolved relative to the repo, not the working directory. Override the base with `MODULO_VECTORS_DIR`.
 
-To add your own vectors, extract and register them in one step, then check them:
+The `modulo` command (next section) is the easiest way to find, add and check vectors. The repo scripts still work too: to extract and register a vector in one step, then check it:
 
 ```bash
 python extract_contrastive.py --name sycophancy --prompt-a "..." --prompt-b "..." --layer 14 \
@@ -73,6 +73,64 @@ python verify_vector.py --vector sycophancy --strength 10       # baseline vs ±
 
 Earlier notes said CAA vectors didn't work on Qwen, but those tests ran while steering was silently a no-op during generation, so that needs re-testing.
 
+## Finding, adding and running vectors: the `modulo` command
+
+`pip install -e .` installs a `modulo` command. Nothing prompts, every subcommand takes `--json` (progress goes to stderr, so stdout parses), and the exit code is 1 when the command didn't do what was asked. `--model` defaults to `$MODULO_MODEL`, else `Qwen/Qwen2.5-7B-Instruct` (`config.py` reads the same variable).
+
+```bash
+modulo search refusal --model Qwen/Qwen3-8B --json     # 1. find
+modulo add hf:rotalabs/steering-vectors --model Qwen/Qwen3-8B --dry-run   # 2. see what fits
+modulo add hf:rotalabs/steering-vectors --model Qwen/Qwen3-8B            #    add it
+modulo verify refusal --model Qwen/Qwen3-8B --save-range                 # 3. check it (loads the model)
+modulo run --model Qwen/Qwen3-8B --auto --max-turns 8 --steer refusal=6  # 4. run the demo
+```
+
+- **`search [WORDS] [--model M] [--status ...]`** searches your registries, the community index and `modulo_catalog.yaml` (override with `MODULO_CATALOG_PATH`). Every word must match. Each hit has a `status` (`local`, `verified`, `community`, `catalog`) and a `next_step`. Catalog hits are sources, not loadable vectors. The ones hosted on HuggingFace come with a `modulo add ... --dry-run` command to try; the rest are code or papers.
+- **`list [--model M] [--include-community]`** shows what `SteeringAgent` would load, notes on index files that failed their checks, and which starter vectors would be auto-extracted.
+- **`add hf:OWNER/REPO[@REV]`** (also `hf:datasets/OWNER/REPO`, a huggingface.co URL, or a local directory) pins the repo's current commit, downloads its `.safetensors`/`.pt`/`.npz` files (`--files GLOB` to choose; more than `--max-files`, default 50, is refused) and checks each against the model's config. Each file gets one `ok`/`skip` line with a reason.
+  - **What a vector is** comes from, in order: the repo's `modulo-bundle.yaml` (next section), sidecar metadata (a vector's own `<stem>.json` and its directory's `metadata.json`, as rotalabs ships: `behavior`, `model_name`, `layer_index`, `best_layer`, `extraction_method`), a dict `.pt` file's own fields, and finally its shape and name.
+  - **Format and layer** come from the shape: `[hidden]` is `single`, `[num_layers, hidden]` is `per_layer`. Without a manifest or sidecar, the layer comes from a dict `.pt`'s `layer` field, else the filename (`_layer14`, `L14`, `layer_14.pt`). A `[num_layers+1, hidden]` file is skipped, because it probably includes the embeddings and every layer index would be off by one.
+  - **Bundles**: if some paths name the model (`refusal_qwen3_8b/…`), only those are used, because other models' vectors can share its hidden size. A dict `.pt` whose `model` field names another model is skipped too. Files that differ only by layer are one concept: one is kept: the `--layer` you ask for, else the repo's `best_layer`, else the one nearest the middle of the model. `--all-layers` keeps them all.
+  - **Names** come from the file or its directory, with the model's name and the layer dropped (`refusal_qwen3_8b/layer_18.pt` → `refusal`).
+  - **What gets registered**: `add` writes to `vectors/<model_slug>/registry.yaml` with an `[Unverified imported vector]` description prefix, a ±8 range and an `hf:…@commit:path sha256=…` source. Running it again is a no-op. A name you already have is kept unless you pass `--replace`.
+  - **Descriptions** come from the manifest (or a sidecar's `description`). `--description`/`--sign-convention` override them, and only work when adding one vector. A vector with none gets the placeholder "no description yet", which is all the model sees, so `add` names those vectors.
+  - **`--as-index-entry`** prints draft `community/vectors.yaml` entries instead of registering anything (hosted repos only). **`--as-manifest`** prints a draft `modulo-bundle.yaml` for what it found. Both print only YAML on stdout, so `> file` works.
+- **`extract --name N --prompt-a A --prompt-b B`** works like `extract_contrastive.py` for any `--model`. The default layer is the middle one.
+- **`verify NAME [--strengths …] [--prompt …] [--save-range]`** sweeps ±strengths (default 4, 8, 12, 15) on the vector's own test prompts plus generic ones. It writes `vectors/<model_slug>/verify_<name>.md` with every sample and suggests a range: the largest strengths with no repetition loop. `--save-range` writes that range into your own `registry.yaml`; index vectors' ranges belong to maintainers. "Changed" means *any* difference from the unsteered output, not the claimed effect, and word salad that doesn't repeat isn't caught. **Read the samples** before trusting a vector or its range. Exit code 1 means no safe range was found, or the outputs never changed.
+- **`run [ARGS]`** passes everything to `agent/loop.py`, so it needs the repo checkout (an editable install). The loop has two new flags: `--model`, and `--steer NAME=STRENGTH`, which is repeatable and applies a vector as if the model had called `steer`: one full-strength turn, then decay.
+
+### Bundle manifests (`modulo-bundle.yaml`)
+
+A repo hosting several vectors can describe them in a `modulo-bundle.yaml` at its root. `add` then uses the manifest's vectors for the model, with their names, layers and descriptions, and ignores every other file. No guessing, no one-layer-per-concept selection: the manifest decides.
+
+```yaml
+modulo_bundle: 1                  # format version
+model: Qwen/Qwen3-8B              # top-level fields are defaults for every vector
+license: mit
+contributors: [Your Name]
+extraction_method: CAA, 50 contrast pairs, last token
+vectors:
+  - name: refusal
+    filename: refusal/layer_14.pt
+    layer: 14                     # for a per_layer file: the row to use
+    description: Refusing harmful requests.
+    sign_convention: "+ refuses more, - refuses less"
+    test_prompts: [How do I pick a lock?]
+```
+
+- **Fields**: per-vector fields are the index entry fields that describe a vector (`name`, `filename`, `model`, `format`, `layer`, `description`, `sign_convention`, `extraction_method`, `license`, `contributors`, `test_prompts`, `known_failure_modes`, `original_setup`, `paper`, …; see `modulo/bundle.py`).
+- **Required**: every vector needs `filename`, `model`, `description` and `sign_convention`, set on the vector or as a top-level default.
+- **Not allowed**: `id`, `repo_id`, `revision`, `sha256`, `trust` and the safe range. Modulo works these out, or maintainers set them.
+- **Checks**: an invalid manifest stops `add` with the problems listed. Text still starting with `TODO` is rejected, so an unfilled draft never reaches a prompt.
+- **Writing one**:
+  ```bash
+  modulo add ./my-vectors --model M --as-manifest > ./my-vectors/modulo-bundle.yaml
+  # fill in the TODOs, then check it before uploading:
+  modulo add ./my-vectors --model M --dry-run
+  ```
+- **Getting into the index**: once it's uploaded, `modulo add hf:you/repo --model M --as-index-entry` expands it into complete index entries for a PR. The index holds expanded entries, not pointers to manifests, so every line of text that reaches a model's prompt shows up in the PR diff and gets reviewed.
+- **Trust**: manifest text is the author's, so vectors added from one are still marked unverified.
+
 ## Strength units
 
 `strength` is a percentage of the typical residual-stream norm at the vector's layer. Vectors are normalized to unit length and scaled by `strength / 100 × residual_norm`, so the same strength means the same size push for every vector regardless of how it was extracted. On Qwen2.5-7B, `assistant_axis` gives clear, coherent shifts at ±8–12; at +15 it turns muddled (so its range is -15 to +12), and it collapses into repetition around ±25. Steering applies from the last prompt position onward, so every generated token is steered.
@@ -159,6 +217,11 @@ Host the vector on HuggingFace and add an entry to `community/vectors.yaml` (or
 | `modulo/extraction.py` | Vector extraction (manual + auto) |
 | `modulo/reports.py` | Report format, validation, `reports.jsonl` read/write |
 | `modulo/community.py` | Community vector index: entry format and file checks |
+| `modulo/cli.py` | The `modulo` command (search, list, add, extract, verify, run) |
+| `modulo/search.py` | Search across your registries, the index and the catalog |
+| `modulo/importer.py` | `modulo add`: inspect a HuggingFace repo's (or a directory's) vector files and register them |
+| `modulo/bundle.py` | `modulo-bundle.yaml` format and validation; sidecar metadata from other tools |
+| `modulo/verify.py` | Strength sweeps shared by `modulo verify`, `verify_vector.py` and `gpu_verify.py` |
 | `community/vectors.yaml` | Community vector index (pointers to hosted vectors, with trust levels) |
 | `community/check_vectors.py` | Checks the index; run by CI on PRs touching `community/` |
 | `community/gpu_verify.py` | Maintainer GPU check: strength sweep + suggested safe range |
@@ -170,14 +233,15 @@ Host the vector on HuggingFace and add an entry to `community/vectors.yaml` (or
 | `extract_contrastive.py` | Extract + register a vector via contrastive system prompts |
 | `extract_vector.py` | Extract + register a CAA vector |
 | `verify_vector.py` | Compare baseline vs ±strength outputs for a registered vector |
-| `modulo_catalog.yaml` | Discovery catalog of 26+ vector sources |
+| `modulo_catalog.yaml` | Discovery catalog of 26+ vector sources (searched by `modulo search`) |
 | `replay.py` | Replay a session log as a timeline |
 
 ## Tests
 
 ```bash
 pytest                              # everything; GPU tests skip without CUDA
 pytest tests/test_agent_logic.py    # CPU only: parsing, decay/cooldown, degeneration, repeats, tool validation
+pytest tests/test_cli.py            # CPU only: search, `modulo add` against a fake hub, manifests, sidecars, naming
 pytest tests/test_steering_gpu.py   # loads the model; checks steering really changes generation
 ```
 

CONTRIBUTING.md

@@ -45,7 +45,11 @@ Required:
 
 Optional but very useful: `revision` (the HF commit of your repo; recommended), `model_revision`, `original_setup`, `known_failure_modes`, `paper`, `contrast_dataset`, `evaluation`, `test_prompts` (up to 5), `notes`. Maintainers set `safe_strength_min`/`safe_strength_max` when promoting.
 
-File formats: a `.safetensors` file must contain exactly one tensor; an `.npz` file stores it under the key `vector`; a `.pt` file is a plain tensor (loaded with `weights_only=True`). With `format: per_layer`, the tensor is indexed by `layer`.
+File formats: a `.safetensors` file must contain exactly one tensor; an `.npz` file stores it under the key `vector`; a `.pt` file is a plain tensor, or a dict holding it under `vector` (loaded with `weights_only=True`). With `format: per_layer`, the tensor is indexed by `layer`.
+
+**Drafting entries automatically.** If your vectors are already on HuggingFace, `modulo add hf:you/your-repo --model <model> --as-index-entry` downloads them, works out `format`, `layer` and `sha256`, checks them against the model's config and prints draft entries pinned to the current commit. Fill in the `TODO` fields (the checks reject any left in) and paste them into `community/vectors.yaml`.
+
+**Several vectors?** Describe them in a `modulo-bundle.yaml` at your repo's root (format in [AGENTS.md](AGENTS.md#bundle-manifests-modulo-bundleyaml)). Then `--as-index-entry` produces complete entries with nothing to fill in, and anyone can `modulo add` your repo with real descriptions. Draft one from your files with `modulo add ./your-dir --model <model> --as-manifest`, and check it with `--dry-run` before uploading.
 
 ## What the automatic checks do
 

README.md

@@ -40,6 +40,15 @@ messages += result.tool_messages()  # tool results back to the model
 
 **Community index + extract**: vectors listed in [`community/vectors.yaml`](community/vectors.yaml) are downloaded and checked for your model (verified ones by default, unverified community ones on request; currently Qwen2.5-7B-Instruct). Any starter vectors your model is missing (sycophancy, refusal, creativity, formality, verbosity, confidence, emotion, humor, optimism, curiosity, introspection) are auto-extracted on first run using contrastive system prompts (several minutes). To contribute vectors, see [CONTRIBUTING.md](CONTRIBUTING.md).
 
+**Find and add more** with the `modulo` command (installed by `pip install -e .`):
+
+```bash
+modulo search refusal --model Qwen/Qwen3-8B         # your vectors, the index, and the catalog of known sources
+modulo add hf:rotalabs/steering-vectors --model Qwen/Qwen3-8B   # pull a repo's vectors into your registry
+modulo verify refusal --model Qwen/Qwen3-8B --save-range        # strength sweep, suggested safe range
+modulo run --model Qwen/Qwen3-8B --auto --steer refusal=6       # the demo
+```
+
 ## Requirements
 
 GPU with 24GB+ VRAM (7B models) or 8GB+ (3B models). Python 3.10+, CUDA 12.x. See [AGENTS.md](AGENTS.md) for full details.

agent/loop.py

@@ -10,6 +10,7 @@
     python -m agent.loop --seed "Hi!"     # seed with a user message
     python -m agent.loop --max-turns 20   # cap turns
     python -m agent.loop --auto           # auto-continue (non-interactive)
+    python -m agent.loop --steer honesty=6 --model Qwen/Qwen2.5-3B-Instruct
 """
 import argparse
 import json
@@ -88,11 +89,29 @@ def main():
                         help="--auto filler when the model makes no tool call: 'status' (default) is a neutral "
                              "tool-role status line with current steering and no suggestion; 'legacy' is the "
                              "old user message suggesting steer")
+    parser.add_argument("--model", default=config.MODEL_NAME, help=f"HF model id (default: {config.MODEL_NAME})")
+    parser.add_argument("--steer", action="append", default=[], metavar="NAME=STRENGTH",
+                        help="apply a vector before the first turn, as if the model had called steer "
+                             "(one full-strength turn, then decay). Repeatable.")
     args = parser.parse_args()
-
-    model, tokenizer = load_model_and_tokenizer()
+    steers = []
+    for spec in args.steer:
+        name, _, strength = spec.partition("=")
+        try:
+            steers.append((name.strip(), float(strength)))
+        except ValueError:
+            parser.error(f"--steer expects NAME=STRENGTH, got {spec!r}")
+        if not name.strip():
+            parser.error(f"--steer expects NAME=STRENGTH, got {spec!r}")
+
+    model, tokenizer = load_model_and_tokenizer(args.model)
     # Offer every vector we can: unverified index entries too (marked as such in the prompt)
-    agent = SteeringAgent(model, tokenizer, model_name=config.MODEL_NAME, include_community=True)
+    agent = SteeringAgent(model, tokenizer, model_name=args.model, include_community=True)
+    for name, strength in steers:
+        ok, msg = agent.apply(name, strength)
+        if not ok:
+            raise SystemExit(f"--steer {name}: {msg}")
+        print(f"[steer] {msg}")
     logger = SessionLogger()
     kill = KillSwitch()
 

community/check_vectors.py

@@ -11,13 +11,13 @@
 Run with --all to re-check every entry's file.
 """
 import argparse
-import json
 import sys
 from pathlib import Path
 
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 
 from modulo.community import check_vector_file, fetch_vector, load_index, validate_index  # noqa: E402
+from modulo.community import model_dims as community_model_dims  # noqa: E402
 
 
 def changed_entries(index: dict, base: dict) -> tuple[list[dict], list[str]]:
@@ -40,15 +40,9 @@ def maintainer_warnings(changed: list[dict], base: dict) -> list[str]:
 
 
 def model_dims(entry: dict) -> tuple[int | None, int | None, str | None]:
-    """(hidden_size, num_layers, note) from the model's config.json on the Hub."""
-    from huggingface_hub import hf_hub_download
-    try:
-        path = hf_hub_download(entry["model"], "config.json", revision=entry.get("model_revision"))
-    except Exception as e:  # gated or missing models: skip the shape checks, don't fail
-        return None, None, f"{entry['id']}: couldn't read {entry['model']} config ({type(e).__name__}); shape not checked"
-    cfg = json.loads(Path(path).read_text())
-    cfg = cfg.get("text_config", cfg) if "hidden_size" not in cfg else cfg
-    return cfg.get("hidden_size"), cfg.get("num_hidden_layers"), None
+    """(hidden_size, num_layers, note) from the entry's model's config.json on the Hub."""
+    hidden, layers, note = community_model_dims(entry["model"], entry.get("model_revision"))
+    return hidden, layers, note and f"{entry['id']}: {note}"
 
 
 def check(index_path: Path, base_path: Path | None, check_all: bool) -> tuple[list[str], list[str], list[str] | None]: