Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,9 @@ vec = extract_vector(model, tokenizer,
### SteeringAgent methods

- `agent.step(messages, max_new_tokens=300, temperature=0.7, top_p=0.9, tools=None) -> StepResult`
- `agent.export_calibration(key=...) -> dict` — independent JSON-compatible snapshot of already measured scalar norms; no generation
- `agent.import_calibration(record, key=...)` — replace all norms from a matching record; `None` clears as a miss
- `agent.clear_calibration()` — forget imported and locally measured norms without disabling steering
- `agent.build_system_prompt() -> str` — system prompt describing the steer tool
- `agent.status() -> str` — active vector status
- `agent.apply(name, strength) -> (bool, str)` — manually apply a vector
Expand All @@ -232,6 +235,18 @@ vec = extract_vector(model, tokenizer,
- `agent.debrief() -> str | None` — end the session's steering; send the returned status as a `tool` message for one last turn
- `agent.menu_order() -> list[str]` — the order vectors are listed in this session (from `menu_seed`)

**Calibration reuse is opt-in startup reuse, not decode acceleration.** The caller's
nonempty key asserts complete checkpoint/tokenizer/weight/adapter/execution identity;
the library's schema, layer, dtype/device and runtime checks are sanity checks, not
automatic mutation detection. Before changing weights, adapters, tokenizer, dtype,
device or numerical settings, clear calibration and use a new key. Missing layers
calibrate normally. Invalid imports raise `ValueError` and leave all norms empty.
Export cannot relabel a bound key until clear; import itself clears/replaces.
Only canonical zero-based layers are reusable; negative-index memos cannot export.
Scalar records hold no live owners and are separate from `export_state()` steering
checkpoints. Normal tracebacks can retain owners until the caller releases errors.
See [Reuse calibration across identical model owners](README.md#reuse-calibration-across-identical-model-owners) for the complete record and operator contract.

### StepResult

- `prose` — model output with tool calls stripped
Expand Down
53 changes: 53 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,59 @@ messages += result.tool_messages() # tool results back to the model

If your harness has tools of its own, pass them with `agent.step(messages, tools=[...])`. Calls to them come back unrun in `result.other_tool_calls`. See [AGENTS.md](AGENTS.md#1-as-a-library-for-existing-agent-harnesses).

#### Reuse calibration across identical model owners

Calibration reuse is opt-in and saves startup work, not decoding work. By default,
each agent measures and memoizes its own residual norms. An operator can keep a
JSON-compatible scalar record after releasing an agent and its model:

```python
key = "my-pinned-model-and-tokenizer; pristine; no-adapters; bf16-4090-sdpa-profile-v1"
agent.residual_norm(layer) # ordinary calibration, or let the first steered step do it
record = agent.export_calibration(key=key) # no extra calibration; json.dumps works
# Release the old agent/model, then load the exact same model and tokenizer.
new_agent = SteeringAgent(new_model, new_tokenizer)
new_agent.import_calibration(record, key=key)
```

**Your key asserts identity.** Include exact checkpoint and tokenizer revisions
(including the chat template), weight mutations, adapter revisions/order/scales
and enabled/merged state, generation configuration, custom code/hooks, and
numerical/backend/device settings. Distinguish physical GPUs, not just `cuda:0`.
Use a stable key for identical pristine reloads and a new unique key for mutations.
Matching keys are not proof of numerical equivalence: validate your execution
profile before reuse. The library does not hash weights or detect mutations.

The library checks schema/recipe, key equality, model name/class, layer count,
hidden size, parameter/buffer dtype/device sets, input device, and Torch/Transformers
versions. Import/export require every model module in evaluation mode. These are
sanity checks, not complete identity checks. Records are trusted operator data,
not input from the model or community. A plausible but incorrect scalar cannot
be detected automatically.

- `export_calibration(key=...)` returns an independent record of measured layers
only, possibly empty. Exporting under a different key requires a clear.
- `import_calibration(record, key=...)` **replaces** the memo. Unknown fields,
mismatches, invalid layer IDs, booleans, and nonpositive/nonfinite values raise
`ValueError`. Failure leaves the entire memo empty, with no partial import.
Norms accept finite positive Python integers/floats with canonical layer strings.
Import clears first, so a different matching record/key needs no separate clear.
- Reusable layers are zero-based (`0` through the last layer). An existing memo
measured via Python negative indexing cannot be exported. For reuse, configure
vectors with equivalent zero-based indices, clear, and recalibrate those layers.
- Missing layers still calibrate normally. `import_calibration(None, key=...)`
clears the memo as an explicit miss; its key is ignored.
- Before changing model, tokenizer, adapters, dtype, device, or execution settings,
call `clear_calibration()` and use a new key. It clears imported **and** locally
measured norms. Recreate the agent when replacing its model. There is no
automatic invalidation, concurrency support, or mid-step import/export.
- To disable reuse, omit imports; clear an existing memo if necessary. Steering
remains enabled. Calibration records are separate from steering checkpoints.

Records contain no tensors, models, vectors, KV, or GPU state, and perform no file
I/O. Ordinary exception tracebacks remain intact and can retain owners/activations;
release caught errors and their tracebacks before expecting model/GPU collection.

### As a standalone demo

```bash
Expand Down
88 changes: 88 additions & 0 deletions modulo/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -547,6 +547,7 @@ def __init__(self, model, tokenizer, registry: Optional[VectorRegistry] = None,
self._turn_generated = False
self._unit_vectors: dict[str, torch.Tensor] = {}
self._residual_norms: dict[int, float] = {}
self._calibration_key: Optional[str] = None
# name -> (strength, turn) of the most recent generation each vector steered
self._last_steered: dict[str, tuple[float, int]] = {}
# name -> turn of the latest report on it (a vector is reportable again once re-steered)
Expand Down Expand Up @@ -925,6 +926,93 @@ def hook(module, inp, output):
self._residual_norms[layer] = h.norm(dim=-1).median().item()
return self._residual_norms[layer]

def clear_calibration(self) -> None:
"""Forget all measured/imported norms. Subsequent use calibrates normally."""
self._residual_norms.clear()
self._calibration_key = None

def _calibration_profile(self) -> dict:
# Deliberately not a weight fingerprint. The operator key covers everything
# not checked here, including tokenizer, adapters and numerical settings.
import transformers

if any(module.training for module in self.model.modules()):
raise ValueError("Calibration reuse requires all model modules in eval mode.")
return {
"recipe": [1, "median-generated-l2-full-replay", CALIBRATION_PROMPT, CALIBRATION_TOKENS],
"model": self.model_name,
"class": f"{type(self.model).__module__}.{type(self.model).__qualname__}",
"layers": len(decoder_layers(self.model)),
"hidden_size": getattr(self.model.config, "hidden_size", None),
"input_device": str(self.model.device),
"placements": [list(p) for p in sorted({(str(t.dtype), str(t.device))
for t in (*self.model.parameters(), *self.model.buffers())})],
"torch": str(torch.__version__),
"transformers": transformers.__version__,
}

def export_calibration(self, *, key: str) -> dict:
"""Return a detached JSON-compatible record of already measured norms.

key is the operator's identity for exact weights/tokenizer/adapters and
execution settings. It is an assertion, not automatic mutation detection.
Does not calibrate missing layers. See README.md for the reuse contract.
"""
if self._calibration_key is not None and key != self._calibration_key:
raise ValueError("Clear calibration before changing its operator key.")
record = {"schema": 1, "key": key, "profile": self._calibration_profile(),
"norms": {str(k): v for k, v in self._residual_norms.items()}}
self._validate_calibration(record, key=key, profile=record["profile"])
self._calibration_key = key
return record

@staticmethod
def _validate_calibration(record, *, key, profile) -> dict[int, float]:
if not isinstance(key, str) or not key.strip():
raise ValueError("Calibration key must be a nonempty operator identity string.")
if (not isinstance(record, dict) or set(record) != {"schema", "key", "profile", "norms"}
or type(record["schema"]) is not int or record["schema"] != 1):
raise ValueError("Invalid calibration record schema.")
if type(record["key"]) is not str or record["key"] != key:
raise ValueError("Calibration operator key mismatch.")
try:
matches = json.dumps(record["profile"], sort_keys=True, allow_nan=False) == json.dumps(profile, sort_keys=True)
except (TypeError, ValueError, RecursionError) as error:
raise ValueError("Invalid calibration profile.") from error
if not matches:
raise ValueError("Calibration profile mismatch.")
norms = record["norms"]
if not isinstance(norms, dict):
raise ValueError("Calibration norms must be a layer-to-scalar dictionary.")
checked = {}
valid_layers = {str(i) for i in range(profile["layers"])}
for layer, value in norms.items():
if (not isinstance(layer, str) or layer not in valid_layers
or type(value) not in (int, float)):
raise ValueError("Calibration norms require valid layer strings and positive finite numbers.")
try:
scalar = float(value)
except OverflowError as error:
raise ValueError("Calibration norm is outside the finite scalar range.") from error
if not math.isfinite(scalar) or scalar <= 0:
raise ValueError("Calibration norms must be positive and finite.")
checked[int(layer)] = scalar
return checked

def import_calibration(self, record: Optional[dict], *, key: str) -> None:
"""Replace all norms with a matching operator-owned record; None is a miss.

Malformed/mismatched records raise ValueError and leave all norms empty.
Missing layers calibrate normally. Call only between steps on unchanged
models; clear before any mutation and use a new key. Tracebacks are not
redacted and can retain owners until the caller releases them.
"""
self.clear_calibration()
if record is not None:
self._residual_norms.update(self._validate_calibration(
record, key=key, profile=self._calibration_profile()))
self._calibration_key = key

def _build_steering_context(self, prompt_len: int):
if not self.active or self.control == "sham":
return nullcontext()
Expand Down
Loading
Loading