Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ Work after the 1.0.0 candidate. It is not part of the staged 1.0.0 release.

### Added

- Sweeps: `features = { dictionary, ids }` in `[sweep]` sweeps the features
of a feature dictionary, one per point: each point gives the swept
interventions `features = { dictionary, ids = [F] }`. `ids` is a list or
a range. The swept interventions stay at the dictionary's site and layer,
so a feature sweep crosses only with `alphas`. `sweep.json` records
`features` and each point's `feature`, and `sweep.csv` and
`sweep-effect.csv` gain a `feature` column for feature sweeps only.
Sweeps without `features` write the same files and `sweep_hash` as
before. `examples/experiments/capital-feature-sweep.toml` sweeps eight
country features across eight capital prompts.
- Examples: `capital-head-feature-mediation.toml` patches layer 9 head 23
with the clean prompt's country feature held at its corrupted value. The
head writes the feature (77% of its clean value on average), and 58% of
Expand Down
111 changes: 111 additions & 0 deletions docs/audits/2026-10-22-feature-sweeps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Sweeps audit (feature sweeps): 2026-10-22

- **Revision/branch:** `feat/feature-sweep` on `main` at `a5f42ae`.
- **Audit date:** 2026-10-22
- **Scope and trigger:** layer sweeps (`src/v05/sweep.rs`,
`src/cli_experiment_sweep.rs`) and feature interventions
(`src/v05/dictionary.rs`, `src/cli_experiment_dictionary.rs`). Trigger: a
public schema change. The `[sweep]` table gains `features`, and
`ember.sweep.v1` gains optional `features` and per-point `feature` fields.
- **Primary:** research maintainer (voidwest).
- **Backup:** not assigned (open since the 2026-10-11 record).
- **Second reviewer:** an automated review agent read the diff without a
build. No human second reviewer.
- **Last audit / next audit:** last: `2026-10-21-head-sweeps.md` (sweeps)
and `2026-10-22-feature-dictionaries.md` (dictionaries). Next: before the
next release tag, or at the next `ember.sweep.v1` change.

## 1. Map and boundaries

- User-facing commands/APIs: `ember experiment validate|run|verify|compare|inspect`
with `features = { dictionary, ids }` in `[sweep]`.
- Source entry points: `SweepDefinition::parse` and `parse_features`;
`point_count` and `points`; the feature check in `run_sweep`;
`verify_and_load_sweep`.
- Adjacent subsystems and handoffs: feature interventions (ADR-0003); the
dictionary cache in `load_dictionaries`; the shared pass and prefix reuse.
- Persistent artifacts or schemas: `sweep.json` (`features`, per-point
`feature`), `sweep.csv` and `sweep-effect.csv` (`feature` column only for
feature sweeps). Point bundles are ordinary `ember.bundle.v1` bundles with
`features = { dictionary, ids = [F] }` on the swept interventions.
- Environment switches and feature gates: none new.
- External references or trusted implementations: none.
- Supported targets and deliberate non-goals: no `ids = "all"`. A feature
sweep does not also move layers, heads or positions, because a dictionary
is trained at one site and layer. It can cross with `alphas`.

## 2. Contracts and invariants

- Ownership/lifetime and resource limits: a sweep sets at most 65,536
features (`MAX_SWEPT_FEATURES`, the width of the largest dictionary that
the weights cap admits). A range is counted before it is collected. The
run checks every id against the dictionary's `d_sae` before the first
point. The points reuse the cached weights.
- Shape, encoding, safety, or error behavior: the swept interventions must
be at the dictionary's site and layer, must not declare `features`, and
must use `zero`, `scale`, `replace`, `add-delta` or `interpolate`. An
unknown dictionary, an empty list or a malformed range fails at parse
time, with the `sweep.features` path.
- Numerical, determinism, trace, or hook boundaries: no change to hooks or
to the feature-space arithmetic. Each point bundle is the bundle that its
derived spec gives alone.
- Compatibility/provenance requirements: sweeps without `features` write
the same `sweep.json`, CSV files and `sweep_hash`; the frozen fixtures
(`tests/sweep_compatibility.rs`) still verify. `verify` compares the
recorded `features` with the definition and each point's `feature` with
its coordinates.
- Reference/oracle path versus optimized path: not applicable.

## 3. Checks and evidence

Host: Apple M1 Pro, macOS, Rust 1.98.1 (GUI) and 1.92.0 (headless MSRV),
`CARGO_BUILD_JOBS=2`.

| Check | Command / fixture | Result | Evidence or artifact |
| --- | --- | --- | --- |
| Static/build | `scripts/ci_local.sh quick` (part of `full`) | pass | fmt, clippy (all configurations), MSRV, docs |
| Focused unit/integration | `feature_sweeps_set_one_feature_per_point`, `feature_sweeps_refuse_what_a_dictionary_cannot_do`, `feature_sweeps_run_one_feature_per_point_and_verify` | pass | grid order and point ids; every refusal; a forged feature or id list fails `verify`; an id past `d_sae` fails before any bundle |
| Full suites | `scripts/ci_local.sh full` | pass: 1,760 Rust tests and 146 Python tests (1 skipped); `cargo audit` not run (not installed locally; CI runs it) | all suites |
| Compatibility | `tests/sweep_compatibility.rs` | pass | frozen sweeps verify with their recorded hashes |
| Smoke | `capital-feature-sweep.toml` on Llama-3.2-1B-Instruct Q8_0 | pass | 9 bundles verify; 2.5 s of compute; 38 MB; a second run gives the same `sweep-effect.csv` |
| Golden logits / external reference | not run | n/a | no inference change |
| Performance (controlled A/B) | not run | n/a | no hot-path change |
| Backup reproduction | not run | n/a | no backup assigned |

Model: `Llama-3.2-1B-Instruct-Q8_0.gguf`, SHA-256
`432f310a77f4650a88d0fd59ecdd7cebed8d684bafea53cbff0473542964f0c3`.
Dictionary: the qresearch layer-9 SAE converted to SAELens, weights SHA-256
`8ade5dc5488d13676a26cad16b4d9bf88c947b9f6df44441996fe87ffff6977f`. The
dictionary is local only and is not in the repository.

## 4. Findings

| Severity | Finding / evidence | Owner or issue | Due / disposition |
| --- | --- | --- | --- |
| required | A range such as `{ start = 0, end = 9223372036854775807, step = 1 }` was collected before any limit, so `validate` could exhaust memory. | research maintainer | fixed before merge: the range is counted first, and a sweep sets at most 65,536 features |
| required | A swept `steer`, `ablate-projection` or `restore-original` intervention passed the sweep parse and failed only when the points were derived, after the model loaded. | research maintainer | fixed before merge: refused at parse time with the `sweep.features` path |
| required | An id past the dictionary's width failed only when its point ran, after the earlier points wrote bundles. | research maintainer | fixed before merge: `run_sweep` checks every id against `d_sae` before the first point |
| observation | `validate` did not show the feature axis. | research maintainer | fixed before merge: the JSON output has `features`, and the text output names the count and the dictionary |
| observation | The first docs said that each feature moves only its own country's prompt. Six off-diagonal entries are not 0. | research maintainer | fixed before merge: the example header explains each of the six from a full feature capture of the rows |
| follow-up | No named backup for the sweeps surface. | project owner | assign at the next audit |

## 5. Newcomer handoff

A feature sweep keeps the swept interventions at the dictionary's site and
layer and gives each point one feature: `features = { dictionary, ids =
[F] }`. Start with
`cargo test --no-default-features -- feature_sweeps`. The real-model
example is `examples/experiments/capital-feature-sweep.toml` (run
`capital-clean.toml` first, and convert the SAE as its header shows).

## 6. Close-out

- [x] Index row updated with this record.
- [ ] Primary and backup reviewed the evidence. (No backup is assigned.)
- [x] Governing docs/contracts updated, or an issue links the needed update.
- [x] Follow-ups are visible and have owners.
- [x] No model weights, private data, or generated tensor payloads were added.

**Reviewer notes:** the automated review found the three required problems
and the two observations above, all fixed before merge. It confirmed that
sweeps without `features` keep their files and hash.
2 changes: 1 addition & 1 deletion docs/audits/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ backup, fixing that is an audit finding.
| K-quant matmul | runtime maintainer / SIMD reviewer | [`crates/ember-core/src/k_quant_matmul.rs`](../../crates/ember-core/src/k_quant_matmul.rs), [`crates/ember-core/src/k_matmul.rs`](../../crates/ember-core/src/k_matmul.rs) | [`k_quant_matmul` tests](../../crates/ember-core/src/k_quant_matmul.rs), pinned known-answer gate | Add the next dated record; see [`CONTRIBUTING.md`](../../CONTRIBUTING.md) |
| Loader, quantization, and model families | runtime maintainer / architecture reviewer | [`crates/ember-core/src/loader.rs`](../../crates/ember-core/src/loader.rs), [`crates/ember-core/src/quant_k.rs`](../../crates/ember-core/src/quant_k.rs), [`docs/models.md`](../models.md) | loader/unit tests plus the relevant golden-logit or parity check | Add the next dated record |
| Execution plan, hooks, and experiments | runtime maintainer / research reviewer | [`crates/ember-core/src/plan.rs`](../../crates/ember-core/src/plan.rs), [`crates/ember-core/src/planned_decode.rs`](../../crates/ember-core/src/planned_decode.rs), [`crates/ember-core/src/experiments/`](../../crates/ember-core/src/experiments/), [`src/experiments/`](../../src/experiments/) | plan/hook tests, reference-vs-planned parity, capture/intervention contract | [2026-10-21 attention heads](2026-10-21-attention-heads.md) |
| Bundles, reproducibility, and agent traces | research maintainer / release reviewer | [`src/v05/`](../../src/v05/), [`src/agent/`](../../src/agent/), [`docs/reproducibility.md`](../reproducibility.md) | verify/reproduce a small fixture or contract test; inspect provenance | [2026-10-11 sweeps](2026-10-11-sweeps.md), [2026-10-21 head sweeps](2026-10-21-head-sweeps.md), [2026-10-22 feature dictionaries](2026-10-22-feature-dictionaries.md) |
| Bundles, reproducibility, and agent traces | research maintainer / release reviewer | [`src/v05/`](../../src/v05/), [`src/agent/`](../../src/agent/), [`docs/reproducibility.md`](../reproducibility.md) | verify/reproduce a small fixture or contract test; inspect provenance | [2026-10-11 sweeps](2026-10-11-sweeps.md), [2026-10-21 head sweeps](2026-10-21-head-sweeps.md), [2026-10-22 feature dictionaries](2026-10-22-feature-dictionaries.md), [2026-10-22 feature sweeps](2026-10-22-feature-sweeps.md) |
| GUI, multimodal, audio, and TTS | product/runtime maintainer / headless reviewer | [`src/gui.rs`](../../src/gui.rs), [`src/multimodal/`](../../src/multimodal/), [`src/tts/`](../../src/tts/) | default-feature and `--no-default-features` builds; applicable focused tests | Add the next dated record |
| Python research and probe pipeline | research maintainer / statistics reviewer | [`probes/`](../../probes/), [`tests/`](../../tests/), [`docs/research.md`](../research.md) | compile/smoke plus the relevant pytest and artifact checks | Add the next dated record |
| CI, release, and documentation | release maintainer / docs reviewer | [`.github/workflows/ci.yml`](../../.github/workflows/ci.yml), [`scripts/check_docs.py`](../../scripts/check_docs.py) | CI-equivalent gates and docs check | Add the next dated record |
Expand Down
8 changes: 4 additions & 4 deletions docs/experiment-schema-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,10 @@ settings are identical whether a default is omitted or explicitly supplied.
## Sweeps

An optional `[sweep]` table turns the spec into a template for a layer
sweep (`layers`, optional `positions`, optional `heads`, optional `alphas`,
optional `interventions`, optional `move_bundle_sources`, optional
`[sweep.effect]`; `layers` may be omitted when `heads` or `alphas` is
given); see `docs/experiments.md#layer-sweeps`. Unknown `[sweep]` fields fail, and a
sweep (`layers`, optional `positions`, optional `heads`, optional
`features`, optional `alphas`, optional `interventions`, optional
`move_bundle_sources`, optional `[sweep.effect]`; `layers` may be omitted
when `heads`, `features` or `alphas` is given); see `docs/experiments.md#layer-sweeps`. Unknown `[sweep]` fields fail, and a
spec with `[sweep]` is only accepted as a sweep.

## Attribution
Expand Down
22 changes: 22 additions & 0 deletions docs/experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,7 @@ an ordinary spec (see `examples/experiments/morphology-layer-sweep.toml`):
layers = "all" # or [2, 4, 8], or { start = 0, end = 16, step = 2 }
positions = [3, 7] # optional: absolute token positions
heads = "all" # optional: one attention head per point (attention-heads only)
# features = { dictionary = "l10", ids = [3756, 8633] } # optional: one feature per point; not with layers, heads or positions
alphas = [0.0, 2.0, 4.0] # optional: sets `alpha` of the swept interventions
interventions = ["replace"] # optional: which interventions move (default: all)
```
Expand Down Expand Up @@ -247,6 +248,27 @@ before it runs. A layers x heads grid makes one
bundle per point, so a sweep of 2 layers of a 32-head model writes 65
bundles.

`features = { dictionary, ids }` sweeps the features of a dictionary (see
[feature dictionaries](#feature-dictionaries)): each point gives the swept
interventions `features = { dictionary, ids = [F] }`, so each acts on one
feature. `ids` is a list or a `{ start, end, step }` range; there is no
`"all"`, because a dictionary can hold tens of thousands of features and a
sweep writes one bundle per point. A dictionary is trained at one site and
layer, so the swept interventions must be at that site with `layers =
[<its layer>]`, must not declare `features` themselves, and the sweep cannot
also move layers, heads or positions; it can cross with `alphas`. The swept
interventions must use `zero`, `scale`, `replace`, `add-delta` or
`interpolate`, as for any feature intervention. A sweep sets at most 65,536
features, and the run checks every id against the dictionary's width before
the first point runs. Point ids are `feature-03756`. `sweep.json` records
`features` (the dictionary and the ids) and each point's `feature`, and
`sweep.csv` and `sweep-effect.csv` gain a `feature` column for feature
sweeps only. See
`examples/experiments/capital-feature-sweep.toml`: eight country features
across eight capital prompts. Each feature has its largest effect on its own
country's prompt. Of the 56 other entries, 50 are exactly 0; the example
header explains the six that are not.

Each *point* is the spec with the swept interventions at `layers = [L]`
(and, with `positions`, `tokens = { kind = "absolute-token", index = P }`);
everything else is unchanged. The *baseline* is the spec without
Expand Down
6 changes: 6 additions & 0 deletions examples/experiments/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,12 @@ copy pinned to `unsloth/Llama-3.2-1B-Instruct` revision
the feature held, 3.58. The head writes the feature, and 58% of its
effect passes through it. Needs the same SAE and `capital-clean.toml`.

13. `capital-feature-sweep.toml` — a feature sweep (`[sweep] features`): each
of the eight country features, patched into all eight corrupted prompts.
Each feature has its largest effect on its own country's prompt. Of the
56 other entries, 50 are exactly 0; the header explains the six that are
not. Needs the same SAE and `capital-clean.toml`.

## Commands

```bash
Expand Down
Loading
Loading