feat(stack-encrypt)!: derive owned record declarations with EncryptTarget and DecryptSource

.cargo/mutants.toml

@@ -81,6 +81,11 @@ exclude_re = [
   'stack-encrypt/src/sem/mod\.rs:\d+:\d+: replace <impl MatchConfig for DefaultMatch>::options -> MatchOptions with Default::default\(\)$',
   # Both unit-context conversions explicitly return Self::default().
   'stack-encrypt/src/target/context\.rs:\d+:\d+: replace <impl From<\(\)> for (DeclaredContext|ExpectedContext<T>)>::from -> Self with Default::default\(\)$',
+  # Omitting an expectation is expecting nothing: `ExpectedContext::omitted`
+  # returns Self::default(). That an omitted expectation opens under the
+  # stored context without checking it is pinned by
+  # `a_stored_context_opens_as_stored_and_checks_an_expectation`.
+  'stack-encrypt/src/target/input\.rs:\d+:\d+: replace <impl AllowsNoContext for ExpectedContext<T>>::omitted -> Self with Default::default\(\)$',
   # A caller-free declaration adds nothing to the declared context: the
   # conversion returns Self::default().
   'stack-encrypt/src/target/input\.rs:\d+:\d+: replace <impl From<NoCallerContext> for super::DeclaredContext>::from -> Self with Default::default\(\)$',

conformance/stack-encrypt/coverage.md

@@ -7,8 +7,8 @@ Paths are relative to the repository root. New profile cases run from `v1/`.
 | Requirement | Existing evidence | Remaining work |
 |---|---|---|
 | Scalar value widths and lossless transport | Initial scalar cases and [value-kind requirements](fixture-contract.md#value-kind-requirements), executed by the existing Rust and generated Go runners | Complete per-kind boundary matrices, bytes and broader floats; Python/TS mappings |
-| Owned opaque ciphertext and storage envelope | `packages/stack-encrypt/tests/scalar.rs`, `frozen_bytes.rs`; initial cross-opening cases | Blob profile; encoded targets; supported version matrix |
-| Record declaration and lowering | [Declaration grammar](declaration-grammar.md); `v1/record-declarations-v1.cases.json` shared by native Rust and Go; native normalization observations; existing record lowering suites | EQL-resolver profile; SDK-to-declaration mappings; runtime source/read shapes |
+| Owned opaque ciphertext and storage envelope | `packages/stack-encrypt/tests/scalar.rs`, `frozen_bytes.rs`; initial cross-opening cases; `owned_records.rs` (owned record derives: passthrough ownership transfer, sealed-field wiping, nested and renamed fields, stored context, one batch per column) | Blob profile; encoded targets; supported version matrix |
+| Record declaration and lowering | [Declaration grammar](declaration-grammar.md); `v1/record-declarations-v1.cases.json` shared by native Rust and Go; native normalization observations; existing record lowering suites; `packages/stack-encrypt/tests/owned_equivalence.rs` seals one mixed, searchable record through a hand-written owned target, the owned derive and the data plan against one recording provider and compares descriptors, term bytes and cross-opening | EQL-resolver profile; SDK-to-declaration mappings; runtime source/read shapes |
 | Context binding and composition | `packages/stack-encrypt/tests/target.rs`, `descriptor.rs`, `record_lowering.rs`; `languages/golang/encrypt/context_field_test.go`; initial wrong-context cases | Shared composition vectors; expected-context and required-input matrix |
 | Query terms and diagnostics | [Query-operator requirements](fixture-contract.md#query-operator-requirements), existing Rust/Go runners and initial equality/query-error cases | Common rejection corpus for every operator, capability and input shape; wider kind/index combinations |
 | Missing, SQL NULL and sealed null | `packages/stack-encrypt/src/dynamic/record.rs` null tests; `languages/golang/encrypt/null_test.go` | Explicit language mappings and common missing/null fixtures |

conformance/stack-encrypt/lifecycle-contract.md

@@ -12,8 +12,8 @@ process. Python and TypeScript are explicitly not enrolled.
 |---|---|---|
 | `lifecycle-ownership-v1` | Actual record operations and owned `Vec<S>` to `Vec<T>` conversions release fulfilment captures exactly once on success, provider failure and unpolled drop; zipped fulfilments do not share key material. | Client keys are consumed/redacted; explicit close and unreachable-client cleanup release guest memory; calls on a closed client fail. GC cleanup is eventual, so callers should close explicitly. |
 | `lifecycle-cancellation-v1` | Dropping a future paused in generation, for a record operation or an owned column, drops the provider future and operation captures; subsequent work on the cipher succeeds. Guest ABI abort spends the handle and drops the operation. | Cancellation before a call sends no request; cancellation between steps or during token fetch leaves unrelated operations usable. A guest step that overruns cancellation closes the client. Discarded late output is wiped. |
-| `lifecycle-partial-output-v1` | A batch with a valid first row and a later authenticated type mismatch releases its decoded prefix without marked heap residue, through the dynamic record path and the owned `Vec<Ciphertext>` to `Vec<String>` conversion alike; owned-column drop witnesses show that prefix is wiped, not leaked, after a late failure or a panicking element conversion. Existing context cases reject late authentication failures without returning partial output. | Generated and raw batch APIs return no partial rows on failure; interrupted HTTP bodies fail; cancelled output is wiped. SDK conversion diagnostics contain no decrypted values. This does not assert erasure of partially populated caller-owned Go objects. |
-| `lifecycle-buffer-cleanup-v1` | The existing allocator observer checks selected write/read/JSON parse paths, including an owned `Vec<String>` column sealed through its guard, and has a negative control. Drop witnesses show an owned column's source is wiped before its pending is returned, including on a refused or panicking context. Guest ABI tests check buffer-registry cleanup, error-buffer release and protected text drop. | Request bodies survive until their owner closes them, then wipe; refused requests wipe; heap growth wipes abandoned backing arrays; guest runtime close releases memory. Post-encryption memory scans are supplementary observations. |
+| `lifecycle-partial-output-v1` | A batch with a valid first row and a later authenticated type mismatch releases its decoded prefix without marked heap residue, through the dynamic record path and the owned `Vec<Ciphertext>` to `Vec<String>` conversion alike; owned-column drop witnesses show that prefix is wiped, not leaked, after a late failure or a panicking element conversion, through a structural record's column field too; an owned record's drop witness shows a field opened before a later field is refused is wiped. Existing context cases reject late authentication failures without returning partial output. | Generated and raw batch APIs return no partial rows on failure; interrupted HTTP bodies fail; cancelled output is wiped. SDK conversion diagnostics contain no decrypted values. This does not assert erasure of partially populated caller-owned Go objects. |
+| `lifecycle-buffer-cleanup-v1` | The existing allocator observer checks selected write/read/JSON parse paths, including an owned `Vec<String>` column sealed through its guard, and has a negative control. Drop witnesses show an owned column's source is wiped before its pending is returned, including on a refused or panicking context and for the elements after one whose declaration panics; and that an owned record wipes the fields it seals, on a refused context too, and moves its passthrough fields without copying them. Guest ABI tests check buffer-registry cleanup, error-buffer release and protected text drop. | Request bodies survive until their owner closes them, then wipe; refused requests wipe; heap growth wipes abandoned backing arrays; guest runtime close releases memory. Post-encryption memory scans are supplementary observations. |
 | `lifecycle-diagnostics-v1` | Guest errors discard untrusted foreign cause messages; failed/successful exports do not hand over stale diagnostics. | Structured errors remain available after failures; SDK errors and redacted display/logging expose types or fields rather than plaintext values. |
 | `lifecycle-validation-before-io-v1` | Existing runtime/read/context cases count provider calls. Wrong shape, independently detectable context mismatch and oversized descriptors fail before key I/O; lazy assembly stops after local failure. | Host refusals happen before entering the guest, and shared runtime/read cases exercise the uninitialised guest's preflight boundary. Go does not yet expose native provider counters. |
 | `lifecycle-logical-batching-v1` | Instrumented `DataKeySource` calls establish one logical generation batch, retrieval grouping by keyset, descriptor ordering and no work for empty batches, for record operations and for an owned `Vec<S>` column of any length including zero. | Go uses the same engine, but this profile does not claim independently instrumented Go key-request counts. |
@@ -61,7 +61,8 @@ These entries in the index are limitations, not successful requirements:
   paths use borrowed/static path text because owned path copies have separate
   cleanup limitations. It sees only blocks that are freed: plaintext leaked and
   never freed is not observed. For the owned `Vec<S>` conversion, drop
-  witnesses in `packages/stack-encrypt/tests/collections.rs` observe the wipe
+  witnesses in `packages/stack-encrypt/tests/collections.rs` (and, for owned
+  records, `packages/stack-encrypt/tests/owned_records.rs`) observe the wipe
   itself, so a leaked source or recovered prefix fails there. The lifecycle
   runner preserves the observer's output.
 - **`caller-owned-host-values` — outside the guarantee.** Caller-owned strings,

conformance/stack-encrypt/v1/lifecycle-v1.enrollment.json

@@ -35,6 +35,14 @@
       "features": ["--all-features"],
       "binary": "collections"
     },
+    "owned-records": {
+      "language": "rust",
+      "lane": "fast",
+      "workspace": ".",
+      "package": "stack-encrypt",
+      "features": ["--all-features"],
+      "binary": "owned_records"
+    },
     "runtime": {
       "language": "rust",
       "lane": "fast",
@@ -142,7 +150,11 @@
         ],
         "collections": [
           "a_late_element_failure_wipes_the_plaintext_recovered_before_it",
-          "a_panicking_element_conversion_wipes_the_plaintext_recovered_before_it"
+          "a_panicking_element_conversion_wipes_the_plaintext_recovered_before_it",
+          "a_late_element_failure_in_a_column_field_wipes_the_recovered_prefix"
+        ],
+        "owned-records": [
+          "a_field_opened_before_a_later_field_is_refused_is_wiped"
         ],
         "contexts": ["shared_context_boundaries"],
         "go": [
@@ -166,7 +178,14 @@
         "collections": [
           "owned_elements_are_wiped_before_the_pending_is_returned",
           "owned_elements_are_wiped_when_the_context_is_refused",
-          "owned_elements_are_guarded_during_context_conversion"
+          "owned_elements_are_guarded_during_context_conversion",
+          "a_panicking_element_declaration_wipes_the_elements_after_it"
+        ],
+        "owned-records": [
+          "the_owned_record_wipes_sealed_fields_and_moves_passthrough_fields",
+          "the_owned_record_wipes_sealed_fields_when_the_context_is_refused",
+          "a_panicking_field_declaration_wipes_the_sealed_fields_after_it",
+          "a_declaration_that_panics_while_constructed_wipes_the_plaintext"
         ],
         "guest-abi": [
           "buffers::tests::wipe_all_leaves_no_live_buffer",

packages/stack-encrypt-derive/Cargo.toml

@@ -24,7 +24,7 @@ syn = { version = "3", features = ["full", "extra-traits"] }
 # The dev-dependencies below are used only by the crate-level doctests.
 # `cargo-udeps` cannot see into rustdoc, so it reports them as unused.
 [package.metadata.cargo-udeps.ignore]
-development = ["stack-encrypt", "tokio"]
+development = ["stack-encrypt", "tokio", "zeroize"]
 
 [dev-dependencies]
 # The plain-segment rule is copied from stack-encrypt (a proc-macro crate cannot
@@ -37,3 +37,5 @@ serde_json = { workspace = true }
 # these doctests are what exercises that forwarding.
 stack-encrypt = { path = "../stack-encrypt", features = ["test-support"] }
 tokio = { workspace = true, features = ["rt", "macros"] }
+# The owned-record doctest derives `Zeroize` on its plaintext, as a user does.
+zeroize = { workspace = true }

packages/stack-encrypt-derive/docs/attributes.md

@@ -1,6 +1,11 @@
 # Attributes
 
-All attributes live under `#[stash(...)]`.
+All attributes live under `#[stash(...)]`. The four derives read the same
+attributes: `EncryptFrom` / `DecryptInto` emit the structural declarations,
+run over a borrowed plaintext; `EncryptTarget` / `DecryptSource` emit the
+owned declarations (and the structural ones beside them as a bridge), run
+over a plaintext handed over by value, and alone accept `passthrough`. Derive
+one pair or the other on a record, not both.
 
 ## On the struct
 
@@ -32,7 +37,8 @@ literal.
 | `identity = "..."` | With `struct` only: key this field under the segment `identity` instead of the plaintext field's name, so it is derived under `("<context>", "<identity>")`. One plain segment. The plan builder's `.identity(segment)`. |
 | `from = field` / `from = 0` | With `struct` only: derive this field from `plaintext.field` (or `plaintext.0` for a tuple struct) when its name differs from its plaintext field's. A plaintext field has one output: two fields `from` one plaintext field are refused (a ciphertext with its terms is one `Encrypted<Terms>` field). |
 | `default` / `default = expr` | Not derived: filled with `Default::default()` or `expr`. Never encrypted, never authenticated. |
-| `decrypt` | Decryption opens this field (`DecryptInto` only). Needed only when the field types cannot decide it — see below. |
+| `decrypt` | Decryption opens this field (`DecryptInto` / `DecryptSource` only). Needed only when the field types cannot decide it — see below. |
+| `passthrough` | With `struct` on the owned derives only: carry the plaintext field of this name (or `from = field`) into the record as it is, **not encrypted and not authenticated**, moved rather than cloned, and move it back on decryption. Keyed under nothing; excludes `identity`, `default` and `decrypt`. |
 
 A field takes no `context = ".."` of its own, on any derive: it is a compile
 error. A plan has exactly one context source, and every output sits under it.
@@ -64,6 +70,103 @@ from a `NonEmpty<T>` or a supported integer. Both encodings and the descriptor's
 structured identity are preserved when borrowing context data is converted into
 an owned declaration. No context is inferred from a Rust type's name.
 
+## Owned records
+
+`#[derive(EncryptTarget, DecryptSource)]` takes the attributes above and
+declares the record the owned way: `value.encrypt_into(&keyset)` consumes
+the plaintext, and `Plaintext::decrypt_from(record, &cipher)` returns it.
+The plaintext type is `Zeroize` (the trait bound says where the wiping
+obligation lies), and the conversion wipes every plaintext field it seals.
+What the caller may or must pass is the record's `ContextInput`:
+
+| Record | Encrypt input | Decrypt input | A call |
+|---|---|---|---|
+| `struct = User, context = "users"` | `ContextExtension<DeclaredContext>` | the same | `user.encrypt_into(&keyset)`, or `.encrypt_into_with_context(&keyset, 7u64)` to extend every field's pair by the row id |
+| `plaintext = T` | `RequiredContext<CallerContext>` (`RequiredContext<AeadContext>` with `context_type = AeadContext`) | the same | `value.encrypt_into_with_context(&keyset, nonempty!("users/email"))` |
+| with a `context_field` of type `C` | `RequiredContext<NonEmpty<C>>` | `ExpectedContext<C>` | `T::decrypt_from(record, &cipher)` opens under the stored context; `decrypt_from_with_context(record, &cipher, nonempty!(..))` also checks it |
+
+A `struct` record's field is handed its pair, extended by the caller's part
+if one was given, so the field's target must accept a caller context: a
+`Ciphertext` (`OptionalContext`), a term or an `Encrypted<Terms>`
+(`RequiredContext<CallerContext>`), a `StackCipherText`
+(`RequiredContext<AeadContext>`), a `plaintext` record, or another `struct`
+record (whose own pairs the outer pair then extends). A target that declares
+its whole context itself (`NoCallerContext`) or requires a context of another
+type (`RequiredContext<u64>`) is refused at that field. A `#[stash(passthrough)]`
+field is moved into the record by the owned conversion; its type is `Clone`
+too, since the record's borrowed form, which a plan or a structural record
+runs, copies it.
+
+A `struct` record accounts for every field of its plaintext, sealing it or
+carrying it through: the owned conversion destructures the struct, and a
+field it does neither would be dropped unwiped, so it is refused at the
+`struct = ..` path ("pattern does not mention field"). For the same reason
+the plaintext struct cannot implement `Drop` (so not `ZeroizeOnDrop`
+either): a type with a destructor cannot be destructured. A field's
+declaration that panics wipes the sealed fields not yet handed to their
+own declarations, and a record's declaration that panics while it is
+constructed wipes the whole plaintext. On decryption a field opened before
+a later field is refused is wiped, not dropped as it is.
+
+A `plaintext` record's fields are all handed the one context the record
+requires, so a term beside an `AeadContext`-only record is refused at the
+term. Omitting a required context is refused at the call
+(`AllowsNoContext`), whatever `Default` the context type has.
+
+Both owned derives also emit the structural declarations: `EncryptTarget`
+emits `EncryptFrom` (the record's legacy context — `DeclaredContext`,
+`CallerContext` or `NonEmpty<C>` — converted into its owned input) and
+`Decryptable`; `DecryptSource` emits `DecryptInto` and `DecryptField`. An
+owned record is therefore a field of a `#[derive(EncryptFrom)]` struct
+record and a typed field of a plan (`encrypt_into::<Record, _>`) without
+a second derive. A hand-written owned target that is to be a record field
+implements `Decryptable` and `DecryptField` beside `DecryptSource`.
+
+```rust
+use stack_encrypt::sem::EqualityTerm;
+use stack_encrypt::target::{DecryptSource, EncryptTarget};
+use stack_encrypt::{Ciphertext, DecryptFrom, EncryptInto, StackCipher};
+
+#[derive(Debug, PartialEq, zeroize::Zeroize)]
+struct User { id: u64, name: String, email: String }
+
+/// A searchable scalar: the ciphertext and an equality term, under the one
+/// context the field (or a caller) supplies.
+#[derive(EncryptTarget, DecryptSource)]
+#[stash(plaintext = String)]
+struct SearchableEmail { ciphertext: Ciphertext, equality: EqualityTerm }
+
+#[derive(EncryptTarget, DecryptSource)]
+#[stash(struct = User, context = "users")]
+struct SearchableUser {
+    #[stash(passthrough)]
+    id: u64,
+    name: Ciphertext,        // ("users", "name")
+    email: SearchableEmail,  // ("users", "email")
+}
+
+# tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap().block_on(async {
+let cipher = StackCipher::builder()
+    .kms(stack_encrypt::kms::FakeDataKeySource::new()).init().await?;
+let keyset = cipher.default_keyset();
+let users = vec![
+    User { id: 42, name: "Alice".into(), email: "alice@example.com".into() },
+    User { id: 43, name: "Bob".into(), email: "bob@example.com".into() },
+];
+// One batched key request for both rows' four sealed fields.
+let rows: Vec<SearchableUser> = users.encrypt_into(&keyset).await?;
+// A query term is the same derivation under the same context.
+let wanted: EqualityTerm = String::from("alice@example.com")
+    .encrypt_into_with_context(&keyset, stack_encrypt::nonempty!("users").with("email"))
+    .await?;
+let matched: Vec<SearchableUser> = rows.into_iter().filter(|r| r.email.equality == wanted).collect();
+let opened = Vec::<User>::decrypt_from(matched, &cipher).await?;
+assert_eq!(opened.len(), 1);
+assert_eq!(opened[0].id, 42);
+# Ok::<(), stack_encrypt::Error>(())
+# }).unwrap();
+```
+
 ## Which context a record takes
 
 `CallerContext` holds both of Vitamin C's encodings of the context — the AEAD