fix(governance): require crit markers with their claims, and run the signing tests CI skips

[KonstantinMirin]

Closes #1524.

This pull request carries the AdCP 3.2 fixes from #1537 to main, where RELEASING.md says fixes land first. next picks them up through the regular sync.

Changes

The commits make the following changes:

1. Hidden signing tests (test(webhooks), test(signing), five commits). With fastapi and svix installed, 26 tests in six signing files fail on main (Six signing test files never run in CI, and 26 of their tests fail on main #1524). The commits fix the tests, not the SDK:
   • The webhook sender tests pass operation_id to send_mcp, which feat(webhooks): add durable task webhook outbox #1066 made required (24 tests).
   • The proxy test's fake client has the stream method that the sender calls since feat(webhooks): add durable task webhook outbox #1066.
   • The dedupe retry repeats the first body: the idempotency key binds the payload, so a second body with its own timestamp is a 409.
   • The HTTP dedupe test signs each delivery. The receiver refuses a replayed nonce before it deduplicates, as WebhookSender.resend expects.
   • The digest-off FastAPI case signs with the 3.1 profile. The 3.2 signer refuses a non-empty body without content-digest (feat(protocol)!: stabilize multi-version SDK 8 beta #1038), as AdCP 3.2 request signing requires.
2. CI install (ci). The dev extra adds fastapi and svix. Every CI job installs .[dev] or .[dev,pg], so the five modules that skip on pytest.importorskip("fastapi"), and the two svix interop tests in test_standard_webhooks.py, run in CI.
3. Critical markers (fix(governance)). verify_governance_authorization compared each claim only with a crit entry whose marker is true. So it accepted a marker for a claim the payload doesn't carry: a crit entry with a false marker, a marker with no crit entry, or a false marker alone. The marker, its crit entry, and its claim must now appear together, with the marker true.
4. Non-string alg (fix(governance)). A header alg that is a JSON list or object raised TypeError, because the verifier tested it for membership in a set. It is now governance_token_invalid, like the other malformed headers.
5. Strict base64url (fix(signing)). JWS and JWK members are base64url: the RFC 4648 section 5 alphabet with the padding omitted (RFC 7515 section 2). RFC 4648 section 3.3 says implementations MUST reject encoded data that contains characters outside the base alphabet. b64url_decode used the standard library's lenient decoder: it accepted padding and the standard alphabet's + and /, dropped other characters, and ignored trailing bits. Verifier step 8 decodes the signer's JWK through public_key_from_jwk, so the request and webhook verifiers accepted a JWK whose x had !! inserted. b64url_decode now refuses =, +, and /, decodes the rest with base64.b64decode(altchars=b"-_", validate=True), which refuses any other character, and refuses non-zero trailing bits (RFC 4648 section 3.5), so each value has the one spelling that b64url_encode produces. The fix uses the standard library; main doesn't depend on joserfc.

Each fix started from a failing test:

• test_critical_marker_crit_entry_and_claim_appear_together re-signs the accepted governance vector with each header, so only the crit check can refuse it. Before the fix, all three cases passed the crit check and failed later with governance_token_not_applicable.
• test_a_non_string_alg_is_an_invalid_token raised TypeError for both cases.
• test_a_re_signed_vector_is_accepted checks that the re-signing helper produces a token the verifier accepts.
• test_jwk_members_are_strict_base64url failed for all four spellings: characters outside the alphabet, the standard alphabet, padding, and non-zero trailing bits.

Spec basis

• Critical markers: security.mdx, governance verification checklist step 4, at AdCP 3.2.4 and 3.2.5: "For authorized_commitment, authorized_task, and authorized_payload_hash, require each boolean protected-header marker, its crit entry, and its payload claim to appear together." The unknown-extension rule adds: "Verifiers MUST also reject when the critical marker and its corresponding claim do not appear together." Ungraded: no case in governance-authorization.json sends an incoherent marker.
• Strict base64url: RFC 7515 section 2 and RFC 4648 sections 3.3 and 3.5. Ungraded by the AdCP vectors.

API change

adcp.signing.b64url_decode is exported, and it now refuses input that it used to decode. It ships as a fix with no breaking-change marker: accepting data outside the base64url alphabet was a conformance defect (RFC 4648 section 3.3), and 9.x is in beta (9.0.0b4). A key published with padded or standard-alphabet base64 no longer verifies.

Testing

CI runs every job for this pull request, because it changes pyproject.toml. The following ran locally on macOS, on Python 3.12.9 and 3.10.20:

{{TESTING}}

[aao-secretariat]

Ladon verdict: Approve

Approve.

Clean fix PR with three correctness/security improvements in the governance + signing verifiers, each driven from a failing test, plus a CI-coverage fix that unmasks six signing modules previously skipped by pytest.importorskip (fastapi/svix moved to the dev extra).

Checked:

• Strict b64url_decode (crypto.py) — correct RFC 7515/4648 implementation (refuses =/+//, validate=True, roundtrip rejects non-zero trailing bits), consistent with _COMPACT_SEGMENT_RE, covered by four parametrized spoilers. Narrowing the public b64url_decode is a documented, defensible semver call (conformance defect, 9.x beta, migration note in PR body).
• type(alg) is not str (governance.py) — closes an uncaught TypeError on attacker-controlled tokens; now a clean governance_token_invalid.
• Critical-marker chained-equality correctly enforces marker/crit-entry/claim coherence per security.mdx step 4.
• operation_id="op_test" additions are not a new break — send_mcp has required it since #1066; these modules simply never ran before.

high_risk is true only because crypto.py was modified; reviewer found no medium-or-higher concerns, so the modification is presumed safe (no row-5 trigger). gated_paths is false and no no-auto-approve team match. No Critical/High/Medium findings → falls through to row 9.

[bokelley]

I checked this PR at 2c4fdb1c against main at its merge base, on Python 3.12 with fastapi 0.148 and svix 2.8 installed.

• Before. On main, the six Six signing test files never run in CI, and 26 of their tests fail on main #1524 files (test_e2e_fastapi, test_webhook_e2e_fastapi, test_webhook_sender_alt_auth, test_webhook_sender_e2e, test_webhook_sender_from_pem, test_standard_webhooks) give 26 failed and 48 passed.
• After. With this PR applied, the same six files plus test_governance_authorization.py and conformance/signing/test_crypto.py give 151 passed and 0 failed.
• CI. CI now runs these files and doesn't skip them. The Test Python 3.11 log shows svix==2.8.0 installed, and test_e2e_fastapi, test_webhook_e2e_fastapi, test_webhook_sender_alt_auth, and the two svix interop tests run and pass. All checks are green.
• Governance. The crit check (name in claims) == (name in critical) == (name in header), plus header.get(name, True) is True, matches security.mdx checklist step 4. So does the type(alg) is not str guard.

One fix before merge. The description's Testing section still holds the template placeholder {{TESTING}}.

Sync to next. next takes main through the regular merge, so this doesn't need a separate forward-port PR. #1537 carries its own version of the crit and base64url changes, so the first main → next sync after both merge may conflict in governance.py and crypto.py. As you said on #1537, keep next's side there.

This covers the main half of the #1537 review (finding 1), so I haven't opened a second PR for it.

🤖 Generated with Claude Code

[sangilish]

Thanks for landing this. I reproduced the same three incoherent marker/claim shapes in AdCP's shared-vector oracle at 80fe5b2c: all signatures verify, but the oracle reports governance_token_not_applicable rather than rejecting the malformed header as governance_token_invalid. The existing 27 vectors still match.

I'll prepare a separate, small AdCP follow-up adding those cases and tightening the test oracle under the existing step-4 rule. This won't touch the Python changes or the 3.3 WBA key-directory migration. Please let me know if that shared-vector slice is already in flight.