Skip to content

fix(ci): make MC/DC, WIT round-trip and claim-check gates fail on bad input - #240

Merged
avrabe merged 2 commits into
mainfrom
fix/gate-potency-mcdc-wit-claims
Nov 13, 2026
Merged

avrabe merged 2 commits into
mainfrom
fix/gate-potency-mcdc-wit-claims

Conversation

@avrabe

@avrabe avrabe commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #239: three required checks reported success on input that should fail them.

Check Change Planted input, exit old → new
MC/DC (witness) mcdc-gate.sh rejects missing or non-integer overall.* before the floor tests report {}: 0 → 1; conditions_proved: "abc": 0 → 1
WIT round-trip (wasm-tools) drop | head -50 || true; scry.wit resolves standalone syntax error appended: wasm-tools exits 1, which the step now propagates
Doc claims (claim-check) exit 2 on no claims; fail a claim with no evidence claims: []: 0 → 2; claim without evidence: 0 → 1

Valid inputs stay green: a report with 160 proved / 4 full exits 0, the real claims.yaml gives 7/7 and exit 0, and the unmodified scry.wit parses with exit 0 under wasm-tools 1.245.1.

The gate tests ran the extracted gate section of mcdc-gate.sh locally (from PROVED= to PASSED), not the full witness pipeline. CI runs the full path.

Two further commits came out of the first CI run:

  • Refs: FEAT-102 on the gate commit, plus artifacts/roadmap-v3.4-gate-planted-input.yaml (FEAT-102, traces to REQ-005). Commit traceability had failed with orphans=1. Locally, rivet validate passes with no FEAT-102 warnings and the trailer guard passes.
  • A clippy fix at crates/scry-pentagon/src/lib.rs:570: clippy 1.100.0 (2026-11-09) added needless_range_loop there. main last went green on 2026-11-08 with an older stable, so main would also go red on its next run. All five CI clippy invocations exit 0 under 1.100.0, and scry-sai-pentagon passes 17/17 tests. Pin the toolchain through varve: covalent realm, rolling 2026.11.10 #238 (varve toolchain pin) addresses the drift itself.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KEjmjcVaTucsMZy5Fubas7

avrabe and others added 2 commits November 13, 2026 21:37
… input

- mcdc-gate.sh: reject a missing or non-integer overall.* field before
  the floor comparisons; `[ null -lt 155 ]` errored, `if` read it as
  false, and the gate printed PASSED.
- ci.yml: drop `| head -50 || true` from the scry.wit check so a parse
  or resolve error fails the step; scry.wit resolves standalone.
- claim-check.py: exit 2 when no claims are declared, and fail a claim
  that has no evidence predicates.

Planted inputs, old -> new exit: report `{}` 0 -> 1; conditions_proved
"abc" 0 -> 1; `claims: []` 0 -> 2; claim without evidence 0 -> 1.
Valid report (160 proved) and the real claims.yaml stay 0.

Fixes #239
Refs: FEAT-102

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEjmjcVaTucsMZy5Fubas7
clippy 1.100.0 (rustc 2026-11-09) adds needless_range_loop at
crates/scry-pentagon/src/lib.rs:570 and fails the Clippy job under
-D warnings. Behaviour is unchanged: lt has n*n = 9 entries and
take(9) keeps the bound explicit. All five CI clippy invocations exit
0 under 1.100.0; scry-sai-pentagon tests 17/17 pass.

Trace: skip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEjmjcVaTucsMZy5Fubas7
@avrabe
avrabe force-pushed the fix/gate-potency-mcdc-wit-claims branch from 9b36de0 to a0abe15 Compare November 13, 2026 20:37
@github-actions

Copy link
Copy Markdown

📐 rivet artifact delta

PR: #240 Base SHA: dc1e6032

Validation

head — `rivet validate` result
  SR-11 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-12 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-13 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-2 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-3 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-4 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-5 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-6 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-7 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-8 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-9 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SYS-1 (system-req, status: accepted) — missing: sys-integration-verification
  SYS-2 (system-req, status: accepted) — missing: sys-integration-verification
  SYS-3 (system-req, status: accepted) — missing: sys-integration-verification
  SYS-4 (system-req, status: accepted) — missing: sys-integration-verification
  SYS-5 (system-req, status: accepted) — missing: sys-integration-verification
  → run `rivet validate --explain SR-1` to see which link type and source types satisfy a gap

Result: PASS (199 warnings)
Schemas: common@0.4.0 (embedded), dev@0.6.0 (embedded), research@0.1.0 (embedded), research-ext@0.1.0 (on-disk), dev-ext@0.1.0 (on-disk), safety-case@0.1.0 (embedded), aspice@0.3.0 (embedded)
base — `rivet validate` result (for comparison)
  SR-11 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-12 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-13 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-2 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-3 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-4 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-5 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-6 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-7 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-8 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SR-9 (sw-req, status: accepted) — missing: sw-integration-verification, unit-verification
  SYS-1 (system-req, status: accepted) — missing: sys-integration-verification
  SYS-2 (system-req, status: accepted) — missing: sys-integration-verification
  SYS-3 (system-req, status: accepted) — missing: sys-integration-verification
  SYS-4 (system-req, status: accepted) — missing: sys-integration-verification
  SYS-5 (system-req, status: accepted) — missing: sys-integration-verification
  → run `rivet validate --explain SR-1` to see which link type and source types satisfy a gap

Result: PASS (199 warnings)
Schemas: common@0.4.0 (embedded), dev@0.6.0 (embedded), research@0.1.0 (embedded), research-ext@0.1.0 (on-disk), dev-ext@0.1.0 (on-disk), safety-case@0.1.0 (embedded), aspice@0.3.0 (embedded)

Artifact stats

base head
Total artifacts 272 273
full stats — head
Artifact summary:
  academic-reference               24
  competitive-analysis             11
  design-decision                  23
  feature                         102
  market-finding                    7
  requirement                      21
  safety-context                    3
  safety-goal                       5
  safety-justification              4
  safety-solution                   6
  safety-strategy                   1
  stakeholder-req                   3
  sw-req                           13
  sw-verification                  13
  sys-verification                  5
  system-req                        5
  technology-evaluation            12
  verification                     15
  TOTAL                           273

Orphan artifacts (no links): 12
  CA-001
  CA-002
  CA-003
  CA-004
  CA-005
  CA-006
  CA-007
  CA-008
  CA-009
  CA-010
  CA-011
  FEAT-078

Diagnostics: 0 error(s), 199 warning(s), 11 info(s)

Diff (base → head)

+ FEAT-102  v3.4 — MC/DC, WIT round-trip and claim-check fail on planted-bad input (scry#239)

1 added, 0 removed, 0 modified, 272 unchanged

AADL model — head

spar/scry.aadl: OK

Posted by the rivet-delta workflow. Informational only — does not gate the PR.

@avrabe
avrabe merged commit 4edc93c into main Nov 13, 2026
13 checks passed
@avrabe
avrabe deleted the fix/gate-potency-mcdc-wit-claims branch November 13, 2026 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Three required checks pass on planted-bad input (MC/DC, WIT round-trip, Doc claims)

1 participant