test(podman): wire podman_preflight into CI and add a Podman e2e coverage check

[politerealism]

Summary

Track A of #3712: wires podman_preflight into CI (it never ran anywhere before this) and adds an automated check that fails CI if a new e2e-podman-eligible test target isn't accounted for anywhere, so the kind of silent gap this issue documents can't recur.

Related Issue

Addresses part of #3712 (Track A only — see the issue comment recording the full plan). Does not duplicate open PR #3637, which independently covers most of the remaining 39-vs-20 gap via a different mechanism; that work is treated as a prerequisite for Track B, tracked separately.

Changes

• podman_preflight wiring: it needs only the standalone openshell-driver-podman binary, not a gateway, so it never fit the gateway-backed PODMAN_CI_TESTS harness and simply never ran. Added a new e2e:podman:preflight mise task and chained it onto the podman-external-driver-e2e job, which already builds and exports that exact binary artifact for other reasons. Both commands now run unconditionally with combined exit-code checking, so a flaky/failing e2e:podman:external-driver run can't silently mask podman_preflight from ever executing — the same invisible-coverage problem this issue describes, just relocated, if chained with && instead.
• Coverage drift check (tasks/scripts/check-podman-e2e-coverage.sh, new podman-e2e-coverage job in branch-checks.yml): enumerates every e2e-podman-eligible test target via cargo metadata plus each auto-discovered file's own #![cfg(feature = ...)] gate, and requires every one to be accounted for in PODMAN_CI_TESTS, a perf-benchmark ignore list (verified via source inspection that every test in those files actually carries #[ignore], since cargo test -- --list doesn't distinguish ignored tests in its output), a separately-wired list (for tests like podman_preflight that don't fit the gateway harness), or a temporary, itemized known-gaps list tied to test(podman): run existing Podman behavioral e2e coverage in CI #3712 for the pre-existing gaps this check surfaces on introduction. It also fails on stale known-gaps entries, so that list can't rot in the other direction once Track B resolves them.
• Documentation: new "Podman e2e test selection" section in CI.md.

Testing

• mise run pre-commit passes
• Unit tests added/updated — N/A, no product code changed
• E2E tests added/updated:
  • podman_preflight built and run for real locally: cargo build -p openshell-driver-podman && OPENSHELL_EXTERNAL_DRIVER_BIN=$PWD/target/debug/openshell-driver-podman mise run e2e:podman:preflight — both assertions pass.
  • check-podman-e2e-coverage.sh verified against both the happy path (39/39 accounted for, matching the issue's own count exactly) and three deliberately-broken cases: a real gap (removing an entry from the known-gaps list), a stale known-gaps entry (adding an already-accounted-for target), and a non-#[ignore]d perf-benchmark test — all three correctly fail with the expected error message and a real non-zero exit code.
  • Traced the full chained-command path: confirmed the cleanup trap in with-podman-gateway.sh stops the driver process but never deletes the binary file, so the two chained tasks don't conflict over the artifact; confirmed OPENSHELL_EXTERNAL_DRIVER_BIN is exported via GITHUB_ENV in an earlier step and persists across both; confirmed the exit-code-combining shell logic behaves correctly for all three pass/fail combinations locally.
  • Confirmed every layer (the test's own 60s timeout, the driver's bounded ~10s retry, with-podman-gateway.sh's bounded 30s/120s wait loops, and the job's 30-minute timeout-minutes) is properly bounded — nothing can hang or retry indefinitely.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable) — not applicable; CI-only change, documented in CI.md instead

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[elezar]

Thanks for investigating the missing Podman coverage. I agree with the underlying problem, but I do not think we should merge this PR in its current form.

The proposed coverage check institutionalizes the legacy PODMAN_CI_TESTS allowlist just as #3597 and #3637 are replacing that model. #3597 moves portable behavior such as sync into driver-agnostic conformance, while #3637 builds a default-inclusive Podman nextest archive with an explicit follow-up list. Once those land, new eligible tests are included automatically, so the main invariant becomes exclusion hygiene—not reconciling several independent selection lists.

The new check also measures whether targets are “accounted for,” rather than whether they execute. Known gaps, benchmark exclusions, and SEPARATELY_WIRED_TARGETS all count as accounted for. In particular, podman_preflight remains accepted if its workflow invocation is later removed, so the same silent omission can recur while the coverage check remains green.

I also do not think podman_preflight belongs in the Podman E2E workflow. It does not require Podman or a gateway; it exercises the standalone driver’s missing-socket retry and error path. Those semantics should be covered by driver unit tests, with at most one executable integration test under crates/openshell-driver-podman/tests/. The normal workspace test lane can then run it without a special mise task, workflow command, or coverage exception.

There is also a correctness problem in the current workflow command: GitHub Actions runs run: scripts under bash -e, so if e2e:podman:external-driver fails, the shell exits before s1=$? and before e2e:podman:preflight runs. The final commit therefore does not actually guarantee that both tasks execute.

I suggest:

1. Land #3597 and #3637 first.
2. Move the preflight behavior into openshell-driver-podman unit/integration tests.
3. Use the tmachine archive’s default-inclusive selection as the source of truth.
4. If an additional check is still needed, make it a small validator for stale or undocumented exclusions, ideally using the same declarative exclusion list consumed by archive construction.
5. Report selected, excluded, and not-run targets separately instead of aggregating them as “accounted for.”

This preserves the useful goal of preventing silent coverage gaps without adding a new required CI job, a 228-line parser, and documentation for a selection model that is already being replaced.

[politerealism]

Closing in favor of #3783, which takes a substantially narrower approach per the review above.

Per your feedback:

• Landed test(e2e): run podman suite with tmachine #3637 and test(conformance): migrate file transfer coverage #3597 first (both merged) rather than duplicating their work.
• Moved podman_preflight into crates/openshell-driver-podman/tests/ as a plain Cargo integration test — it's driven by CARGO_BIN_EXE_openshell-driver-podman and runs via the normal required workspace test job, no special mise task, workflow command, or coverage exception.
• Dropped the coverage/exclusion check entirely rather than reworking it. test(e2e): run podman suite with tmachine #3637's tmachine e2e-podman archive is already default-inclusive, which closes the actual silent-gap failure mode this PR was trying to guard against — a stale-exclusion validator would only catch list hygiene, not coverage gaps, so it isn't worth the added CI surface right now.

The rest of #3712 (shared-target coverage, podman_oci_identity/podman_resource_limits/provider_refresh_handles against a real gateway, rootful/rootless explicitness, Release Dev/Tag qualification, selected-vs-eligible reporting) is being tracked as a separate follow-up rather than folded into this PR.