Skip to content

ci-validation.md

Latest commit

 

History

History
199 lines (166 loc) · 11.8 KB

ci-validation.md

File metadata and controls

199 lines (166 loc) · 11.8 KB
Raw

Native platform evidence

Rust core: Dagger and Actions

.github/workflows/native-core.yml runs the Linux checks through the local Dagger module in .dagger/. Native Windows and macOS runners execute the same tools/check_native.sh script directly. Dagger's Linux container does not establish Windows or macOS correctness. The Go SDK is CI tooling only; the product core remains Rust.

From the repository root on Windows, with the Podman machine running and the native Dagger CLI on PATH (PowerShell):

dagger call evidence --source . --revision (git rev-parse HEAD) export --path D:\BuildArtifacts\lesscode\native-evidence
dagger call check --source . --revision (git rev-parse HEAD)

No WSL distribution is involved; the CLI starts its engine as a container in Podman. On 2026-11-10 check passed this way in 250 s from a cold engine with CLI 0.21.11, while dagger.json and the Actions workflow stay pinned to 0.21.8. On Linux or macOS use the same two commands with "$(git rev-parse HEAD)" and a local export path.

Use local-working-tree as the revision for uncommitted changes. The revision argument is a provenance label, not an integrity check on the supplied directory. The Actions checkout and its recorded SHA identify the tested CI revision. No host Go installation or Dagger Cloud account is required. Module bindings are generated by Dagger and excluded from Git; commit the module source and Go locks.

The pipeline pins Rust 1.98.1, nextest 0.9.143 and Dagger 0.21.8. The official Dagger Actions integration is pinned to v8.4.1's commit. Only native source, tests and the check script enter the container; local environments, experiments and build outputs are filtered. See Dagger's directory filtering.

Both the Actions runners and the Dagger container install nextest with tools/install_nextest.sh, which downloads the 0.9.143 release binary for the platform and verifies it against a SHA-256 digest pinned in the script. Compiling nextest from source took between two and twelve minutes per job and was most of each run. Updating nextest means changing the version and the digests together; the script header shows how to list them. No build cache is used.

Checks cover formatting, Clippy with warnings denied, nextest with four workers, doctests, build and CLI help. Every command retains a log and exit code. evidence exports these even when a check fails; check enforces the aggregate exit code. Actions runs both in order and uploads evidence with if: always(). Setup or container failures remain failed jobs, even if no evidence directory could be produced. Artifacts are retained for 14 days. Dagger may reuse execution results for identical content and arguments; source changes or a different revision invalidate the check execution. The enforcement call reuses the same evidence result rather than running the suite twice. Cargo dependency/build caches are separate from this content-addressed execution cache.

The first hosted run 41333477507 passed all three jobs: Linux 103 tests, Windows 102 and macOS 103, with zero skips. All check exit codes were zero. The separate doctest command passed with zero doctests defined. The durable audit records the PR merge checkout, matching reviewed native tree and artifact IDs/digests. Local WSL Dagger also passed, including an intentional failure probe that confirmed evidence export and nonzero enforcement; see the local receipt. Its pending-Actions label describes the earlier capture time, superseded by the hosted audit.

Multi-Platform and Multi-Architecture CI Matrix

.github/workflows/native-core.yml verifies native core correctness across operating systems and CPU architectures:

  • Linux x86_64: Verified through containerized Dagger execution (rust:1.98.1-bookworm).
  • Linux aarch64 (ARM64): Verified through native GitHub-hosted ubuntu-24.04-arm runners.
  • macOS aarch64 (Apple Silicon): Verified natively on Apple Silicon through GitHub-hosted macos-15 runners.
  • macOS x86_64 (Intel): Verified natively on Intel through GitHub-hosted macos-15-intel runners.
  • Windows x86_64: Verified natively through GitHub-hosted windows-2025 runners.

Adopter check in containers

tools/adopter-check.compose.yaml installs the released binary in a clean ubuntu:24.04 container with the published install.sh and runs the documented commands against this repository, mounted read-only (tools/adopter_check.sh). It asserts the exit codes of density, analyze (Rust, TypeScript, Java) and budgets, and prints the two R55 gaps (Python analyze, mutate) so that a fix shows up as changed output.

docker compose -f tools/adopter-check.compose.yaml up --abort-on-container-failure
docker compose -f tools/adopter-check.compose.yaml down

Set INSTALLER=local to test scripts/install.sh from the checkout before a release, or LESSCODE_VERSION to pin a release. The linux-arm64 service (run --rm linux-arm64) needs arm64 emulation in the container engine. It is a local check; no workflow runs it.

Release binaries

.github/workflows/release.yml builds and signs the adopter binaries. Pushing a tag of the form vX.Y.Z runs two stages:

  1. build compiles lesscode-core with Rust 1.98.1 and --locked on the same five runners as the correctness matrix, one per target: x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-pc-windows-msvc, x86_64-apple-darwin, aarch64-apple-darwin. Each job runs the built binary once (--version), packs it with LICENSE, LICENSES.md and THIRD_PARTY_NOTICES.txt into lesscode-core-<target>.tar.gz (.zip on Windows) and keeps the unsigned archive as a workflow artifact. These jobs hold no secrets and cannot write to the repository.
  2. publish runs only if all five builds succeeded, in the release environment, so it waits for the owner's approval. It checks that the tag equals the version in native/Cargo.toml, installs zipsign 0.2.2, signs every archive, writes the .sha256 files and SHA256SUMS after signing, signs SHA256SUMS, attests build provenance, uploads all files together with scripts/install.sh and scripts/install.ps1 to a draft release and then publishes it as latest. It fails closed without the signing key or when the key is not listed in scripts/release-public-keys.txt. Release signing has the owner setup and how to verify.

Asset names carry no version, so https://github.com/norandom/lesscode/releases/latest/download/<asset> always resolves to the newest release. The install scripts rely on that.

Starting the workflow by hand (workflow_dispatch) runs only the build stage. Use it to check packaging before tagging. The release workflow does not run the test suite; tag only a commit whose Native core correctness run is green. The Linux binaries link against the runner's glibc (2.39 on Ubuntu 24.04).

To cut a release: bump the version in native/Cargo.toml, native/Cargo.lock, CITATION.cff, .zenodo.json and the README badge; push; wait for green CI; push the tag; approve the release environment when asked. Zenodo archives the release when it is published.

Historical Python baseline (Archived)

The Python prototype (lesscode-baseline) and its corresponding .github/workflows/baseline-platforms.yml workflow were retired and archived at commit fd0b5b9. The historical evidence and reports remain preserved in evidence/. runtime targets or environment configuration during integrity verification.

tools/ci_validate.py retains commands, working directories, exit outcomes, stdout/stderr, timings and runtime identities. The helper supplies a canonical dedicated scratch directory through TMPDIR, TEMP and TMP; its path is recorded in runtime.json. This avoids macOS's default /var alias while preserving strict source and artifact symlink guards. Metric staging independently canonicalizes its owned temporary root before launching analyzers, keeping Windows 8.3 paths and actual process cwd consistent. It runs canonical verbose unittest discovery with absolute source PYTHONPATH, followed by the native parent/descendant timeout test. Host-inapplicable unittest skips remain visible in the canonical log. Other failures remain failures. A failed test stage does not prevent independent build and installed-package evidence from being gathered.

The same job builds the wheel, creates a fresh uv environment outside the checkout, installs the wheel without dependencies, and checks both console and module help/version from that external directory with PYTHONPATH absent. It then runs tools/validate_pilot.py: six genuine evaluations, repeated structural metrics, known complexity/duplication values, preserved originals, behavior rejection and experiment aggregation. Completed pilot recovery uses --resume and verifies the retained artifacts read-only. Partial pilots require a fresh output directory and cannot become a resumed success.

The baseline-<runner>-<attempt> artifact is uploaded even after a failed stage. It retains bootstrap logs, validation/commands.json, runtime.json, native test diagnostics, probe results, the wheel and pilot artifacts for 14 days. The disposable installed environment is excluded. GitHub job logs also retain action setup failures. Analyzer dependencies are locked; only uv's dependency cache is enabled. Evaluations and result artifacts are never cached. Uploaded analysis inputs are exclusively the checked-in public fixtures; there are no model calls or uploads of user projects.

Notice files use targeted Git -text attributes because their names identify exact distribution bytes, including upstream CRLF. The toolchain tests verify all retained notice hashes and Git's effective attribute. Preserve those bytes when changing the inventory; never normalize an upstream notice to repair its filename hash.

A green workflow configuration or a local Windows check is insufficient cross-platform evidence. Read the native job outcomes, canonical skips, validation/result.json and pilot validation.json for the tested revision. An unexecuted platform remains outstanding. This validates the harness on the declared fixtures; it does not establish general simplification effectiveness.

Action SHA pins were resolved from official upstream tags on 2026-10-31:

Runner labels are documented in the GitHub-hosted runner reference.

The independently reviewed native run 41198054658 tested exact runtime revision 778e46258dbca42d5b07191a4f011b2cb858fb61. All three jobs passed 363 tests each and all 51 stages passed. The durable compact audit summary retains exact job and artifact IDs, archive digests, skips and measured pilot outcomes. Raw workflow artifacts have 14-day retention and may expire; rerun the pinned workflow at the desired revision to regenerate inspectable raw evidence. See integrated readiness for runtime equivalence, fresh installed validation, all-criteria coverage and remaining review gates. This record establishes fixture-level evaluator validation only.