Skip to content

ci: run touched port tests on pull requests - #509

Merged
dmealing merged 9 commits into
mainfrom
fm/mo-pr-ci
Oct 30, 2026
Merged

dmealing merged 9 commits into
mainfrom
fm/mo-pr-ci

Conversation

@dmealing

@dmealing dmealing commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

pr-ci run on this PR (all jobs green, java-slow included): https://github.com/metaobjectsdev/metaobjects/actions/runs/40996491721

What Changed

  • Adds .github/workflows/pr-ci.yml. On each pull request it selects the ports touched by the change via scripts/ci-affected-ports.sh and runs their ci-local.sh lanes with --no-integration. Docs-only changes run just the gates lane. A final pr-ci job passes only when every selected lane passed.
  • Updates CONTRIBUTING.md and the conformance.yml and scripts/ci-local.sh header comments to describe the new PR behavior. Docker integration suites stay on release tags and manual dispatch.
  • Adds a paid-work commitment to GOVERNANCE.md: a port maintainer gets first call on paid work in their port, as an offer only, with no role, vote or review exception.
Validation

Risk Assessment

✅ Low: Additive PR workflow that wraps the existing ci-local.sh lanes the same way conformance.yml does, plus docs; both earlier-round fixes (empty env key removed, ts timeout raised to 60) are verified in place.

Testing

Driven live: the port selector maps a real commit range to the expected ports, a root-markdown-only change selects no ports, and the ci-local plan lists the TS lanes. Baseline TS lanes were green. Not driven: the pr-ci workflow loading on GitHub and running a touched port's tests on a real PR, which needs a pushed PR. The full repository regression suite (scripts/ci-local.sh full run, including Java, Python, C# lanes) was not run in this step; only the scoped baseline ran. Overall: no failures found, but live coverage is too thin to call go.

  • Live validation: ⚠️ inconclusive - 4 of 7 scenarios driven live against the product
Scenario Result Live Evidence
pr-ci detect job loads with no empty env key on the sel step ⏸️ untested no Needs GitHub's workflow parser, which only runs on a pushed PR. Pushing is outward-facing and was not authorized. To drive: push a PR branch to metaobjects and check the pr-ci run loads.
Port selector maps a touched port's commit range to that port ✅ pass live scripts/ci-affected-ports.sh HEAD~1 HEAD -> ts java python csharp
Docs-only change at repo root selects no ports ✅ pass live scripts/ci-affected-ports.sh <CONTRIBUTING-only commit> -> empty output
Empty commit range selects no ports (adversarial) ✅ pass live scripts/ci-affected-ports.sh 33ac841 33ac841 -> empty output
ts lane plan lists build, typecheck, conformance and mutation steps ✅ pass live MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
pr-ci runs the touched port's tests on a real PR on GitHub ⏸️ untested no Needs a pushed PR and GitHub Actions run. Not authorized to push here. To drive: open a PR touching server/typescript and confirm the ts job runs and reports.
Full repository regression suite ⏸️ untested no Full scripts/ci-local.sh run covers Java, Python, C# lanes and the Bun suite (>5 min for TS alone). Not run in this step. To drive: run scripts/ci-local.sh with no flags.
Evidence: selector and plan command output (live)
scripts/ci-affected-ports.sh HEAD~1 HEAD -> ts java python csharp
scripts/ci-affected-ports.sh <CONTRIBUTING-only commit> -> (none)
MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only ts-fast --only ts-unit -> ts build+typecheck, conformance: typescript, ts unit suites, completeness-gate (mutation)
  • Outcome: ⚠️ 1 warning across 1 run (25m33s)

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • ⚠️ .github/workflows/pr-ci.yml:89 - The ts job sets timeout-minutes: 40, and its ts-fast matrix leg carries the ~30-minute mutation gate. Measured hosted durations: 35m13s on this branch's own pr-ci run (40810755990, 03:12:02 to 03:47:15). The nightly conformance.yml ts-fast leg took 30m51s and 27m53s. That leaves about 5 minutes of headroom. A slow or contended hosted runner will hit the cap and report the lane cancelled. The pr-ci aggregate then goes red on a correct TS change, which teaches contributors to ignore the check. conformance.yml sets no timeout for the same lane (default 360). Fix: raise timeout-minutes on the ts job to about 60. Alternatively, give the ts-fast leg its own larger value and keep ts-unit's cap small.
  • ℹ️ .github/workflows/pr-ci.yml:121 - The java job declares a strategy.matrix with one entry (lane: [java-slow]) and then routes it through a LANE env var. The matrix only adds indirection. A plain step running scripts/ci-local.sh --only java-slow --no-integration --strict-toolchains is the narrower form with the same behavior. pr-ci is not yet a required check, so the job name changing from java (java-slow) to java breaks nothing.

🔧 Fix applied.
1 info still open:

  • ℹ️ .github/workflows/pr-ci.yml:121 - The java job declares a strategy.matrix with one entry (lane: [java-slow]) and then routes it through a LANE env var. The matrix only adds indirection. A plain step running scripts/ci-local.sh --only java-slow --no-integration --strict-toolchains is the narrower form with the same behavior. pr-ci is not yet a required check, so the job name changing from java (java-slow) to java breaks nothing.
⚠️ **Test** - 1 warning
  • ⚠️ live validation verdict: inconclusive (4 of 7 scenarios were driven live against the product); untested: pr-ci detect job loads with no empty env key on the sel step, pr-ci runs the touched port's tests on a real PR on GitHub, Full repository regression suite
  • Live validation: ⚠️ inconclusive - 4 of 7 scenarios driven live against the product
Scenario Result Live Evidence
pr-ci detect job loads with no empty env key on the sel step ⏸️ untested no Needs GitHub's workflow parser, which only runs on a pushed PR. Pushing is outward-facing and was not authorized. To drive: push a PR branch to metaobjects and check the pr-ci run loads.
Port selector maps a touched port's commit range to that port ✅ pass live scripts/ci-affected-ports.sh HEAD~1 HEAD -> ts java python csharp
Docs-only change at repo root selects no ports ✅ pass live scripts/ci-affected-ports.sh <CONTRIBUTING-only commit> -> empty output
Empty commit range selects no ports (adversarial) ✅ pass live scripts/ci-affected-ports.sh 33ac841 33ac841 -> empty output
ts lane plan lists build, typecheck, conformance and mutation steps ✅ pass live MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
pr-ci runs the touched port's tests on a real PR on GitHub ⏸️ untested no Needs a pushed PR and GitHub Actions run. Not authorized to push here. To drive: open a PR touching server/typescript and confirm the ts job runs and reports.
Full repository regression suite ⏸️ untested no Full scripts/ci-local.sh run covers Java, Python, C# lanes and the Bun suite (>5 min for TS alone). Not run in this step. To drive: run scripts/ci-local.sh with no flags.
  • scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
  • sed -n 54,58p .github/workflows/pr-ci.yml (confirmed no empty env: on the sel step)
  • scripts/ci-affected-ports.sh HEAD~1 HEAD on merge commit 797c5b38a -> ts java python csharp (live selector run)
  • scripts/ci-affected-ports.sh &lt;P&gt; &lt;C&gt; over the only-CONTRIBUTING.md commit -> no ports (live, docs-only at repo root selects nothing)
  • scripts/ci-affected-ports.sh 33ac84184 33ac84184 (empty range) -> no ports
  • MO_CI_LIST_ONLY=1 scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains (live plan listing)
  • Baseline scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains (given, already green)
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@dmealing
dmealing merged commit 8957798 into main Oct 30, 2026
9 checks passed
@dmealing
dmealing deleted the fm/mo-pr-ci branch October 30, 2026 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant