Skip to content

ci: enforce the self-hosted fleet label contract - #14

Merged
Aurora Capital (auroracapital) merged 1 commit into
mainfrom
ci/fleet-label-guard
Oct 17, 2026
Merged

Aurora Capital (auroracapital) merged 1 commit into
mainfrom
ci/fleet-label-guard

Conversation

@auroracapital

Copy link
Copy Markdown
Contributor

Waarom

Een self-hosted job waarvan runs-on het fleet-label mist, matcht geen enkele scale-up queue. Hij maakt dus nooit een eigen runner: hij kan alleen landen op een instance die een andere job toevallig maakte, en zit anders in de wachtrij tot GitHub's 24-uurs timeout.

Dat is precies hoe het misging:

  • de org-brede main-source-gate bleef hangen
  • en niet-gematchte events vielen door naar de on-demand queue, wat on-demand kosten veroorzaakte

De webhook-matcher matcht exact tegen een vaste labelset ([self-hosted, linux, x64, aws] en varianten), dus het label is functioneel, niet cosmetisch. Niets dwong het af.

Wat deze PR doet

scripts/fleet-label-guard.py parset elke workflow en faalt als een self-hosted runs-on geen fleet-label heeft.

Overgeslagen (met reden):

  • niet-self-hosted runners
  • self-hosted runners met een niet-Linux OS-label (macOS, Windows) — die horen bij een andere runner
  • ${{ ... }} expressies, want die resolven pas at runtime

Toegevoegd als blocking stap in organization-pr-baseline.yml, direct na de PyYAML-installatie.

Bewijs

Guard op de huidige workflows:

Checked 4 workflow file(s) for the fleet label contract.
OK: every self-hosted runs-on carries a fleet label.

Negatieve test op een expres kapotte workflow:

::error:: job 'bad' runs on self-hosted without a fleet label (['Linux', 'X64', 'self-hosted']). Add one of ['aws', 'aws-arm', 'aws-large', 'aws-od', 'aws-win'].
::error:: job 'alsobad' runs on self-hosted without a fleet label (['Linux', 'self-hosted']). ...

De goede, macOS- en dynamische jobs werden correct overgeslagen.

actionlint lokaal: exit 0.

Let op: afhankelijk van #13

main-source-gate mist vandaag het label, dus deze check is rood tot #13 landt. Deze branch is daarom op #13 gebaseerd. Merge #13 eerst.

Context

Onderdeel van de ephemeral runner optimalisatie. Item #3 uit het voorstel: het labelcontract afdwingbaar maken in plaats van een afspraak.

Copilot AI balanced review requested due to automatic review settings October 17, 2026 10:05
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-17T10:05:38.584965Z 6ad80d7 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

A self-hosted job whose runs-on omits the fleet label matches no scale-up
queue. It therefore never spawns its own runner: it can only land on an
instance another job happened to create, and otherwise sits queued until
GitHub's 24-hour timeout. That is how the org-wide main-source-gate
stalled, and how unmatched events fell through to the on-demand queue.

The webhook matcher matches exactly against a fixed label set
([self-hosted, linux, x64, aws] and its siblings), so the label is
load-bearing rather than cosmetic. Nothing enforced it.

scripts/fleet-label-guard.py parses each workflow and fails when a
self-hosted runs-on carries no fleet label. It skips non-self-hosted
runners, non-Linux self-hosted runners such as macOS, and expressions
that resolve at runtime.

Wired into organization-pr-baseline.yml as a blocking step, placed after
PyYAML is installed.

Depends on #13: main-source-gate omits the label today, so this check is
red until #13 lands. The branch is based on #13 for that reason.
@auroracapital
Aurora Capital (auroracapital) merged commit 77fc502 into main Oct 17, 2026
5 of 6 checks passed
@auroracapital
Aurora Capital (auroracapital) deleted the ci/fleet-label-guard branch October 17, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants