Repository navigation
CI: a change to one tool or test project runs the test projects it reaches, not the whole solution #268
Copy link
Copy link
Closed
Labels
area/toolingThe repository's tools: prober, verdict, guard, docs-query, smoke testThe repository's tools: prober, verdict, guard, docs-query, smoke testchoreBuild, CI, tooling, packaging, refactoring with no behaviour changeBuild, CI, tooling, packaging, refactoring with no behaviour change
Milestone
Description
Why
CI picks its lanes by path class (tools/CiLanes, decision 113), and each lane builds and tests the whole solution. A change to one tool or one test project therefore runs every test project on Linux and Windows.
#266 changed tools/BashGuard/Program.cs and its tests, plus docs and one line of .gitignore. ci-lanes classify put the files in these classes:
| Files | Class | Lanes |
|---|---|---|
AGENTS.md, CONTRIBUTING.md, design/** |
prose | Linux |
tools/BashGuard/Program.cs, test/CodoMetis.TypeKit.BashGuard.Tests/GuardTests.cs |
code no package reads | Linux, Windows |
.gitignore |
unlisted | Linux, Windows, Pack |
Its first run (41215127177) took 12m55s on Linux and 13m32s on Windows, and it packed and smoke-tested too. The tests that could notice the change were CodoMetis.TypeKit.BashGuard.Tests (112 tests, 5 s locally) and Conventions. A docs-only PR like #267 runs test/Prose.slnf alone, in 1m49s.
Proposal
- The test suite follows the projects a change reaches, not only its lane. When every changed path that isn't prose is under
tools/<Name>/**ortest/<Project>/**, CiLanes outputs a suite of the changed test projects, every test project that references a changed project (transitively, throughProjectReference), Conventions, and thetest/Prose.slnfprojects when prose changed too. The legs build and test that suite the way they buildtest/Prose.slnftoday. Any other path (src/**,tools/Shared/**,tools/Directory.Build.*, the build files, an unlisted path) keeps the whole solution. - A test that reads a project by path, not by reference, counts as reaching it. Conventions reads the repository by globbing, and some tests may run a tool's DLL or read its files. Measure which tests do that first. Either declare those dependencies so CiLanes reads them, or keep that tool on the whole solution. A dependency left undeclared means a test that silently stops running.
.gitignoregets a class. Find which tests read it (git listings, the packability rules) and send it to the lanes those tests run in. Pack is not one of them, sincedotnet packdoesn't read.gitignore. Check the other root dotfiles at the same time:.gitattributessets line endings, so it stays on Windows.- The Windows leg keeps its reason. "code no package reads" goes to Windows because of paths, line endings and child processes. A narrower suite on Windows still includes Conventions and the changed tool's tests.
- A decision record amending 0113's selection, numbered with the next free number when it is merged (0125 and 0126 are taken, CI: split the Linux test step across two jobs, since it is bound by the runner's four cores #253's text assumes 0127 and 0128 for Release: publish the packages CI packed instead of building them again, and check the tag before CI #247 and CI: cancel a superseded release-branch push run, and select the next push's lanes from the last green one #248, so check against
release/2.4.0before merging).
Done when
- CiLanes.Tests have rows for a tool-only change, a test-project-only change, a change to
tools/Shared, a mix withsrc/, and.gitignore. Each is shown failing first against the current selection (verify-the-guard). - A conventions test fails when a test project reads a tool by path without declaring it, shown by seeding one.
- A PR shaped like build: run parallel work in worktrunk worktrees beside the repository #266 (one tool, its tests, docs) runs a narrowed suite on both legs, with no Pack. Its run time is measured in this issue against 41215127177.
- The gate still fails a run whose selected suite was skipped or cancelled.
Metadata
Metadata
Assignees
Labels
area/toolingThe repository's tools: prober, verdict, guard, docs-query, smoke testThe repository's tools: prober, verdict, guard, docs-query, smoke testchoreBuild, CI, tooling, packaging, refactoring with no behaviour changeBuild, CI, tooling, packaging, refactoring with no behaviour change
Activity
CaffeinatedCoder commentedon Oct 31, 2026
Implemented in #274 (decision 0127).
This PR's own run, 41236579959, cannot show the narrowing: it changes
tools/CiLanesand.github/workflows/dotnet.yml, so the changes job selected the whole solution on Linux and Windows with Pack and Consumer, exactly as before (Changes 10 s). Its timings go here when it completes, against 41215127177: Changes 27 s, Test (ubuntu-latest) 12m55s, Test (windows-latest) 13m32s, Pack 7m26s, Consumer 6m10s, wall clock 14m15s.What a #266-shaped PR selects with #274, measured locally with
ci-lanes suiteover #266's eleven paths (af3f95e^1..af3f95e):.gitignorewas unlisted)test/Reach.slnf: the 5 projects oftest/Prose.slnf,tools/BashGuard,CodoMetis.TypeKit.BashGuard.Teststest/Reach.slnfwritten as the leg writes it builds locally (17 projects with the shipping projects they reference). The first CI measurement of a narrowed run needs the first tool-or-test PR intorelease/2.4.0after #274 merges; that run's leg times belong here too.Measured while implementing: the only test projects that read another project by path rather than by reference are the conventions (the whole repository) and the oracle tests (every test project's sources, for each claim's test); both now declare it with
<ReadsByPath>. No test reads.gitignore: the conventions list the index (git ls-files), anddotnet packdoes not read it.CaffeinatedCoder commentedon Oct 31, 2026
Landed on
release/2.4.0by #274 (merge commit 500231d), with decision 127. The narrowed suite is measured on the first PR shaped like #266 after this merge.