Repository navigation
test(solid): split the slow growth and round-trip test files, and time check:ci's parts in CI - #565
Merged
Merged
Conversation
Vitest runs a file's tests one after another, so the fourteen growth tests of solidDeckRepository.growth.test.ts ran in a row on one worker while the others sat idle, and that one file took most of the solid package's test task, the longest Turbo task of the check job. They are now four files, by what they write: a deck's cards (releases added or refused, distractors changed, cards removed), whole decks (removed and changed in a catalog), older formats brought up to date, and a learner's study (review states, answers, chapters). The setup they share moves to src/testing/deckWrites.ts. Every test keeps its name and assertions; growth() compares the CPU time of the test's own process, so running the files on separate workers doesn't change what a test measures. Refs: #562
… files releaseDrafts.roundtrip.test.ts took every release of the library in one file, so test:roundtrip ran them one after another on one worker, in the check job and in solid-memo-client's. The releases are now dealt into four files by the size of their Turtle, the largest first, each to the smallest file so far, so the large taxonomy and Solid Fundamentals versions spread over the files. The dealing is deterministic and lives in src/testing/releaseShards.ts, whose test holds that every release is in exactly one shard and that there is a file for each shard and no other, so no release is dropped or run twice. Refs: #562
The check job ran npm run check:ci as one step, so the jobs API, which anyone can read, gave one time for the lints, every Turbo task and the repository's checks together; finding the slow part took a local run. check:ci's parts are now scripts of their own, check:lints, check:tasks and check:repository, which npm run check and check:ci both call, so a local run and CI run the same things. The check job runs each as its own step, the later ones also when an earlier one failed, so a run still shows every failure, and coverage:summary still reports after the tests. Closes: #562
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
When the run has runners to spare, the
checkjob is the last required job:npm run check:citook 430–600 s when Turbo's cache missed, while the next jobs ended at about 300 s. Most of that is solid's test task, and most of that task was one file,solidDeckRepository.growth.test.ts. Vitest runs a file's tests one after another, so its growth tests ran in a row on one worker while the others sat idle. The round trip had the same problem: every release of the library in one file. This branch does what #562 proposes, in three commits.solidDeckRepository.cards,.decks,.formatsand.progress.growth.test.ts. The setup they share moves tosrc/testing/deckWrites.ts. All 14 tests keep their names, sizes and assertions.growth()compares the CPU time of the test's own process, so running the files on separate workers doesn't change what a test measures.releaseDrafts.{1..4}.roundtrip.test.ts. The releases are dealt into them by the size of their Turtle, the largest first, each to the smallest file so far, so the large taxonomy and Solid Fundamentals versions are spread out. The dealing is deterministic and lives insrc/testing/releaseShards.ts. Its test holds that every release is in exactly one shard, and that there is a file for each shard and no other, so no release is dropped or run twice.check:ci's parts are steps of their own in the check job:check:lints,check:tasksandcheck:repository, so the jobs API shows how long each takes.npm run checkandcheck:cicall the same scripts, so a local run and CI run the same things. A later part still runs when an earlier part fails, so a run shows every failure, but none runs afternpm ciornpm audit signaturesfails.src/testing/.Closes: #562