Skip to content

Fix stale ccache keys and slow Debug CI tests - #301

Merged
zfergus merged 1 commit into
mainfrom
chore/ci-speedup
Nov 10, 2026
Merged

zfergus merged 1 commit into
mainfrom
chore/ci-speedup

Conversation

@zfergus

@zfergus zfergus commented Nov 9, 2026

Copy link
Copy Markdown
Member

Description

Fixes the CI caching and Debug test setup across the Build, Python, Coverage, CUDA, and Docs workflows, plus one pathological unit test. On #300, CI took ~40 min wall-clock, and Linux Debug failed on a timeout.

  • ccache keys — every workflow except the ODR check used a fixed key (Linux-Debug-cache, coverage-cache, …). Cache entries are immutable, so each was saved once on 2026-10-12 and never updated. Every build job on Fix Eigen ABI pin when a parent project shadows the Eigen recipe #300 got 0/465 hits, while the ODR job, which keys by run_id, got 459/465 on the same commit.
  • Windows — Windows Debug's /Zi writes to a shared PDB that ccache can't cache (465/465 uncacheable). Windows Python used the Visual Studio generator, which ignores CMAKE_CXX_COMPILER_LAUNCHER and compiles each target's files serially (2,340 s build vs 616 s on Linux).
  • Debug tests — at -O0, the Debug test step took 1,079–1,713 s vs 131–166 s in Release. CCD stops at the thickness alone took 1,319 s on macOS and timed out at 1,500 s on Linux, also failing main in run 42642942563.

Impact:

  • PR builds restore the latest main ccache instead of compiling from scratch.
  • The Debug test step is no longer bound by one 20+ minute test, and the flaky timeout is gone.
  • No library or API changes.

CI changes

  • Restore with actions/cache/restore using key <prefix>-ccache-${{ github.run_id }} and restore-keys: <prefix>-ccache-, the same pattern as odr-check.yml.
  • Save with actions/cache/save only on pushes to main (always for docs.yml, which only runs on push), so PR caches don't evict main's under the 10 GB limit.
  • Windows Debug configures with /Od /Ob0 /RTC1, matching what cuda.yml already does.
  • Windows Python sets up MSVC and builds with CMAKE_GENERATOR=Ninja.
  • Linux/macOS Debug builds at -O1 instead of -O0; NDEBUG stays undefined, so asserts are still checked.

Test change

  • CCD stops at the thickness uses a TightInclusionCCD with a 1e-3 tolerance instead of the default 1e-6.
  • Tight Inclusion slows down sharply as the minimum separation grows, and this test uses 0.2. Its assertions only check the step to within 0.1.
  • Removed the Windows-Debug SKIP, since the test is now cheap everywhere.
before after
CCD stops at the thickness, Release (5 broad phases) 15.6 s 0.83 s
Full suite, Debug -O1, 456 tests — 12.8 s
  • The old *-cache entries no longer match any key and will expire after 7 days unused.
  • This PR's own builds will still be cold: the new keys only get populated by the first push to main after merge.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (non-breaking change which improves existing functionality)

How Has This Been Tested?

  • ipc_toolkit_tests "[thickness]" in Release: all 9 test cases pass (6,347 assertions).
  • Full ctest -j 8 on a Debug build configured with -DCMAKE_CXX_FLAGS_DEBUG=-O1: 456/456 pass, 12.8 s.
  • Full ctest -j 8 in Release: 457/457 pass, 43.4 s.
  • All five edited workflows parse as YAML.
  • Workflow behavior (cache restore/save, Windows Ninja Python build) is verified only by this PR's CI and the first post-merge run on main.
  • Before numbers come from the job logs of Fix Eigen ABI pin when a parent project shadows the Eigen recipe #300.

Test Configuration:

  • OS and Version: macOS 26 (Apple Silicon)
  • Compiler and Version: Apple Clang 21

Checklist

  • I have followed the project style guide
  • My code follows the clang-format style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

- Key ccache by run ID and restore by prefix; save only on main
- Drop /Zi from Windows Debug so ccache can cache it
- Build Windows Python with Ninja so ccache runs
- Build Linux/macOS Debug at -O1 to keep asserts but inline Eigen
- Loosen the CCD tolerance in the thickness test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.86%. Comparing base (f388443) to head (2827265).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #301   +/-   ##
=======================================
  Coverage   96.86%   96.86%           
=======================================
  Files         229      229           
  Lines       23047    23047           
  Branches     1619     1618    -1     
=======================================
  Hits        22324    22324           
  Misses        723      723           
Flag Coverage Δ
unittests 96.86% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@zfergus
zfergus merged commit 522d6d1 into main Nov 10, 2026
22 checks passed
@zfergus
zfergus deleted the chore/ci-speedup branch November 10, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant