Share a linked worktree's local cache with the main worktree

[mshanemc]

Problem

git worktree add gives each linked worktree its own .wireit cache, so the same inputs miss in every checkout.

Change

Set WIREIT_CACHE_WORKTREES=true to write a linked worktree's local cache into the main worktree's matching .wireit folders. Shared entry names use package-relative paths, so both worktrees hit the same entry. A second write of an existing shared entry is a hit.

Any other value, including unset, keeps the historical layout: each worktree's own cache, absolute fingerprints, and an error if an entry already exists. Fingerprints and GitHub Actions caching are unchanged. The flag follows CI: only the exact string true enables it.

A package inside a submodule keeps its own cache. The nearest .git is that submodule, and its git dir has no commondir.

Test plan

• Two absolute checkouts produce different fingerprint strings and the same shared local-cache entry name
• A second set of the same entry throws with the flag off, and is a hit with the flag on
• A linked worktree restores a file written into the main worktree's local cache
• WIREIT_CACHE_WORKTREES=true enables the flag; yes leaves it off
• npm run test:cache-local, npm run test:cli-options, and npm run test:errors-usage

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[mshanemc]

CLA: done

[aomarks]

Thanks for the PR! I think this is a good feature. Just a few things to address before landing:

1. Shared cache location with less-common git layouts

Currently this PR would create new shared caches in some odd locations when worktrees are used with bare clones, submodules, and with --separate-git-dir. I'm fine with either not sharing a cache in that case, or with picking a more appropriate cache location for each case:

1. Worktrees outside the main checkout
   • Cache location: ✅ <main checkout>/<package>/.wireit/<script>/cache
2. Worktrees inside the main checkout (e.g. .claude/worktrees/)
   • Cache location: ✅ <main checkout>/<package>/.wireit/<script>/cache
3. Bare clone + worktrees (e.g. a .bare folder beside the worktrees)
   • Cache location: ❌ <parent of bare clone>/<package>/.wireit/<script>/cache
   • Problem: cache saved outside the repo
   • How to detect: the shared git folder's config file sets core.bare = true
   • What to do: skip sharing, or use <bare clone>/wireit/<package>/.wireit/<script>/cache
4. Worktree of a submodule
   • Cache location: ❌ <parent of submodule's git folder>/<package>/.wireit/<script>/cache
   • Problem: cache saved inside the outer repo's git folder
   • How to detect: the shared git folder's config file sets core.worktree
   • What to do: skip sharing, or use <core.worktree checkout>/<package>/.wireit/<script>/cache
5. Separate git folder + worktrees (git clone --separate-git-dir)
   • Cache location: ❌ <parent of git folder>/<package>/.wireit/<script>/cache
   • Problem: cache saved outside the repo
   • How to detect: the shared git folder isn't named .git (git only names it .git when it sits inside the main checkout, so any other name means its parent isn't a checkout)
   • What to do: skip sharing, or use <git folder>/wireit/<package>/.wireit/<script>/cache (worktrees only)
   • Note: check this case last, because the git folders in cases 3 and 4 usually aren't named .git either

2. Lock each cache entry while restoring or evicting it

Problem: Simultaneous wireit processes in worktrees share a common cache folder with one another, but they don't share script locks. That's good, since they don't need to block each other running. However, that does leave a new kind of race, where one process could be evicting a cache entry from the shared cache while another process is simultaneously restoring from it. That could cause a partial (and hence corrupt) restore, because the cache entry folder contents could have been partially deleted when the files were copied.

Fix: in shared mode, add a new lock for each cache entry, kept in a folder beside the shared cache folder, such as cache-locks/<entry>.

• To restore an entry, acquire its lock, then check if the entry folder still exists. If it's gone, it got evicted, so treat it as a cache miss.
• To evict entries, skip over any entry whose lock is held and evict the next oldest instead (since an entry whose lock is held is probably becoming more-recently-used from a restore).

3. Merge in atomic cache writes from #1516

Problem: I landed #1516 before getting to this one, which changes cache saving in a similar way to your #writeShared method, such that entry is first copied into a per-script temp/ folder, and is then renamed atomically. It also then moves anything else it finds in left in temp/ to a package-wide trash/ folder, assuming those are leftovers from aborted saves.

That's safe today because the script's lock guarantees no other save is in progress. But with the new shared mode, worktrees share that temp/ but not the lock, so one worktree can trash another worktree's save while it is in progress, which will cause a build failure.

Fix: I think the simplest is to say that in shared mode, we should only trash discovered leftover temp folders whose modification time is over an hour old. We could also have a lock for this, but the age thing sounds simpler. It's a rare case anyway, but good to fix.

(Also, #writeShared shouldn't be needed anymore, since cache writes are now always atomic. We can just make #1516’s save succeed in shared mode when the entry already exists.)