Reuse compiler signatures and runtime CMIs in OCaml rewatch

[fhammerschmidt]

Summary

• Reuse an expanded WebAPI signature graph on each OCaml rewatch compiler domain after verifying that the previous request left it clean. Restore a saved snapshot when the graph changed, and invalidate the entry when its CMIs change.
• Delay snapshot preparation until two distinct requests expand the same alias, avoiding the setup cost on small builds.
• Reuse decoded Stdlib and Pervasives interfaces within each worker, with load-path and file checks before each hit. Share one immutable prepared signature image across workers while keeping mutable graphs private to each domain.
• Add tests for sequential reuse, cross-domain isolation, dirty-graph recovery, load-path changes, and CMI invalidation; document the cache experiments and measured tradeoffs.

Performance

The initial per-worker expanded signature cache reduced median clean-build time from 1.45 s to 1.27 s in six interleaved eight-worker pairs on the isolated testrepo fixture, while median peak RSS rose from 349 MiB to 523 MiB.

In a later eleven-pair release-profile comparison against that initial cache, the runtime CMI and shared-image changes reduced median clean-build time from 1,199 ms to 1,014 ms (15%). Median sampled peak process-tree RSS fell from 545,312 KiB to 457,660 KiB. Both builds made identical compiler requests, and generated artifact bytes matched. These figures are specific to the test fixture and host.

Stack

Depends on #8672. This PR is the next performance experiment above the compiler tracing work.

Validation

• make checkformat
• make test
• make test-rewatch

[fhammerschmidt]

The implemented change caches the expanded WebAPI signature in memory for each Rewatch compiler worker. A worker reuses it across requests after checking that the source CMI files are unchanged and the previous request left the cached graph clean. It prepares the cache only when multiple requests need the signature.

In the benchmark, the eight-worker clean build became about 12% faster (1.45 → 1.27 seconds), while peak memory rose from 349 → 523 MiB. Small incremental builds stayed around 0.09 seconds, and the compared outputs were identical.

[codecov]

Codecov Report

❌ Patch coverage is 22.13115% with 380 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.27%. Comparing base (26ec8df) to head (5d68298).

Files with missing lines	Patch %	Lines
compiler/ml/env.ml	20.65%	361 Missing ⚠️
compiler/ml/btype.ml	16.66%	10 Missing ⚠️
compiler/ext/ident.ml	46.66%	8 Missing ⚠️
compiler/ext/compiler_phase_trace.ml	66.66%	1 Missing ⚠️

Additional details and impacted files

@@                        Coverage Diff                        @@
##           faster-rewatch-ocaml-followup    #8673      +/-   ##
=================================================================
- Coverage                          78.69%   78.27%   -0.43%     
=================================================================
  Files                                481      481              
  Lines                              65183    65663     +480     
=================================================================
+ Hits                               51296    51396     +100     
- Misses                             13887    14267     +380     

Files with missing lines	Coverage Δ
compiler/bsc/rescript_compiler_driver.ml	75.64% <100.00%> (+0.15%)	⬆️
compiler/ml/types.ml	69.23% <ø> (ø)
compiler/ext/compiler_phase_trace.ml	21.56% <66.66%> (+2.81%)	⬆️
compiler/ext/ident.ml	82.67% <46.66%> (-4.94%)	⬇️
compiler/ml/btype.ml	87.17% <16.66%> (-2.29%)	⬇️
compiler/ml/env.ml	59.16% <20.65%> (-18.02%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[fhammerschmidt]

After the last commit (5d68298) the eight-worker clean-build median fell from 1,199 ms to 1,014 ms—185 ms faster, or 15%. All 11 paired runs were faster. Unchanged and single-edit builds were effectively unchanged (about 44–45 ms).

[fhammerschmidt]

Superseded by consolidated PR #8680, which preserves this work as one commit in the combined review branch.