Skip to content

cacheqm: don't panic in evict() when globals exceed maxEntries - #3946

Open
0z5a wants to merge 1 commit into
google:masterfrom
0z5a:fix/cacheqm-evict-global-specs
Open

0z5a wants to merge 1 commit into
google:masterfrom
0z5a:fix/cacheqm-evict-global-specs

Conversation

@0z5a

@0z5a 0z5a commented Nov 14, 2026

Copy link
Copy Markdown

manager.evict() computes the number of entries to evict as len(m.cache) - m.maxEntries, which counts global specs. But the eviction candidates exclude globals, because they are never evicted. If the cache holds more global entries than maxEntries allows, evicts ends up larger than len(buckets) and the loop panics with index out of range. The panic happens in the goroutine started from GetTokens, so it takes down the server.

The quota interceptor requests {Global, Read} and {Global, Write} (both Refundable: true), so with --quota_max_cache_entries=1 (or --mysql_quota_max_cache_entries=1 with --mysql_quota_min_batch_size > 0) the log server crashes as soon as it has served one read and one write RPC.

This change caps the number of evictions at the number of non-global candidates. It also fixes the NewCachedManager error for an invalid maxEntries, which printed minBatchSize instead. Behaviour for other values is unchanged: the existing TestCachedManager_GetTokens_EvictsCache, which counts globals towards maxEntries, still passes.

Side note, not changed here: the NewCachedManager doc says maxEntries is the limit "apart from global quotas", while the flag help and the existing test treat it as the total number of cached specs. This PR keeps the current behaviour.

Fixes #3945

Testing:

  • New TestCachedManager_GetTokens_MaxEntriesBelowGlobalSpecs: panics (index out of range [1] with length 1) without the fix, passes with it.
  • go test -race ./quota/cacheqm/ and go test ./quota/... pass; golangci-lint run ./quota/cacheqm/... reports 0 issues.

Checklist

This change was drafted with Claude Code (claude-opus-5-5); the test results above are from the runs described.

evict() computes the number of entries to evict over the whole cache,
including global specs, but global specs are never eviction candidates.
When the cache holds more global specs than maxEntries (e.g.
--quota_max_cache_entries=1 after one read and one write RPC), the loop
indexes past the candidate list and panics in the goroutine started by
GetTokens, which takes down the server.

Cap the number of evictions at the number of candidates. Also report
maxEntries, not minBatchSize, in the NewCachedManager error message for
an invalid maxEntries.
@0z5a
0z5a requested a review from a team as a code owner November 14, 2026 18:50
@0z5a
0z5a requested a review from roger2hk November 14, 2026 18:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cacheqm: evict() panics when max cache entries is lower than the number of cached global specs

1 participant