You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
review: one-shot calls without a cache write - #248
Sends the review's intent, description and sweep calls (sweep2 too) with NoCache.
Why: each of these prompts is sent once, so its cache breakpoint only bought a 1.25x cache write that nothing read. In tagged reviews since 2026-11-13 these stages wrote 30.3M tokens in 3 days. Dropping the premium saves an estimated $200–230 a month. The prompt the model sees is unchanged.
Rollback:KAI_REVIEW_ONESHOT_CACHE=on puts the breakpoints back.
Before merging:go.mod pins kai-engine to the unreleased NoCache commit from the kai-engine provider-no-cache-oneshot branch. Bump it to the release first.
Written by Kai from the change itself. Kai only edits this section; everything above it is the author's.
Sets NoCache on the review's intent, description and sweep requests (sweep2 included), so these one-shot prompts no longer carry a cache breakpoint. KAI_REVIEW_ONESHOT_CACHE=on restores the old behaviour.
Changes
New cmd/kai/review_commit_oneshot_cache.go: adds rcOneShotNoCache, read once from KAI_REVIEW_ONESHOT_CACHE at startup. rcOneShotNoCacheFrom returns false only when the value is on (trimmed, case-insensitive). Any other value, including unset, returns true.
Intent and description calls:rcInferIntent (review_commit.go) and rcWriteDescription (review_commit_description.go) now pass NoCache: rcOneShotNoCache in their requests.
Sweep calls:rcRunSweepWith (review_commit_sweep.go) passes NoCache: rcOneShotNoCache for each chunk request. This covers both the sweep and sweep2 stages.
Dependencies:go.mod and go.sum move kai-engine from v0.6.87 to the pseudo-version v0.6.88-0.20261116103328-cbdfb4c0019b, which adds the NoCache request field. The author notes this is an unreleased commit and must be bumped to a release before merging.
Testing
TestOneShotCallsSendNoCacheBreakpoint: runs intent, description, sweep and sweep2 against an httptest server behind a provider.Kailab client. It checks that cache_control is absent from every request body when rcOneShotNoCache is true and present when it is false. The sweep case uses oversized patches so the sweep splits into more than one chunk, and the test asserts at least two requests were sent.
TestOneShotCacheSwitch: checks rcOneShotNoCacheFrom for "", "on", " ON ", "off" and "1".
Each of these prompts is sent once, so its cache breakpoint was billed as
a 1.25x cache write that nothing read (tagged reviews since 2026-11-13:
30.3M write tokens on these stages in 3 days, almost no reads). The
prompt the model sees is unchanged. KAI_REVIEW_ONESHOT_CACHE=on restores
the breakpoints.
Pins kai-engine to the unreleased NoCache commit; bump to the release
before merging.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR text and commit messages in the change description
Findings
go.mod:64 — go.mod requires kai-engine at the unreleased pseudo-version v0.6.88-0.20261116103328-cbdfb4c0019b from the provider-no-cache-oneshot branch, though the PR says to bump it to a release before merging. (also: go.sum:114, go.sum:115)
go.mod replaces the released v0.6.87 of kai-engine with an untagged pseudo-version taken from the provider-no-cache-oneshot feature branch, so the build depends on a commit that is on no release line. The PR description says this pin must be replaced with a release before merging, and that hasn't happened yet. go.sum carries the matching hashes for that commit.
Remedy: Cut a kai-engine release that contains NoCache, then bump go.mod to that version and update go.sum with its h1 and /go.mod hashes.
Size M · Risk M · +106 −1 · 7 files · reaches 84 · the full analysis · Reply to any of my comments and I'll answer, or say @kaicontext anywhere on this PR — a question, or "take another look at the retry logic". · Some repos review new pushes only when asked. Comment @kaicontext review any time for a review right away.
The reason will be displayed to describe this comment to others. Learn more.
kai-engine pinned to unreleased feature-branch commit
go.mod replaces the released v0.6.87 of kai-engine with an untagged pseudo-version taken from the provider-no-cache-oneshot feature branch, so the build depends on a commit that is on no release line. The PR description says this pin must be replaced with a release before merging, and that hasn't happened yet. go.sum carries the matching hashes for that commit.
When it happens: When this change is merged as it stands, with the pseudo-version still in go.mod.
Suggested fix: Cut a kai-engine release that contains NoCache, then bump go.mod to that version and update go.sum with its h1 and /go.mod hashes.
Prompt to fix with AI
Fix a defect that Kai's code review found in this change.
Where: go.mod:64
Problem: `go.mod` replaces the released `v0.6.87` of `kai-engine` with an untagged pseudo-version taken from the `provider-no-cache-oneshot` feature branch, so the build depends on a commit that is on no release line. The PR description says this pin must be replaced with a release before merging, and that hasn't happened yet. `go.sum` carries the matching hashes for that commit.
When it happens: When this change is merged as it stands, with the pseudo-version still in `go.mod`.
Suggested fix: Cut a `kai-engine` release that contains `NoCache`, then bump `go.mod` to that version and update `go.sum` with its h1 and `/go.mod` hashes.
This is a review report, not a set of instructions: read the evidence lines and confirm the problem is real before changing anything. If the code does not bear it out, stop and say why.
Keep the change to what fixes this defect. Add or update a test that fails without the fix and passes with it, and run the tests for the packages you touched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sends the review's intent, description and sweep calls (sweep2 too) with
NoCache.Why: each of these prompts is sent once, so its cache breakpoint only bought a 1.25x cache write that nothing read. In tagged reviews since 2026-11-13 these stages wrote 30.3M tokens in 3 days. Dropping the premium saves an estimated $200–230 a month. The prompt the model sees is unchanged.
Rollback:
KAI_REVIEW_ONESHOT_CACHE=onputs the breakpoints back.Before merging:
go.modpins kai-engine to the unreleasedNoCachecommit from the kai-engineprovider-no-cache-oneshotbranch. Bump it to the release first.🤖 Generated with Claude Code
Kai context
Written by Kai from the change itself. Kai only edits this section; everything above it is the author's.
Sets
NoCacheon the review's intent, description and sweep requests (sweep2 included), so these one-shot prompts no longer carry a cache breakpoint.KAI_REVIEW_ONESHOT_CACHE=onrestores the old behaviour.Changes
cmd/kai/review_commit_oneshot_cache.go: addsrcOneShotNoCache, read once fromKAI_REVIEW_ONESHOT_CACHEat startup.rcOneShotNoCacheFromreturns false only when the value ison(trimmed, case-insensitive). Any other value, including unset, returns true.rcInferIntent(review_commit.go) andrcWriteDescription(review_commit_description.go) now passNoCache: rcOneShotNoCachein their requests.rcRunSweepWith(review_commit_sweep.go) passesNoCache: rcOneShotNoCachefor each chunk request. This covers both the sweep and sweep2 stages.go.modandgo.summovekai-enginefrom v0.6.87 to the pseudo-versionv0.6.88-0.20261116103328-cbdfb4c0019b, which adds theNoCacherequest field. The author notes this is an unreleased commit and must be bumped to a release before merging.Testing
TestOneShotCallsSendNoCacheBreakpoint: runs intent, description, sweep and sweep2 against anhttptestserver behind aprovider.Kailabclient. It checks thatcache_controlis absent from every request body whenrcOneShotNoCacheis true and present when it is false. The sweep case uses oversized patches so the sweep splits into more than one chunk, and the test asserts at least two requests were sent.TestOneShotCacheSwitch: checksrcOneShotNoCacheFromfor"","on"," ON ","off"and"1".