Repository navigation
fix(agent-core-v2): backport prompt-cache compaction fix (#4456) - #4471
Conversation
) * fix(agent-core-v2): only pre-shrink compaction history when the request overflows the window The pre-shrink budget (~0.74 of the window) sat below the auto-compaction trigger (min(0.85W, W-50k)), so every auto compaction on windows above ~195k dropped the oldest messages and broke the prompt-cache prefix. Gate pre-shrinking on the real input limit (window minus output reserve) and keep the safety budget as the shrink target. * fix(agent-core-v2): pin degraded media to the count captured at the 413 Media degradation recomputed 'keep the 2 most recent' on every request, so each new image after a 413 replaced an older kept image and broke the prompt-cache prefix. Record how many leading media were degraded when the provider rejected the request, re-capture only when an inherited degraded request is rejected again with more than 2 visible media, and drop the prompt-cache guard's tolerance for rolling degradation. * feat(agent-core-v2): probe compaction prompt cache prefixes * chore: drop duplicate degraded media changeset * fix(agent-core-v2): report prefix loss after compaction shrinks history * fix(agent-core-v2): clamp compaction completion budget to the remaining window Sending the full history leaves less room than the configured output cap, so pass the estimated request size as usedContextTokens and let the provider clamp max_tokens to the window that is actually left. * Delete .changeset/fix-compaction-pre-shrink-cache.md Signed-off-by: 7Sageer <sag77r@hotmail.com> --------- Signed-off-by: 7Sageer <sag77r@hotmail.com> (cherry picked from commit 68a2838)
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ffd57c7fa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return media; | ||
| }; | ||
| const captureMediaDegradePolicy = (): { readonly degradeCount: number } => { | ||
| const media = { degradeCount: this.projector.captureMediaDegradeCount(shaped) }; |
There was a problem hiding this comment.
When a previous-release consumer reads an agent state snapshot after a request-size recovery, llmRequester.mediaRecovery.media now contains { degradeCount } instead of the documented 'degraded' literal, so clients matching the old variant will no longer recognize this state. Preserve compatibility or version/migrate the snapshot contract for those consumers.
AGENTS.md reference: AGENTS.md:L123-L125
Useful? React with 👍 / 👎.
| toolsHash: recorded.toolsHash, | ||
| systemPromptHash: recorded.systemPromptHash, | ||
| messageCount: input.messages.length, | ||
| historyHash: fingerprint(JSON.stringify(input.messages)), |
There was a problem hiding this comment.
When a turn carries large inline media (the resolver allows up to 20 MiB per request in AgentMediaResolverService), this JSON.stringify materializes another string for the entire provider-visible history on every successful turn step just to hash it. That adds large transient heap allocations and a full-history traversal to each such request, with spikes multiplying across concurrent agents; feed message segments to the hash incrementally instead of serializing the whole array at once.
Useful? React with 👍 / 👎.
| if (media === 'degraded') | ||
| return degradeOlderMediaParts(projected, MEDIA_DEGRADE_KEEP_RECENT, undefined, mediaPaths); | ||
| if ('degradeCount' in media) | ||
| return degradeLeadingMediaParts(projected, media.degradeCount, undefined, mediaPaths); |
There was a problem hiding this comment.
When a prior turn saved degradeCount: 8 after a 413, a manual or automatic compaction in the same model and context revision can pre-shrink that history to only three media parts; the inherited count is still applied here, replacing all three instead of keeping the most recent two. This initial inherited projection does not trigger the warning path, so compaction can silently summarize without media still present in its retained history. Recompute or clamp the count for pre-shrunk histories.
AGENTS.md reference: AGENTS.md:L131-L133
Useful? React with 👍 / 👎.
Requirement or Bug
Bug Reproduction Steps
See #4456.
Root Cause
See #4456。这是那笔 squash 的原样回移，不是新的修复。
Code Changes
补丁与
68a2838941逐字节一致，15 个文件，+614/-121。没有带入 #4457。不新增 changeset。落地的 squash 在合入前删掉了
.changeset/fix-compaction-pre-shrink-cache.md，净 diff 里没有 changeset；#4195、#4383 的回移同样不加。release 上已有的fix-compaction-prompt-cache-deferred-tools.md属于 #4410，2.4.0 说明里已经有了。合入这条 release 不会发版。发版仍要等 #4463 在这之后合入。
Behavior Changes and Affected Users
与 #4456 相同。
/compactmax_tokensmaxOutputSize（或默认min(W, 128k)、modelOverrides.maxCompletionTokens），不随输入收窄min(上述上限, W − 请求估算 − 2%W)；显式 opt-out（≤ 0）仍不发送prompt_cache_probesource=compaction；fork 行为不变source过滤的缓存命中率聚合会混入 compaction 样本source=fork/source=compaction分别聚合docs/state-manifest.d.ts的llmRequester.lastTurnRequestmessageCount/historyHash；不持久化消息，不新增 wire 字段受影响模块：
packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts、src/agent/contextProjector/、src/agent/llmRequester/、src/app/telemetry/events.ts，以及docs/state-manifest.d.ts。测试随 #4456 的提交一起回移，覆盖完整历史前缀、超窗预裁剪、输出上限钳制、媒体降级前缀和 compaction probe。Checklist
/approve).gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.