fix: bound shell output and exclude runtime files from recursive search - #1603
Open
santoshkumarradha wants to merge 21 commits into
Open
santoshkumarradha wants to merge 21 commits into
santoshkumarradha wants to merge 21 commits into
Conversation
A background job's log used to grow without limit: every byte the job wrote went to one <id>.log forever, a single huge Write grew the in-memory ring by its whole length before trimming, and a failed or short or unclosable spool write was swallowed whole. A watcher that prints for a week filled the disk at whatever rate it printed. The spool is now a window: at most jobSpoolChunks chunks of jobSpoolChunkBytes (4MB) on disk — <id>.log live, <id>.log.1 kept — rotated by rename and never rewritten per write, the chunk beyond the window deleted and counted. One huge Write spools in chunk-sized pieces and hands only its newest 64KB to the ring, so no temporary grows to match it. A failed, short or failed-to-close spool write records one notice, stops the retries, and never stops the drain or kills the job. Every footer a model reads — jobs output and the completion note alike — stays honest about the bound: full log while the file is everything the job wrote, the truncation or the failure named beside the path where it is not. The manual's promises of the whole log, the jobs tool description and PERF.md move with it. Retained output stays addressable by the read tool exactly as before. Aggregate retention across jobs and search exclusion are deliberately not in this change. Issue #1599. Assisted-by: CodeAF Co-Authored-By: CodeAF <267109073+agentfield-bot@users.noreply.github.com>
santoshkumarradha
added
bug
santoshkumarradha
marked this pull request as ready for review
September 27, 2026 05:50
santoshkumarradha
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
A recursive search could read its own runtime output and fill the disk. Both shell output paths now have explicit storage limits: background jobs keep their latest two 4 MiB chunks, while foreground bash retains an initial 8 MiB snapshot alongside its latest in-memory result tail. Foreground snapshots include the initial bytes; incomplete or failed snapshots no longer claim to contain full output.
Completed managed job logs and foreground snapshots each have a 128 MiB / 64-file budget per directory with seven-day expiry. Independent file leases protect active writers across processes. Background job IDs remain durable across cleanup. Foreground snapshots use exclusive random names under the state home's
logs/bash/; promotion closes that snapshot before forwarding later output to the job sink. Logging failures remain visible while the child keeps draining.Structured recursive search excludes known runtime output, including custom state homes, foreground snapshots, input history and project task indexes, and legacy
pi-bash-*.logfiles in the current temporary directory. Source worktrees remain searchable; explicit file inspection is bounded. Arbitrary shell commands still need explicit search exclusions, and their captured output is bounded independently.Selected workspace/session folder aliases are resolved before opening owned log storage. Links inside that storage remain refused. This prevents the new retention checks from breaking jobs for users with symlinked home/workspace paths.
Fixes #1599
Fixes #1600
Fixes #1601
Fixes #1602
Fixes #1606
Validation
Actual default hosted tmux chat on final combined revision
830d4c2ebpassed all three corrected probes: structured search in 1.9 seconds returned only the source marker, foreground output completed in 1.7 seconds with an exactly 8,388,608-byte snapshot and a truthful limit notice, and a managed background job completed in 2.3 seconds with exit 0 and retained chunks of 4,194,304 and 3,097 bytes. All 13 completed calls useddeepseek/deepseek-v4.1-flashthrough OpenRouter, including auxiliary roles; no call errors occurred.The initial terminal search exposed prompt-history self-ingestion. The corrected exact-path exclusions were added and the final terminal probe above verified that runtime history no longer enters source results. Live test configuration now pins all 22 text roles from the canonical role vocabulary, every tier, and fallback settings.
On Spark, a controlled reproduction against the original PR revision
b22359d07wrote a 12,582,912-byte foreground spill. The fixed real-shell regression retains exactly 8 MiB and preserves the latest result tail with an explicit incomplete-snapshot notice.Both engines reproduced the prompt-history self-ingestion before the fix; the final history/task-index and foreground-spill exclusion regressions passed in 0.87 seconds.
Full
internal/exec/baresuite passed in 12.06 seconds. Focused race checks covering foreground spills, cross-process writer leases, retention, promotion, failures, and both search engines passed in 4.05 seconds.Focused background spool/retention suite passed in 14.27 seconds. Final regressions for selected-folder aliases, startup expiry, unsafe linked descendants, and an actual shell search consuming its own job log passed in 0.52 seconds.
make buildpassed at9dbd1c286. Manual, performance limits, and change entries describe both storage contracts and their limits.The local light gate passed build, vet, manual checks, changelog validation, all 30 law packages, and the affected bare/e2e package suites. The final two-shard session suite ran for 423 seconds and had one failing test:
TestAChildLandingAfterItsParentSettledReachesTheConversationtimed out while observing its transient steering queue. The same failure reproduced on the original PR revision (4/30 runs); diagnostic repeats proved all four sampled failures had already delivered the exact message (queued=false,durable=true). The assertion now reads the existing transcript-plus-queues helper and preserves its timeout, expected content, and settled-parent report check. The corrected test passed 100 repeats in 3.686 seconds at734d6eb3d. This full local gate remains recorded as failed. Fresh isolated CI at public head734d6eb3d0bbd9eb311ca70a6e3e42c9a2dfda80subsequently passed all checks, including touched packages: https://github.com/Agent-Field/CodeAF/actions/runs/36329255094.Boundaries
These are per-job/per-directory limits, not a machine-wide disk quota. Active writers and unmarked/unsafe legacy files are outside completed-file retention. Existing large legacy files are preserved. No production logs were deleted by this change.