Skip to content

FIX: keep session recovery controls available after errors - #93

Merged
merefield merged 3 commits into
mainfrom
fix/session-error-recovery
Nov 4, 2026
Merged

merefield merged 3 commits into
mainfrom
fix/session-error-recovery

Conversation

@merefield

@merefield merefield commented Nov 4, 2026 •

Copy link
Copy Markdown
Owner

Why

A stopped Codex turn can report systemError after model capacity, availability or other failures. Codexometer treated that runtime as unknown and restricted settings choices to idle, hiding the follow-up composer and blocking /model precisely when the user needed recovery. A terminal error without a later completion event could also retain a consumed editor or obsolete active-turn controls.

Recovery in the shared Go adapter

  • Recognise systemError as a confirmed failed, non-running runtime, not unknown or WORKING. Preserve the ERRORS category, attention pills and observed explanation.
  • Offer ordinary follow-ups for connected, subscribed sessions in idle or systemError, and accept those same states in the fresh thread/read preflight before an explicit turn/start.
  • Allow supported model/reasoning and other idle-only settings choices, including directory recovery, in a stopped error state. Retain current command revision, advertised options, permissions confirmation, directory queue/verification and target-session checks. Existing active-turn speed behavior is unchanged.
  • Apply the shared behavior to terminal Sessions and opt-in writable browser controls. Local-only observations do not grant control, and browser read-only mode remains read-only.

Capability lifetime and safety

  • A definitive non-retrying turn failure clears its spent composer/active-turn capability even if no turn/completed event follows. Ended turns cannot receive stale steering/queue/interrupt requests.
  • Failed read-only preflights no longer permanently spend the composer: a later authoritative ready state can expose a fresh capability. Once turn/start is dispatched, uncertain writes remain consumed and are never automatically retried.
  • Bind cleanup and final preflight validation to the individual attempt, so an older in-flight read cannot send over, or release, a newer attempt. Newer non-ready runtime notifications also invalidate pending recovery.
  • Track the dispatch phase under the runtime mutex: invalidated preflights can recover when a later authoritative idle/systemError arrives, but the same notifications cannot revive uncertain dispatched writes.
  • Retain blocked states for running/retrying turns, unsupported or mixed input/approval requests, unknown/unloaded sessions, disconnected sources and stale capabilities. No automatic prompt retry, model switch or approval.

Interface and documentation

  • Keep the recovery composer and slash discovery usable on eligible failed sessions in full and wide terminal detail and writable browser detail.
  • Update browser error guidance so it does not imply the user must leave for Codex when supported recovery controls are available.
  • Explain recovery requirements and limits in README; regenerate embedded browser assets. No dependencies, release version or normal terminal layout changes.

Transport race found during CI

  • The macOS run exposed an existing decoder-close race: when the error branch drained a final buffered response, it discarded EOF/the decoder error and a later read could wait indefinitely. Preserve that terminal signal while still delivering buffered messages in order.
  • Add deterministic EOF and non-EOF regression coverage, and bound the previously unbounded read-loop test with a context. No assertions were skipped and no CI timeout was increased.

Validation

  • Full core Go suite with race detection and coverage (go test -race -cover -timeout 10m -skip '^TestLocalisedScreens$' ./...); the unchanged all-locale screen matrix remains covered separately by CI.
  • Go vet, gofmt and diff-whitespace checks.
  • Transport close/error-draining regressions passed 100 consecutive race-enabled runs on Linux and 100 native Windows runs. The deterministic test failed before the fix for both EOF and a non-EOF decoder error.
  • Synthetic app-server wire tests across model-capacity, unavailable-model, policy-capacity, network, authentication, quota, context-limit and generic failures, with and without completion events; explicit retry and scoped model/directory changes; one-use, stale-runtime, superseded-attempt and uncertain-write guards.
  • Terminal full/wide and browser control-route regression coverage for all four displayed failure categories; native Windows execution of the terminal recovery tests.
  • Frontend type/format checks and production build; all 106 browser tests passed, including four new stopped-error composer and /model discovery cases.

The official app-server runtime-status documentation distinguishes idle, systemError, notLoaded and active; this change does not treat arbitrary error labels or unknown states as permission to write.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A moderate active issue can leave SessionPrompt empty indefinitely after a preflight recovery is invalidated, preventing users from recovering.

1 open issue
What changed in this PR

This pull request makes confirmed systemError sessions recoverable through the shared Go adapter, terminal UI, and writable browser controls while preserving safeguards for active, unknown, disconnected, and uncertain states.

Changes:

  • Enables explicit follow-ups, supported settings changes, and directory recovery after stopped errors.
  • Clears stale turn capabilities and adds attempt-scoped preflight and concurrency protections.
  • Updates terminal/browser guidance, README documentation, regression tests, and embedded web assets.
  • Addresses a moderate capability-lifecycle issue before approval: a preflight-invalidated attempt can leave the composer permanently spent.

This pull request requires changes.

File Description
web/​tests/​browser.spec.ts Adds browser coverage for recovery composers and /model discovery after stopped errors.
web/​src/​Sessions.svelte Updates browser error guidance for supported recovery actions.
README.md Documents stopped-error recovery requirements and safety limits.
internal/​web/​session_failure_test.go Tests browser recovery offers, explicit follow-ups, replay protection, and command discovery.
internal/​web/​dist/​index.html References the regenerated browser JavaScript bundle.
internal/​web/​dist/​assets/​index-CoCF2Nd3.js Regenerated browser bundle containing updated error guidance.
internal/​ui/​monitor_failure_test.go Tests terminal recovery composers and slash-command discovery across detail modes.
internal/​codex/​session_turn_unix.go Prevents active-turn controls from surviving ended failures.
internal/​codex/​session_recovery_unix_test.go Adds recovery, preflight, stale-attempt, runtime, and settings tests.
internal/​codex/​session_prompt_unix.go Enables safe follow-ups for failed runtimes and scopes capability cleanup to attempts.
internal/​codex/​session_prompt_unix_test.go Extends prompt fixtures with preflight failures and race scenarios.
internal/​codex/​session_directory_unix.go Allows verified directory changes in stopped error states.
internal/​codex/​session_context_daemon.go Tracks prompt attempts and clears stale capabilities after definitive failures.
internal/​codex/​session_commands_unix.go Allows idle-only settings actions for stopped error runtimes.
internal/​codex/​daemon_status.go Recognizes systemError and defines accepted follow-up runtime states.
internal/​codex/​daemon_status_test.go Covers systemError parsing and follow-up eligibility.

Review effort: Lite

Comment thread internal/codex/session_prompt_unix.go Resolved
@merefield

Copy link
Copy Markdown
Owner Author

CI follow-up: the macOS timeout in TestAppServerReadLoopAndClose exposed an existing transport race, fixed separately in 45fbff9. If the error branch drained a final buffered response, it discarded EOF/the decoder error and the next read could block indefinitely. The fix retains that terminal signal while preserving message order. A deterministic regression reproduced both EOF and non-EOF loss before the fix; the transport tests now pass 100 race-enabled Linux runs and 100 native Windows runs. The original close test now has a bounded context, with all assertions retained and no increased CI timeout. Full core race/coverage tests and vet pass. The earlier review issue is fixed in 4a87795; waiting for fresh CI on the latest head before merging.

@merefield
merefield merged commit 30c8024 into main Nov 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants