[zhongkechen]

A callback can complete while a durable function is suspended, and replay may never visit it. This change preserves telemetry for visible terminal operations using the existing plugin snapshots and hooks.

Changes

• Keep completion delivery inside OTel. Each invocation's plugin retains terminal input/checkpoint updates, exports them through natural operation hooks or parent registration, and drains unvisited completions before the invocation span ends. An OTel export ledger suppresses duplicate observations while allowing each newly opened replay/virtual context segment to complete. Core no longer adds a completion scheduler, ancestor scans, or extra operation-map copies.
• Preserve replay parents and recover missing parentage. Pending children use the actual replay segment before it closes. Completed context references remain available for later updates. At invocation End, an unavailable parent uses a logged fallback: the invocation span in Invocation view, or a derived logical parent context in Execution view. This does not manufacture missing operation records or reconstruct an unavailable virtual parent's original span.
• Retry failed completion exports. In-progress claims are separate from the successful-export ledger. Failed starts/ends remain pending and can retry through a later notification or invocation End. Retries preserve the span ID, parent, original timestamps and exception events; a failed context segment is completed before another segment opens. During invocation End, failures are isolated per operation with at most one ordinary retry. Other completions and Invocation/Workflow finalization continue, and errors are reported only after cleanup; control exceptions are not retried.
• Isolate operation errors. OperationInfo.from_operation constructs the requested info subclass and copies mutable error stack traces. Terminal hooks use this conversion, so plugin mutations cannot alter checkpointed errors.
• Report terminal outcomes consistently. FAILED, TIMED_OUT and STOPPED map to ERROR even without diagnostic details; exception events use the normalized span-end timestamp.
• Bound flush waiting per provider. One outstanding attempt per provider, with a 30-second caller budget. A blocked provider does not prevent another provider from flushing. Later callers on the same provider perform a fresh flush after admission. Lifecycle hooks and context cleanup remain on their owner.

Stack and scope

main → #758 → #795 → #796

The testing-library implementation is unchanged here, and installed-wheel checks use the published testing library. #795 carries replay visibility and successful-input acknowledgment fixes, local testing-wheel installation, and the CI filters needed by the remaining stack. All five workflow PR filters in #758 target only main. #796 adds the conformance handlers and qualification coverage.

Validation and limits

Validation on Python 3.13.17 passes:

• 875 installed OTel tests against published testing-library 1.3.0, 12 lifecycle compatibility cases, and four released-OTel registration rejection cases for fix(otel): preserve replay completion telemetry #758.
• 906 installed OTel tests, 16 lifecycle/callback cases and four legacy rejection cases for the complete stack with its built testing library.
• 112 workflow/conformance-package checks, plus type, lint, formatting, artifact hash/source verification and package builds.

The stack retains linear history. The additional #758 fix is carried into #795/#796 by rebasing their unchanged layer contents; independent tree checks verify those boundaries. Commit-message validation passes, with the accepted documentation suggestion and its coauthor attribution preserved. GitHub CI runs separately on the new pushed heads.

A processor can deliver a span and then raise, so retrying partial delivery can export it again. Deduplication is invocation-scoped; failed delivery can still be exported again after redelivery because durable state has no exporter acknowledgment. Shared conformance #131 remains a merge prerequisite. The separate virtual-FLAT replay persistence issue JS #997 remains outside this change.

[github-actions]

Codex AI review · Finding arf_v1_wh6we4ebru4v7r5faoxepdlqa5

[P2] Preserve non-retryable control failures through cleanup

If a processor raises SystemExit or KeyboardInterrupt during a virtual child's terminal hook, this re-raises while leaving the completion pending. Core subsequently invokes invocation End during exception cleanup. Its fresh InvocationFinalization has no record of that control failure, so it recreates the span and calls the processor again, potentially repeating partial delivery despite the stop signal. Invocation view has the same path. Retain non-retryable failures for the whole invocation and exclude them from subsequent drains. Add a public virtual-child regression asserting one processor attempt and propagation of the original exception.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

[github-actions]

Codex AI review · Finding arf_v1_opkwyjxdk23t3ijca65noht5av

[P2] Isolate deferred export failures from parent scope setup

This drain runs inside on_user_function_start, before _attach_context. When replay registers a parent whose pending callback export raises, _render_operation_end propagates the exception because _finalization is still None. Core contains it and runs the parent's body without attaching its OTel context, so user-created spans and correlated logs acquire the enclosing parent instead. Later retries cannot repair that ancestry. Contain ordinary failures per pending completion while preserving retry state and allowing parent setup to finish. Cover this with a failing-processor replay test; Execution view's registration drain needs the same protection.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

[github-actions]

Codex AI review · Finding arf_v1_tkwzf5lq7oyzoym3afwwuxplhb

[P2] Materialize pending parents before installing fallback references

When the full input snapshot contains updated terminal context P, terminal child C, and an unavailable ancestor of P, this installs a DurableParentSpan for P before processing P's deferred completion. _finish_operation_end subsequently treats the placeholder as an existing span: its end() does nothing, but P enters _ended_operation_ids without being exported. Process pending ancestor completions before installing fallback-only references. Extend the nested-fallback regression with the full operations snapshot populated; its empty snapshot currently bypasses this path.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

[github-actions]

Codex AI review · Finding arf_v1_544qkisfhmjei34ezfmfd3d24g

[P3] Return the new flush mechanism to the design phase

Rules 2, “Simplify before you add,” and 3, “Write a design note before the code,” require alternatives and their rejection reasons before adding this feature. The description summarizes daemon workers, provider admission, and a fixed deadline, but does not explain the separate flushing problem and cause or evaluate simpler options. Return to the design phase under rule 8, step 6, and obtain agreement on a note comparing provider-managed timeouts with this additional lifecycle mechanism before continuing implementation.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

[github-actions]

Codex AI review · Finding arf_v1_n7tcowjse2q5cim5i5ayi5css2

[P3] Reconsider the untested cyclic-parent recovery

This default adds a recovery path for cyclic parent graphs, duplicated in Execution view, but none of the tests supplies a cyclic graph. The missing-ancestor tests remain acyclic and do not require this fallback. That violates rule 4, “Every branch has a test that fails without it.” Return to the design phase under rule 8, step 6, and establish whether cyclic metadata is a supported input at all; relying on the operation hierarchy's tree structure would be simpler than adding untested recovery behavior.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

[github-actions]

Codex AI review · Finding arf_v1_yvpbx24lqmo75xwa5nxeobikxa

[P3] Document the changed test contracts before proceeding

Rule 5.4 requires edited test expectations and their reasons to be listed in the PR description. This changes test_invocation_is_released_when_span_shutdown_fails from expecting RuntimeError to expecting ExceptionGroup; status-mapping and worker-context expectations also change elsewhere, but the description does not identify those tests. These are changes to observable contracts, not merely additional coverage. Return to the design phase under rule 8, step 6, and record the intended contract and rationale for each changed expectation.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

[github-actions]

Codex AI review

Four correctness findings and three approach findings require changes. Locations: packages/aws-durable-execution-sdk-python-otel/src/aws_durable_execution_sdk_python_otel/execution_plugin.py:619 and :986; packages/aws-durable-execution-sdk-python-otel/src/aws_durable_execution_sdk_python_otel/invocation_plugin.py:593, :930, and :959; packages/aws-durable-execution-sdk-python-otel/src/aws_durable_execution_sdk_python_otel/_flush.py:70; packages/aws-durable-execution-sdk-python-otel/tests/test_log_filter.py:666.

Tests were not run under the read-only review constraints.

Suppressed repeats

• arf_v1_sbn32537tiw6bn2pupqpzjw64o — deferred (maintainer verdict)

Reviewed commit 0d407e1152c6fbd3f083588c5919a15902093096. Workflow run

[zhongkechen]

Closing this PR in favor of smaller, independently scoped issues:

• [Bug]: Isolate terminal plugin error snapshots from checkpointed state #852 — Isolate terminal plugin error snapshots from checkpointed state.
• [OTel]: Report terminal failures and keep exception events within span bounds #853 — Report terminal failures and keep exception events within span bounds.
• [OTel]: Export unvisited replay completions with the correct parent context #854 — Export unvisited replay completions with the correct parent context.
• [OTel]: Recover failed completion exports and isolate invocation finalization errors #855 — Recover failed completion exports and isolate invocation finalization errors.
• [OTel]: Bound provider flush waiting without blocking unrelated providers #856 — Bound provider flush waiting without blocking unrelated providers.

Each issue includes implementation guidance, reproduction scenarios, acceptance criteria, code and test references, relevant review findings, and scope boundaries. Please use those issues for focused implementation PRs; this PR remains a reference for the original implementation and review discussion.