Skip to content

fix(flow): revive a method's Pydantic output after a checkpoint restore - #9536

Open
Harvey DU (Luckyou-d) wants to merge 3 commits into
crewAIInc:mainfrom
Luckyou-d:fix/flow-checkpoint-revive-pydantic-output
Open

Harvey DU (Luckyou-d) wants to merge 3 commits into
crewAIInc:mainfrom
Luckyou-d:fix/flow-checkpoint-revive-pydantic-output

Conversation

@Luckyou-d

Copy link
Copy Markdown

Summary

Checkpoints serialise _method_outputs with model_dump_json(), so a method that returned a BaseModel comes back as a plain dict. A listener that reads model attributes then raises on the resumed run: charge(quote) fails on quote.amount, and a @human_feedback listener loses result.feedback. The run that was restored to recover from a failure fails again, permanently.

Reproduced on main (435262c) with the issue's setup (CheckpointConfig(location=..., on_events=["method_execution_finished"])):

first run  -> charge receives Quote
restored   -> charge receives dict  -> AttributeError: 'dict' object has no attribute 'amount'

The checkpoint file stores {"method": "price", "output": {"amount": 42}}.

Change

_revive_method_output() re-validates a restored output against the method's annotated return type. Two details that mattered:

  • the annotation is resolved on the wrapped function (method._meth), because StartMethod/ListenMethod replace __annotations__ with their own generic ones, so get_type_hints(wrapper) returns {};
  • anything that does not fit is kept exactly as restored - a dict that really is a dict, a method this instance does not know, or a value that fails validation - so nothing that used to work changes.

Applied to both the checkpoint (_restore_from_checkpoint) and from_pending restore paths.

Verification

  • New test_checkpoint_restore_revives_pydantic_listener_inputs in tests/test_flow_resumability_regression.py: fails on main (received == ["Quote", "dict"]), passes with the change (["Quote", "Quote"]), and the flow reaches state["charged"] == 42
  • pytest tests/test_flow.py tests/test_flow_persistence.py tests/test_flow_resumability_regression.py tests/test_flow_human_input_integration.py tests/test_flow_ask.py -> 167 passed, 1 failed; that failure is test_persist_complex_types_dict_state, which fails identically on main in this environment (a Windows Path separator assertion), unrelated to this change
  • ruff check and ruff format --check clean on both files

Fixes #8974

Checkpoints serialise `_method_outputs` with `model_dump_json()`, so a method
that returned a BaseModel comes back as a plain dict. A listener that reads
model attributes then raises: `charge(quote)` fails on `quote.amount`, and a
`@human_feedback` listener loses `result.feedback` on the resumed run, so the
run that was restored to recover from a failure fails again permanently.

Re-validate each restored output against the method's annotated return type,
resolving the annotation on the wrapped function because the flow-method
wrappers carry their own generic annotations. Outputs that do not fit (a dict
that really is a dict, a method this instance does not know, a value that fails
validation) are kept exactly as restored. Applied to both the checkpoint and
`from_pending` restore paths.

Fixes crewAIInc#8974
@strix-security

strix-security Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 28 commits after the last Strix review (5e35335). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: On a resumed run, a method's Pydantic return value came back as a plain dict from the checkpoint, so listeners reading attributes crashed; the PR re-validates restored method outputs against the method's annotated return type on both the checkpoint and pending-feedback restore paths, and adds a regression test.

Review summary

The change is confined to the flow checkpoint restore path: _revive_method_output() re-validates a restored method output against the method's annotated return type when that annotation is a concrete Pydantic model, and it is applied on both the checkpoint and from_pending restore paths, with a regression test alongside it. I reviewed both restore call sites, the annotation lookup, the in-place replacement of entry["output"], and what happens to the revived value downstream. No security issue was confirmed. The only behavioural regression found is non-security: because only ValidationError is caught around model_validate, a flow author's model that raises a different exception type (or still has unresolved forward refs) now aborts a restore that previously succeeded with the raw dict, and that failure escapes outside the error handling that emits the checkpoint-restore-failed event — worth widening the catch, but reachable only by the author of the model or the operator running the restore.

Ruled out (3)
  • Can a tampered checkpoint or pending-feedback record turn a restored dict into a validated Pydantic instance that then passes type gates in listeners?
    Only the party who can write the checkpoint blob or pending-feedback row can supply method outputs, and that same document already carries more powerful pre-existing fields (checkpoint state, completed-method sets) on the identical restore path, so the revive adds no new capability.
  • Does passing a model instance instead of a dict let an action, expression or inline-script consumer skip a trust gate or receive content it could not otherwise…
    Expression and action consumers serialise outputs through model_dump, so the JSON shape is identical to the old dict, the inline-script gate is decided at build time from an environment variable regardless of the value's type, and a non-restored run already hands listeners the same live model.
  • Can the new validation step abort a checkpoint restore that previously succeeded, in a way that harms another user or the service?
    Whether a non-ValidationError escapes depends only on the flow author's own return model, the affected values come from a store the operator configures, and the worst case reproduces a restore failure for the party who owns both the model and the checkpoint — a robustness and observability regressi…

Review 1 on this PR · 5e35335 · requested Nov 18 08:56 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings · Try a deeper review with @strix-security /deep

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] After a checkpoint restore, a listener receives its trigger's Pydantic output as a dict, so @human_feedback listeners lose .feedback

1 participant