Skip to content

fix(task): export structured output before guardrails run - #9533

Open
Harvey DU (Luckyou-d) wants to merge 3 commits into
crewAIInc:mainfrom
Luckyou-d:fix/task-guardrail-exports-structured-output
Open

Harvey DU (Luckyou-d) wants to merge 3 commits into
crewAIInc:mainfrom
Luckyou-d:fix/task-guardrail-exports-structured-output

Conversation

@Luckyou-d

Copy link
Copy Markdown

Summary

Task._execute_core and Task._aexecute_core skipped output conversion whenever a guardrail was configured (elif not self._guardrails and not self._guardrail: exported, else: left both None). task_output.pydantic was therefore None inside the guardrail on the first attempt, so a guardrail inspecting the structured model crashed with AttributeError, and it stayed None when the guardrail returned a TaskOutput instance.

Change

Both paths now export unconditionally. The guardrail machinery already re-exports when a guardrail transforms the raw string (_invoke_guardrail_function), so a guardrail that returns modified text keeps the configured format, and the final output is unchanged from what a guardrail-less task produces.

Verification

  • pytest lib/crewai/tests/test_task_guardrail_pydantic.py -> 2 passed; both fail on main (isinstance(None, Report) inside the guardrail) and pass with the change - one for guardrail=, one for guardrails=[...]
  • pytest lib/crewai/tests/test_task.py lib/crewai/tests/test_task_guardrails.py -> 108 passed; test_multiple_guardrails_with_pydantic_output needed a test-side tweak (the mock agent's converter now gets called and needs a ConverterError side effect), no production behaviour change there
  • ruff check and ruff format --check clean on all three files

Resolves #9495

`_execute_core` and `_aexecute_core` skipped output conversion whenever a task
had a guardrail, so `task_output.pydantic` was `None` inside the guardrail on
the first attempt - a guardrail that inspected the structured model (for
example `task_output.pydantic.score`) crashed with AttributeError. It also
stayed `None` when the guardrail returned a `TaskOutput`, because nothing
re-ran the export on that path.

Export unconditionally. The conversion result is attached to the TaskOutput
before validation, a guardrail that transforms the raw string re-exports via
the existing `_invoke_guardrail_function` path, and the final output keeps the
configured format.
@strix-security

strix-security Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 28 commits after the last Strix review (012e510). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Refactors `Task._execute_core` / `Task._aexecute_core` so the configured structured output (`output_pydantic` / `output_json`) is exported before guardrails run rather than only when no guardrail is configured, so a guardrail can inspect `task_output.pydantic` and `json_dict` on the first attempt; adds a regression test and adjusts an existing guardrail test's mocks.

Review summary

The change makes Task._execute_core and Task._aexecute_core export the configured structured output before guardrails run, instead of skipping conversion whenever a guardrail was configured, so a guardrail can inspect task_output.pydantic/.json_dict on the first attempt. I reviewed both hunks and traced their sinks: the export helpers and conversion fallbacks, the guardrail dispatch (which re-exports from the guardrail's transformed text or takes the guardrail's own TaskOutput), and the post-guardrail consumers (output_file write, callbacks, completed event, crew output). Nothing security-relevant was found: the populated object is the same agent answer the guardrail already received as raw, every post-guardrail consumer still sees a re-export of the approved text, and the change strictly tightens guardrails that previously aborted on None. No vulnerabilities found.

Ruled out (2)
  • Can a brace-heavy, non-JSON model answer stall the process through the output conversion that now runs before the guardrail?
    The candidate scan is genuinely super-linear, but it was already reached on the same agent answer before this change (the retained no-guardrail branch, the guardrail-returns-a-string re-export, and the guardrail retry export), and crewAI ships a library with no shared server or queue, so any stall…
  • Does a guardrail now judge a structured object that differs from the output actually delivered, letting unvalidated content through?
    Every in-tree and documented guardrail decides on `raw` text, a guardrail that returns modified text has the structured output rebuilt from that approved text, a guardrail returning a `TaskOutput` ships its own object, and retries already behaved this way before the change - the only residual effec…

Review 1 on this PR · 012e510 · requested Nov 18 07:44 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings · Try a deeper review with @strix-security /deep

# dropped when it returns a TaskOutput).
raw = result
pydantic_output, json_output = None, None
pydantic_output, json_output = self._export_output(result)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Agentic Security Review

Severity: HIGH

Description: With output_pydantic or output_json set, the first-attempt path now builds task_output.pydantic / json_dict before the guardrail runs. That object is a lossy projection of the model string: convert_to_model accepts the whole string only when it is valid schema JSON, and otherwise handle_partial_json keeps the first {...} object (DOTALL) or convert_with_instructions replaces the string with a schema-only LLM result. A custom guardrail that accepts on those structured fields and returns the raw string — the pattern in the new tests — therefore approves the projection while TaskOutput.raw stays the original model text. Crew._get_context then joins that raw string into the next task.

Impact: The task guardrail is the policy boundary on model output before it becomes context for later agents. A model result (or injected content that shaped it) can place a schema-valid object inside extra text. The guardrail sees only the object, returns the raw string, and the extra text is delivered as trusted context to the next task with no further check. Exact JSON and BaseModel results stay aligned, and LLMGuardrail still judges task_output.raw; the bypass is the structured-field accept path this change makes possible. No credentials are required beyond content that reaches task execution.

Remediation: Let the guardrail observe a structured object only when that object is a pure parse of the entire raw string. Leave pydantic and json_dict empty when parsing needs handle_partial_json or convert_with_instructions, and run those only after the guardrail accepts. If a guardrail approves a structured value, set the forwarded raw from that value (model_dump_json / json.dumps) so later tasks cannot receive text the projection dropped.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: TaskOutput.pydantic is None inside task guardrails when output_pydantic is configured

1 participant