Repository navigation
fix(task): export structured output before guardrails run - #9533
Harvey DU (Luckyou-d) wants to merge 3 commits into
Conversation
`_execute_core` and `_aexecute_core` skipped output conversion whenever a task had a guardrail, so `task_output.pydantic` was `None` inside the guardrail on the first attempt - a guardrail that inspected the structured model (for example `task_output.pydantic.score`) crashed with AttributeError. It also stayed `None` when the guardrail returned a `TaskOutput`, because nothing re-ran the export on that path. Export unconditionally. The conversion result is attached to the TaskOutput before validation, a guardrail that transforms the raw string re-exports via the existing `_invoke_guardrail_function` path, and the final output keeps the configured format.
Strix Security ReviewWarning This pull request has 28 commits after the last Strix review ( ✅ No open security findings. PR summary: Refactors `Task._execute_core` / `Task._aexecute_core` so the configured structured output (`output_pydantic` / `output_json`) is exported before guardrails run rather than only when no guardrail is configured, so a guardrail can inspect `task_output.pydantic` and `json_dict` on the first attempt; adds a regression test and adjusts an existing guardrail test's mocks. Review summaryThe change makes Ruled out (2)
Review 1 on this PR · Reviewed by Strix |
| # dropped when it returns a TaskOutput). | ||
| raw = result | ||
| pydantic_output, json_output = None, None | ||
| pydantic_output, json_output = self._export_output(result) |
There was a problem hiding this comment.
🔒 Agentic Security Review
Severity: HIGH
Description: With output_pydantic or output_json set, the first-attempt path now builds task_output.pydantic / json_dict before the guardrail runs. That object is a lossy projection of the model string: convert_to_model accepts the whole string only when it is valid schema JSON, and otherwise handle_partial_json keeps the first {...} object (DOTALL) or convert_with_instructions replaces the string with a schema-only LLM result. A custom guardrail that accepts on those structured fields and returns the raw string — the pattern in the new tests — therefore approves the projection while TaskOutput.raw stays the original model text. Crew._get_context then joins that raw string into the next task.
Impact: The task guardrail is the policy boundary on model output before it becomes context for later agents. A model result (or injected content that shaped it) can place a schema-valid object inside extra text. The guardrail sees only the object, returns the raw string, and the extra text is delivered as trusted context to the next task with no further check. Exact JSON and BaseModel results stay aligned, and LLMGuardrail still judges task_output.raw; the bypass is the structured-field accept path this change makes possible. No credentials are required beyond content that reaches task execution.
Remediation: Let the guardrail observe a structured object only when that object is a pure parse of the entire raw string. Leave pydantic and json_dict empty when parsing needs handle_partial_json or convert_with_instructions, and run those only after the guardrail accepts. If a guardrail approves a structured value, set the forwarded raw from that value (model_dump_json / json.dumps) so later tasks cannot receive text the projection dropped.
Summary
Task._execute_coreandTask._aexecute_coreskipped output conversion whenever a guardrail was configured (elif not self._guardrails and not self._guardrail:exported,else:left bothNone).task_output.pydanticwas thereforeNoneinside the guardrail on the first attempt, so a guardrail inspecting the structured model crashed withAttributeError, and it stayedNonewhen the guardrail returned aTaskOutputinstance.Change
Both paths now export unconditionally. The guardrail machinery already re-exports when a guardrail transforms the raw string (
_invoke_guardrail_function), so a guardrail that returns modified text keeps the configured format, and the final output is unchanged from what a guardrail-less task produces.Verification
pytest lib/crewai/tests/test_task_guardrail_pydantic.py-> 2 passed; both fail onmain(isinstance(None, Report)inside the guardrail) and pass with the change - one forguardrail=, one forguardrails=[...]pytest lib/crewai/tests/test_task.py lib/crewai/tests/test_task_guardrails.py-> 108 passed;test_multiple_guardrails_with_pydantic_outputneeded a test-side tweak (the mock agent's converter now gets called and needs aConverterErrorside effect), no production behaviour change thereruff checkandruff format --checkclean on all three filesResolves #9495