Skip to content

feat(hooks): let a tool hook opt into failing closed - #9465

Open
Harvey DU (Luckyou-d) wants to merge 10 commits into
crewAIInc:mainfrom
Luckyou-d:feat/hook-error-policy-abort
Open

Harvey DU (Luckyou-d) wants to merge 10 commits into
crewAIInc:mainfrom
Luckyou-d:feat/hook-error-policy-abort

Conversation

@Luckyou-d

Copy link
Copy Markdown

Summary

Tool hooks are fail-open by design: dispatch swallows every exception except HookAborted, which protects the framework from a buggy user hook. For a hook used as a security control that is the wrong default - a policy hook whose service times out, or a redaction hook that crashes, silently stops protecting anything, and only verbose mentions it.

Change

register_before_tool_call_hook and register_after_tool_call_hook take an on_error keyword:

register_before_tool_call_hook(policy, on_error="abort")
register_after_tool_call_hook(redact, on_error="abort")
  • "fail_open" (default) keeps the current behaviour exactly.
  • "abort" turns any exception other than HookAborted into HookAborted with the original exception as its cause and the hook as its source. A before hook therefore blocks the tool call, and a failing after hook cannot let an unredacted result reach the model. A hook that raises HookAborted itself keeps its own reason.

The policy lives on the dispatch engine (HookErrorPolicy + set_error_policy) and is keyed by the hook callable rather than wrapping it, so nothing new is inserted into the global hook list that legacy modules alias, and unregister_* / clear_* drop the policy together with the hook.

Verification

pytest lib/crewai/tests/hooks/test_hook_error_policy.py -> 6 passed. Five of them fail on main (the keyword does not exist yet) and the sixth pins that the default is still fail-open.

The last test is the reproduction from the issue, driven through execute_tool_and_check_finality: with on_error="abort" the tool body does not run and the agent receives Tool execution blocked by hook. Tool: send_email.

  • pytest lib/crewai/tests/hooks -> 206 passed, 1 skipped, 1 failed; the failure (test_acall_runs_after_llm_call_hooks) fails identically on main without this change
  • ruff check and ruff format --check clean on all four files

Refs #9434

Tool hooks are fail-open: dispatch swallows every exception except
`HookAborted`, so a hook used as a security control that cannot reach a
decision lets the call through. A policy hook whose service times out, or a
redaction hook that crashes, silently stops protecting anything, and only
`verbose` mentions it.

`register_before_tool_call_hook` and `register_after_tool_call_hook` take an
`on_error` keyword. The default keeps the historical fail-open behaviour;
`on_error="abort"` turns any other exception into `HookAborted` with the
original exception as the cause, so the before hook blocks the tool call and a
failing after hook cannot let an unredacted result reach the model.

The policy lives on the dispatch engine (`HookErrorPolicy`, `set_error_policy`)
and is keyed by the hook callable, so no wrapper is inserted into the registry
that legacy modules alias, and unregistering or clearing a hook drops its
policy with it.

Refs crewAIInc#9434
@strix-security

strix-security Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 53 commits after the last Strix review (90a26e2). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Adds an on_error keyword to register_before_tool_call_hook and register_after_tool_call_hook so a hook can opt into failing closed: any exception other than HookAborted becomes HookAborted, blocking the tool call or preventing an unredacted result from reaching the model. The policy is held in a new dispatcher-level mapping keyed by the hook callable, cleaned up by unregister/clear, with tests for both policies.

Review summary

Reviewed the new per-hook on_error policy for tool hooks: HookErrorPolicy/set_error_policy/_error_policy in hooks/dispatch.py, the _invoke_hook abort branch, the two register functions in hooks/tool_hooks.py, the export, and the added tests. The fail-closed path was traced end to end through every dispatch seam and every tool-execution caller — before-hook aborts skip the tool body, and an aborted after hook propagates as HookAborted from execute_tool_and_check_finality, execute_single_native_tool_call and the agent executors before any result is written to messages, steps or memory, so the unredacted tool result does not reach the model. The roundtrip is also closed (dispatch looks the policy up on the exact object registration stored, including bound methods, LLM snapshot lists and execution-boundary points), and the converted exception's text is never placed in model context. No security vulnerability was confirmed. The issues that surfaced are API ergonomics and lifecycle consistency: per-callable policy keying lets a second registration with the default reset an existing opt-in, and the legacy tool-hook unregister/clear helpers bypass dispatch so policies are not dropped there — all requiring in-process application code that already owns the hooks, with no untrusted input involved and no capability gained.

Ruled out (5)
  • Can a hook registered with on_error=abort silently revert to fail-open when the same callable is registered again with the default, or when a sibling registrat…
    The mechanism is real (the policy is keyed by the callable and a fail_open write pops it), but only in-process application code can trigger it, and any principal able to re-register the guard can already unregister it outright, so no attacker or second-party victim exists.
  • Is on_error=abort a silent no-op for hooks registered through the filtered @​on / @​before_tool_call(tools=[...]) path, leaving a guard fail-open?
    Filtered registration stores a generated wrapper, but the decorators and CrewBase reject on_error entirely, so fail-closed cannot be requested there, while the documented register_*_tool_call_hook(on_error="abort") path keys and looks up the same object and blocks correctly — a feature-completeness…
  • Do stale policy entries survive the tool-hook clear/unregister helpers and bleed into later execution contexts or unrelated hooks?
    Reproduced: entries do survive those legacy helpers, but ABORT is the only value ever stored, so inheritance can only make a later hook stricter (fail-closed), never weaken a control, and no untrusted input reaches the mapping.
  • Can the new policy bookkeeping crash the hook reset API or break previously valid registration for unhashable callables?
    The crash is real for callable objects without __hash__ but needs a non-empty policy dict and in-process registration code, and every failure mode is fail-closed (hook not appended, or hooks left installed), with dispatch-time behaviour unchanged.
  • Does a hook that carries an abort policy become unexpectedly fail-closed at other interception points it is registered for?
    Keying by callable does let a policy follow the callable to another point, but the direction only removes operations that would otherwise proceed, only the developer who set the policy can create the state, and the harm lands on the operator's own run rather than another party.

Review 1 on this PR · 90a26e2 · requested Nov 15 09:44 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings · Try a deeper review with @strix-security /deep

Comment thread lib/crewai/src/crewai/hooks/dispatch.py Resolved
A filtered hook (`@before_tool_call(tools=...)`, `@on(..., agents=...)`, a
`@CrewBase` filter) is registered as a wrapper while the caller keeps the
original function, so `set_error_policy(func, "abort")` attached the policy to a
callable dispatch never sees. `_error_policy` read only the invoked wrapper,
missed, and fell back to fail-open: a filtered approval or redaction hook set to
fail closed still let the operation through.

Walk the wrapper chain (`__wrapped__`, and `_registered_hook` for `@on`) so a
policy set on either the original or the registered callable is honoured, and
drop the whole chain when the hook is unregistered or cleared.
`set_error_policy` is exported so the two public APIs match.
@Luckyou-d

Copy link
Copy Markdown
Author

Valid catch, and it was reachable from the public API — thanks. Fixed in 03e1a9a.

A filtered hook is registered as a wrapper (@before_tool_call(tools=...) and @on(..., agents=...) both append filtered/filtered_hook) while the caller keeps the original function, so set_error_policy(policy, "abort") attached the policy to a callable dispatch never invoked.

_error_policy now walks the wrapper chain — __wrapped__ (set by functools.wraps) and _registered_hook (set by @on) — so a policy set on either the original or the registered callable is honoured. unregister/clear drop the whole chain, so a re-registered hook cannot inherit a stale policy. set_error_policy is also exported now, so the two public APIs match.

Reproduced before the fix: with @before_tool_call(tools=["send_email"]) plus set_error_policy(policy, "abort"), a raising hook let the call through (run_before_tool_call_hooks returned False). After: it blocks. Four tests cover the filtered decorator, @on, a filter that does not match (the wrapper still skips first), and unregister-then-re-register — two of them fail without this commit.

Hooks suite: 210 passed, 1 skipped, 1 failed; that failure is test_acall_runs_after_llm_call_hooks, which fails identically on main.

Comment thread lib/crewai/src/crewai/hooks/dispatch.py Resolved
A `@CrewBase` hook method is registered as a bound method, while the decorator
returns - and the docs tell callers to pass - the plain function. `set_error_policy`
on that function therefore missed, because the bound method is a different dict
key, and the miss returns fail-open: the guard still let the operation through.

Walk `__func__` as well (`__wrapped__` and `_registered_hook` are kept), and link
the crew-base filter closures to the method they wrap with `functools.wraps`, so
a filtered crew method resolves too.

Tests cover a policy set on the class function, on the bound method, and on a
filtered crew method.
`@on` sets `_registered_hook` only for plain functions, so a method is linked
back through the filter wrapper's `__wrapped__` and the bound method's
`__func__`. The test pins that path; it fails without the `__func__` walk.
@Luckyou-d

Copy link
Copy Markdown
Author

Valid again, and reachable through the public API - thanks. Fixed in f0333a8 (+25895b5 for the @on-method test).

Confirmed the first scenario before the fix: with register_before_tool_call_hook(instance.guard) and set_error_policy(Crew.guard, "abort"), run_before_tool_call_hooks returned False (the tool ran); now it returns True.

Three changes:

  1. _policy_candidates also walks __func__, so a policy set on the plain function a @CrewBase decorator returns resolves to the bound method dispatch invokes.
  2. The crew-base filter closures (make_filtered_before_tool and the three siblings) now use functools.wraps(bound_fn), so a filtered crew method keeps the link back to the method it wraps. That also gives telemetry and the swallowed-error message the real hook name instead of filtered.
  3. _pop_error_policy only pops hashable candidates, since the walk now visits objects that may not be keys.

On the last point: per-executor LLM hook lists snapshot the callables, not the policy, and _error_policy reads the dispatcher-level map at invoke time by callable identity, so a snapshot of a hook registered with on_error="abort" still resolves. Worth a maintainer's eye if that path changes, but there is no copy of the policy to go stale today.

Tests: 14 in test_hook_error_policy.py, 7 of which fail on main; the three crew-method cases fail without f0333a8. Hooks suite: 213 passed, 1 skipped, 1 failed (test_acall_runs_after_llm_call_hooks, fails identically on main).

`register_before_tool_call_hook` and `register_after_tool_call_hook` always
wrote the policy map before appending, and the default (`on_error="fail_open"`)
was implemented as `_error_policies.pop(hook)`. Two reachable wipes followed:

- `set_error_policy(hook, "abort")` and then `register_*_tool_call_hook(hook)`
  with the default dropped the abort before the hook ever ran, although the
  docstring says the callable "was or will be registered";
- re-registering an already-registered callable with the default removed the
  abort the earlier registration stored, while the hook stayed on the list.

The default no longer touches the map: only an explicit `abort` writes a policy.
Removal goes through `clear_error_policy` (the public counterpart of the
internal walk), which `unregister_before/after_tool_call_hook` now call, so
unregistering a hook still drops its policy.

Two tests pin the wipes; both fail without this commit.
@Luckyou-d

Copy link
Copy Markdown
Author

Valid again - both wipes reproduce from the public API. Fixed in cad7796.

Reproduced before the fix: set_error_policy(policy, 'abort') then register_before_tool_call_hook(policy) (the default pop wiped it), and run_before_tool_call_hooks returned False, so the tool ran.

The fix:

  1. The default no longer touches the map at all - register_*_tool_call_hook only writes when on_error='abort' is explicit. The docstring's 'was or will be registered' now holds in both directions: set_error_policy(abort) survives a default re-register, and re-registering a live hook with the default no longer deletes the abort the earlier call stored.
  2. Removal still drops the policy: unregister_before/after_tool_call_hook now call a public clear_error_policy (the internal walk), so unregister-then-re-register yields fail-open, as the existing tests pin.

Two notes on the scenario analysis:

  • The decorator and CrewBase paths are unaffected by the wipe either way, as the finding says - they register a distinct wrapper or bound method, so the plain function's abort entry stays intact.
  • One honest consequence of (1): an explicit set_error_policy(hook, 'fail_open') is now a no-op rather than a reset. Nothing in the docs advertises resetting a live hook's policy, and unregister plus re-register covers it; happy to add that if a maintainer wants a public reset.

Tests: 16 in test_hook_error_policy.py; the two new ones fail without this commit. Hooks suite: 216 passed, 1 skipped, 1 failed (test_acall_runs_after_llm_call_hooks, fails identically on main).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant