Repository navigation
feat(hooks): let a tool hook opt into failing closed - #9465
Harvey DU (Luckyou-d) wants to merge 10 commits into
Conversation
Tool hooks are fail-open: dispatch swallows every exception except `HookAborted`, so a hook used as a security control that cannot reach a decision lets the call through. A policy hook whose service times out, or a redaction hook that crashes, silently stops protecting anything, and only `verbose` mentions it. `register_before_tool_call_hook` and `register_after_tool_call_hook` take an `on_error` keyword. The default keeps the historical fail-open behaviour; `on_error="abort"` turns any other exception into `HookAborted` with the original exception as the cause, so the before hook blocks the tool call and a failing after hook cannot let an unredacted result reach the model. The policy lives on the dispatch engine (`HookErrorPolicy`, `set_error_policy`) and is keyed by the hook callable, so no wrapper is inserted into the registry that legacy modules alias, and unregistering or clearing a hook drops its policy with it. Refs crewAIInc#9434
Strix Security ReviewWarning This pull request has 53 commits after the last Strix review ( ✅ No open security findings. PR summary: Adds an on_error keyword to register_before_tool_call_hook and register_after_tool_call_hook so a hook can opt into failing closed: any exception other than HookAborted becomes HookAborted, blocking the tool call or preventing an unredacted result from reaching the model. The policy is held in a new dispatcher-level mapping keyed by the hook callable, cleaned up by unregister/clear, with tests for both policies. Review summaryReviewed the new per-hook on_error policy for tool hooks: HookErrorPolicy/set_error_policy/_error_policy in hooks/dispatch.py, the _invoke_hook abort branch, the two register functions in hooks/tool_hooks.py, the export, and the added tests. The fail-closed path was traced end to end through every dispatch seam and every tool-execution caller — before-hook aborts skip the tool body, and an aborted after hook propagates as HookAborted from execute_tool_and_check_finality, execute_single_native_tool_call and the agent executors before any result is written to messages, steps or memory, so the unredacted tool result does not reach the model. The roundtrip is also closed (dispatch looks the policy up on the exact object registration stored, including bound methods, LLM snapshot lists and execution-boundary points), and the converted exception's text is never placed in model context. No security vulnerability was confirmed. The issues that surfaced are API ergonomics and lifecycle consistency: per-callable policy keying lets a second registration with the default reset an existing opt-in, and the legacy tool-hook unregister/clear helpers bypass dispatch so policies are not dropped there — all requiring in-process application code that already owns the hooks, with no untrusted input involved and no capability gained. Ruled out (5)
Review 1 on this PR · Reviewed by Strix |
A filtered hook (`@before_tool_call(tools=...)`, `@on(..., agents=...)`, a `@CrewBase` filter) is registered as a wrapper while the caller keeps the original function, so `set_error_policy(func, "abort")` attached the policy to a callable dispatch never sees. `_error_policy` read only the invoked wrapper, missed, and fell back to fail-open: a filtered approval or redaction hook set to fail closed still let the operation through. Walk the wrapper chain (`__wrapped__`, and `_registered_hook` for `@on`) so a policy set on either the original or the registered callable is honoured, and drop the whole chain when the hook is unregistered or cleared. `set_error_policy` is exported so the two public APIs match.
|
Valid catch, and it was reachable from the public API — thanks. Fixed in 03e1a9a. A filtered hook is registered as a wrapper (
Reproduced before the fix: with Hooks suite: 210 passed, 1 skipped, 1 failed; that failure is |
A `@CrewBase` hook method is registered as a bound method, while the decorator returns - and the docs tell callers to pass - the plain function. `set_error_policy` on that function therefore missed, because the bound method is a different dict key, and the miss returns fail-open: the guard still let the operation through. Walk `__func__` as well (`__wrapped__` and `_registered_hook` are kept), and link the crew-base filter closures to the method they wrap with `functools.wraps`, so a filtered crew method resolves too. Tests cover a policy set on the class function, on the bound method, and on a filtered crew method.
`@on` sets `_registered_hook` only for plain functions, so a method is linked back through the filter wrapper's `__wrapped__` and the bound method's `__func__`. The test pins that path; it fails without the `__func__` walk.
|
Valid again, and reachable through the public API - thanks. Fixed in f0333a8 (+25895b5 for the Confirmed the first scenario before the fix: with Three changes:
On the last point: per-executor LLM hook lists snapshot the callables, not the policy, and Tests: 14 in |
`register_before_tool_call_hook` and `register_after_tool_call_hook` always wrote the policy map before appending, and the default (`on_error="fail_open"`) was implemented as `_error_policies.pop(hook)`. Two reachable wipes followed: - `set_error_policy(hook, "abort")` and then `register_*_tool_call_hook(hook)` with the default dropped the abort before the hook ever ran, although the docstring says the callable "was or will be registered"; - re-registering an already-registered callable with the default removed the abort the earlier registration stored, while the hook stayed on the list. The default no longer touches the map: only an explicit `abort` writes a policy. Removal goes through `clear_error_policy` (the public counterpart of the internal walk), which `unregister_before/after_tool_call_hook` now call, so unregistering a hook still drops its policy. Two tests pin the wipes; both fail without this commit.
…kyou-d/crewAI into feat/hook-error-policy-abort
|
Valid again - both wipes reproduce from the public API. Fixed in cad7796. Reproduced before the fix: The fix:
Two notes on the scenario analysis:
Tests: 16 in |
Summary
Tool hooks are fail-open by design:
dispatchswallows every exception exceptHookAborted, which protects the framework from a buggy user hook. For a hook used as a security control that is the wrong default - a policy hook whose service times out, or a redaction hook that crashes, silently stops protecting anything, and onlyverbosementions it.Change
register_before_tool_call_hookandregister_after_tool_call_hooktake anon_errorkeyword:"fail_open"(default) keeps the current behaviour exactly."abort"turns any exception other thanHookAbortedintoHookAbortedwith the original exception as its cause and the hook as its source. A before hook therefore blocks the tool call, and a failing after hook cannot let an unredacted result reach the model. A hook that raisesHookAborteditself keeps its own reason.The policy lives on the dispatch engine (
HookErrorPolicy+set_error_policy) and is keyed by the hook callable rather than wrapping it, so nothing new is inserted into the global hook list that legacy modules alias, andunregister_*/clear_*drop the policy together with the hook.Verification
pytest lib/crewai/tests/hooks/test_hook_error_policy.py-> 6 passed. Five of them fail onmain(the keyword does not exist yet) and the sixth pins that the default is still fail-open.The last test is the reproduction from the issue, driven through
execute_tool_and_check_finality: withon_error="abort"the tool body does not run and the agent receivesTool execution blocked by hook. Tool: send_email.pytest lib/crewai/tests/hooks-> 206 passed, 1 skipped, 1 failed; the failure (test_acall_runs_after_llm_call_hooks) fails identically onmainwithout this changeruff checkandruff format --checkclean on all four filesRefs #9434