fix(callbacks): a security callback that crashes must not read as approval - #777
Open
jasonxi89 wants to merge 1 commit into
Open
fix(callbacks): a security callback that crashes must not read as approval#777jasonxi89 wants to merge 1 commit into
jasonxi89 wants to merge 1 commit into
Conversation
…roval
`_trigger_callbacks` isolates errors by reporting a crashed callback as
`None`. The two phases carrying a `{"blocked": True}` protocol read
`None` as "no objection" — command_runner.py says so in a comment:
# Callbacks can return None (allow) or a dict with blocked=True (reject)
So a guard that fails to complete is indistinguishable from one that
approved, and whether that happens depends only on whether each plugin
remembered to wrap itself. The three guards in code_puppy_core_plugins
disagree: shell_safety catches internally and denies; force_push_guard
catches only AttributeError/OSError; destructive_command_guard documents
fail-open as intentional in its AGENTS.md. Two are decisions, one is
silence, and the framework offers no way to say which you meant.
`register_callback(phase, func, fail_closed=True)` makes it explicit. A
marked callback's exception becomes a block result instead of `None`.
The default is unchanged, so every existing registration behaves exactly
as before, and the flag is rejected on phases whose consumers would
misread a block result.
Design notes:
* Reported as a result, not by re-raising. The existing `raise_on_error`
cannot serve this: pydantic_patches wrapped the pre_tool_call block in
`except Exception: pass`, so a raised deny was swallowed and the tool
ran anyway.
* Keyed by (phase, callback), so one callable registered on several
phases can hold a different policy on each.
* The synthesized message names the callback and exception type but not
the exception text, which reaches the user and the model; the full
traceback stays in the log.
* The sync trigger's "async callback reached from a running loop" branch
is covered too — undecided is not unopposed.
Separately, a hook could already return an explicit `{"blocked": True}`
and still have the tool run: a non-string `reason` made
`"[BLOCKED]" in raw_reason` raise inside that same broad `except`.
`_block_reason` is now total — extraction, coercion and rendering are
all inside the guard, because `.get` and a plugin object's truthiness
can raise just as easily as `__str__`. The block decision itself moved
out of the broad `except`, so a failure while emitting the warning no
longer turns a deny into an allow.
No plugin in this repository opts in; the consumer lives in
code_puppy_core_plugins and is a separate change.
Author
|
@thomwebb — could I get your eyes on this when you have a moment? (I can't add reviewers directly from a fork.) Flagging it because unlike a straightforward bug fix this one needs a call from a maintainer rather than just a correctness check:
The part that is just a bug, and stands on its own regardless of the above: a hook returning CI is green (macos 3.13 / windows-encoding / quality) and the branch is mergeable. |
thomwebb
self-requested a review
August 18, 2026 05:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moved here from the Walmart-internal fork at the maintainers' request — the defect is entirely in open-source code.
Problem
_trigger_callbacksisolates callback errors: it logs the exception and appendsNoneto the results. The two phases that carry a{"blocked": True}protocol readNoneas approval.code_puppy/tools/command_runner.pystates it outright:# Callbacks can return None (allow) or a dict with blocked=True (reject)A security callback that fails to complete is therefore indistinguishable from one that approved.
Today the outcome depends only on whether each plugin remembered to wrap itself, and the three guards in
code_puppy_core_pluginsdisagree:shell_safety{"blocked": True}force_push_guardAttributeError/OSErrordestructive_command_guardTwo of those are deliberate.
shell_safetychose deny in code.destructive_command_guard/AGENTS.mdchose allow in writing — "fail closed on ambiguity, fail open on our own errors."force_push_guardstates nothing. The framework gives no way to express the choice.It is reachable, not theoretical.
shell_safety_callbackreads three config values before its owntry.ConfigParserinterpolates lazily, so a value likeyolo_mode=%parses cleanly — the corruption quarantine never sees it — and raises only when the option is read. I verified each of these raises rather than returning a default:The guard raises, the dispatcher returns
None, and the command runs with no safety assessment.Change
register_callback(phase, func, fail_closed=True)lets a callback declare that its failure means deny. The default is unchanged, so every existing registration behaves exactly as before.raise_on_errorcannot serve this:pydantic_patcheswrapped the pre_tool_call block inexcept Exception: pass, so a raised deny was swallowed and the tool ran anyway.(phase, callback), so one callable registered on several phases can hold a different policy on each.load_promptorgit_branch_providerwould misread.Second, independent fail-open
A hook could already return an explicit
{"blocked": True}and still have the tool run, with no crash involved: a non-stringreasonmade"[BLOCKED]" in raw_reasonraise inside the same broadexcept. Reproduced against the pre-change logic:_block_reasonis now total. Extraction, coercion and rendering are all inside the guard, because.getand a plugin object's truthiness can raise as easily as__str__. The block decision moved out of the broadexcept, so a failure while emitting the warning no longer turns a deny into an allow.Scope
No plugin in this repository opts in — the guards live in
code_puppy_core_plugins, so wiringshell_safetyis a separate change I'm happy to open next.destructive_command_guardshould stay as it is regardless: its fail-open is a documented product decision for its owners, not something to change from an infrastructure PR.Testing
tests/test_callbacks_fail_closed.py— 25 cases: default behavior unchanged, both dispatchers plus the unawaitable-async branch, phase restriction, per-phase policy isolation, repeat-registration tightening,raise_on_errorprecedence, no exception text in user-facing output, registry housekeeping, and_block_reasonagainst non-string reasons, a hostile__str__, and a hostile__bool__.tests/conftest.pynow snapshots and restores_fail_closed_callbacksbeside_callbacks; restoring one without the other would hand the next test callbacks whose policy had quietly gone missing.tests/plugins/test_aws_bedrock.py::TestSupportsAdaptiveThinking::test_supports_adaptive_thinking[opus_4_5_minor_version_not_adaptive], is pre-existing and order-dependent: a cleanmainfull run reproduces it (7310 passed, same 1 failure), while running that file alone passes on both.ruff checkandruff formatclean on the changed files.