fix: reject agent_main rebinding that lets a decoy pass static anti-cheat screening - #207
Closed
kai392 wants to merge 2 commits into
Closed
fix: reject agent_main rebinding that lets a decoy pass static anti-cheat screening#207kai392 wants to merge 2 commits into
kai392 wants to merge 2 commits into
Conversation
Screening resolves agent_main with find_module_function_def and validates that definition, but the runner executes whatever the module global is bound to after import. Autovara#151 closed only the plain duplicate-def case: a decoy def plus a module-level assignment, an import alias, a def nested in module-level control flow, a `global` rebind, a shadowing star import, or a decorator all still let a submission pass every contract and anti-cheat check while executing a function screening never saw -- the exact route a canned/hardcoded-answer agent needs. Require the inspected definition to be the only module-scope binding of the name: count bindings scope-aware (descending into module-level control flow, stopping at nested function and class scopes), and reject a decorated entrypoint. Nested defs and methods named agent_main bind in their own scope and stay allowed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Author
|
Both workflow runs on this PR are held at Ready for a maintainer to approve the workflow runs whenever convenient. |
The binding-count check only sees the name in a binding position, so the same
decoy still worked through `globals()["agent_main"] = ...`, `globals().update(...)`,
`setattr(sys.modules[__name__], "agent_main", ...)` and `exec("agent_main = ...")` --
one line away from the routes just closed. These reach the module namespace from any
scope (a helper called at import time works too), so they are checked across the whole
bundle rather than in module scope only.
Narrow by construction: a namespace write with a constant key other than the entrypoint
(`globals()["CACHE"] = {}`) provably cannot rebind it and stays allowed, while a computed
key or an opaque `update(...)` argument is not knowable statically and fails closed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Member
|
It is already fixed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #206.
Root cause
screen_bundle_miner_contractvalidates the definitionfind_module_function_defresolves, but the runner executes whatever the module globalagent_mainis bound to after import. #151 closed only the plain duplicate-defroute by countingdefs inmodule_tree.body. A module global is equally rebound by an assignment, animport ... as, a loop/with/excepttarget, adefnested in module-level control flow, aglobaldeclaration, a decorator, or namespace mutation (globals(),setattr,exec) — none of which that check can see.Consequence: every contract and anti-cheat check (no-arg invocation,
vulnerabilitiesreport shape, async rejection, and any subnetstatic_screen) inspects a function the sandbox never calls, so a submission can show real analysis code to screening and execute a canned report — the shape of the incidents in #86 and #122.Every route below passes screening with zero findings on
main(36b2e77):agent_main = _cannedbundle.agent_main_rebounddefnested inif/try/while/with/matchbundle.agent_main_reboundfrom helpers.canned import report as agent_mainbundle.agent_main_reboundwith/excepttarget, tuple or starred unpacking, walrusbundle.agent_main_reboundclass agent_main:shadowing the functionbundle.agent_main_reboundglobal agent_mainrebind from a nested scopebundle.agent_main_reboundagent_mainbundle.agent_main_reboundglobals()["agent_main"] = .../globals().update(...)/vars()[...]bundle.agent_main_reboundsetattr(sys.modules[__name__], "agent_main", ...), incl. from a helperbundle.agent_main_reboundexec("agent_main = ...")bundle.agent_main_rebound@decoratoronagent_mainbundle.agent_main_decorateddef(#151)Fix approach
Require the inspected definition to be the only binding of the name the module ever produces.
kata/ast_utils.py—iter_module_scope_nodeswalks what Python executes in module scope: it descends intoif/try/for/while/with/matchbodies, which run at import time, and stops at nested function and class scopes, which bind locally.count_module_scope_name_bindingscounts every binding form on top of it;has_module_scope_star_importanddeclares_global_namecover the two indirect routes;rebinds_name_dynamicallycovers namespace mutation.kata/screening/rules.py— reject more than one module-scope binding, reject namespace mutation that could reach the entrypoint, and reject a decorated entrypoint.Two deliberate narrowings keep the rules precise rather than blunt:
agent_main;globals()["CACHE"] = {}) provably cannot rebind it and stays allowed, while a computed key or an opaqueupdate(...)argument is unknowable statically and fails closed.Because namespace mutation reaches the module global from any scope — a helper called at import time works just as well — that one check spans the whole bundle instead of module scope only.
No new dependencies, nothing outside these two modules plus tests, and the existing
bundle.agent_main_duplicaterule id and message are untouched so the bot's handling is unchanged.Impact and risk
Makes the static contract binding rather than advisory: the function screening validates is now provably the one the runner imports.
False positives are the real risk here, so the rules are scope-aware and tested from both directions. These all stay legal and are covered: methods and inner
defs namedagent_main, bare annotations (agent_main: Callable),if TYPE_CHECKING:blocks,try/except ImportErrorfallbacks, decorated helpers,__all__/__main__guards, and namespace writes that cannot reach the entrypoint. Both bundles currently inkings/pass unchanged.An honest submission that today decorates its entrypoint would need to inline the wrapper. That is the intended tightening — a decorator's return value is by definition not the inspected body.
Scope boundary worth stating plainly: this closes the routes a submission can take silently. Static screening cannot bound arbitrary runtime reflection in general, but it can ensure that replacing the entrypoint requires code no reviewer would read as ordinary indirection.
Verification
python -m ruff check kata testsclean;python -m pytest80 passed (14 new regression tests — one per route, plus the must-still-pass cases).