Proposal: canonical dangerous-callable resolver + shared evasion corpus as a fitness gate

[zied-jlassi]

Problem

The behavioral analyzers detect dangerous primitives by matching a set of name spellings, and each new way to spell the same primitive has needed its own branch: #114/#115 (import alias), #166 (getattr), and the builtins/importlib branch (#180). This enumerate-instead-of-canonicalize pattern leaks a fresh blind spot per missed spelling — e.g. subscript __builtins__["exec"] / os.__dict__["system"], and importlib.util / runpy / code machinery, which are currently undetected.

Proposal

One chokepoint resolve_to_canonical_sink(node, aliases, type_map) reducing any spelling — bare/alias/from/as, builtins.*, importlib.import_module, getattr("lit"), subscript __builtins__["lit"] / <mod>.__dict__["lit"], and dynamic-import / code-exec siblings — to one canonical sink id, reusing the existing alias/type machinery and subsuming the per-idiom branches. Paired with a shared, parametrized evasion corpus (tests/evasion_corpus/): several equivalent spellings per primitive plus FP-neighbors that must NOT match, wired as a fitness test so a future missed spelling fails a shared gate instead of shipping silently.

Scope / non-goals

In scope: statically-resolvable spellings. Explicit non-goals (def-use / taint territory, kept residual): non-literal / kwarg getattr(os, name), variable subscript keys, computed attribute strings, module-name-from-variable, and local indirection (f = os.system; f(x)). Receiver-gated instance methods (spec.loader.exec_module, code.InteractiveInterpreter().runsource) fire only when the receiver resolves to the real machinery.

Status

I have a working prototype (chokepoint + corpus + fitness test) that reproduces all current detections with no regression, adds the subscript / __dict__ / sibling spellings, and composes with #166. Looking for a maintainer yes/no on the direction before opening the PR.

Prepared with AI assistance; reviewed and validated by the author.

[zied-jlassi]

I've pushed a complete working prototype as a branch — no PR yet, happy to open one if the direction is welcome:

Branch: zied-jlassi/SkillSpector:feature/canonical-sink-evasion-corpus

What it contains

• resolve_to_canonical_sink — the single chokepoint described above, reusing the existing alias / type-map / dynamic-import machinery (no parallel resolution scheme).
• behavioral_ast and behavioral_taint_tracking wired to call it, so the per-idiom ladders stay simple. Reflective subprocess invocations grade AST9-HIGH for parity with reflective os.system; a direct subprocess.* call keeps baseline AST4-MEDIUM; a small guard keeps AST9 and the canonical fallback complementary so they never double-fire.
• tests/evasion_corpus/ — the shared parametrized corpus, wired as a fitness gate (every spelling → its canonical id and a real sink rule; benign neighbours → no sink).

Validation (run on the real analyzers in an isolated container, not a bespoke harness)

• evasion corpus: 108/108 pass
• full suite: +108 tests, zero regressions (1000 passed; the single pre-existing test_static_patterns::TestRunStaticPatternsAgentSnooping::test_no_same_line_duplicate failure is unrelated and present on the base branch too)
• ruff==0.15.2 check + ruff format + mypy all clean

Stacking / rebase note
The branch is currently stacked on #180 (the builtins.* / importlib.import_module sink-evasion fix) because the resolver reuses resolve_dynamic_import_call introduced there. Once #180 lands I'll rebase this onto main so the diff shows only the canonical layer. (If you'd rather land the canonical layer first and fold #180's helpers into it, that ordering works too — your call.)

AI disclosure: prepared with AI assistance; every line was reviewed and validated by me, and the results above come from running the repo's real analyzers + test toolchain.

Wanted a yes/no on the approach before opening a PR.

[zied-jlassi]

Update: I've gone ahead and opened the concrete PR for this — #182 — so the approach can be reviewed on real, tested code rather than just a branch link.

Same content as the prototype branch above (canonical resolver + analyzers wired + evasion-corpus fitness gate; 108/108 corpus, +108 suite / zero regressions, ruff + mypy clean). The PR is stacked on #180 and discloses that up front: its diff currently includes #180's commit until #180 lands, after which I'll rebase onto main. Suggested merge order is #180 first, then #182.

Still very happy to adjust the approach (or close) based on your feedback here — opening the PR just makes the diff tangible.