fix(sanitization): port robust scanner + two-step fail-closed gate

[NimbleCoAI]

The finding

This repo's sanitization gate could report clean while doing half its job, and
its verdict parser threw away good verdicts. Both were reproduced locally against
the pre-change code before anything was edited.

Reproduced (Matilde): with the key unset or set to an empty string, the gate
printed a warning, printed ok <file> for a file the semantic layer never looked
at, and exited 0 reporting sanitization: clean. In CI, an empty or revoked
ANTHROPIC_API_KEY repo secret therefore produces a green check that means
strictly less than it appears to.

Reproduced (osint-engine): worse — the repo had no deterministic layer at
all. AKIA… committed to engine/ exited 0 with
sanitization: no sensitive files changed — skipping. Any credential outside the
four sensitive prefixes walked straight through.

The parser: text[first "{" : last "}"] breaks on the most common real reply
shape — a correct verdict followed by a sentence containing a brace. The span then
holds two objects, json.loads raises Extra data, and the last-{ retry lands
inside the prose. A perfectly good verdict became a hard parse error. This is
the failure that hit agentic-startup-generalist once its key was fixed.

What changed

Scanner ported from swarm-map, kept config-driven so this repo's
sanitize.config.json — its domain_noun, flag_examples,
do_not_flag_examples, prefixes and allowlist — remains the single place domain
knowledge lives. The scanner carries detection machinery only.

	before	after
Verdict parse	first { → last }	walk every {, raw_decode one value, take the first real verdict
Broken/absent key	warning, exit 0, "clean"	"the layer did not run"; --require-semantic exits 2
Unparseable reply	traceback	same fail-closed path, with the reply echoed (truncated)
ok <file>	printed regardless	only for files the semantic layer reviewed
Summary line	clean	names the layers that actually ran
Prompt injection	content passed raw	fenced with a per-call random nonce, declared untrusted

Deterministic layer gains the shapes swarm-map detects and this repo missed:
Stripe, Twilio, SendGrid, Google OAuth client secret, GitHub fine-grained PAT,
Slack webhook, JWT, bearer/basic auth, credentials embedded in connection
strings, quoted and entropy-gated unquoted credential assignments, SSN, and
Luhn-valid card numbers. Wholly low-entropy credential bodies (documentation
stubs) are not reported; PII patterns are never placeholder-gated.

allow_substrings is now matched against the detected value rather than the
whole line. Line scoping silently exempts neighbours — one documentation address
would clear a real address sitting beside it and the log would show nothing.

Workflow: two steps, on purpose

1. Secrets & PII (deterministic) — no key, no network, runs on every PR
   including forks. Red here means a leak is in the diff.
2. Particulars (semantic) — --require-semantic, fails closed. Red here
   means particulars were found or the layer could not run, and the log says
   which.

Collapsing these into one step is what made the gate dishonest: a broken key
degraded to deterministic-only, warned, and exited 0. Fork PRs get step 1 plus a
loud annotation, because GitHub withholds repository secrets from them by design.

Also adds an on-demand full-tree audit lane. Its cron is deliberately left
commented out: the deterministic layer is verified clean tree-wide, but the
semantic layer has never run over the whole tree here, so enabling a nightly run
before anyone triages the first result would just manufacture a standing red.
Instructions to enable are in the file.

Verification

Everything below was actually run, not assumed:

• Scanner self-tests: 63 passed.
• Full offline suite (what tests CI runs): unchanged and green.
• Whole tree, deterministic layer: clean — and it was not clean on the first
  attempt. See "what the new layer found", below.
• Simulated both CI steps against this PR's own diff: step 1 exits 0; step 2 with
  no key exits non-zero with Failing closed rather than reporting a half-checked diff.

The semantic layer runs for real on this PR. docs/sanitization-gate.md is
content-bearing, so step 2 exercises the key end-to-end here rather than passing
vacuously on a diff with nothing to review. If it comes back red, read the
reasons — that is the gate working, and the fix is the content, not the gate.

What the new deterministic layer found

Running it tree-wide surfaced real hits that had never been scanned before. None
were waved through silently:

• The scanner's own comments. Illustrative samples in docstrings (an AWS key
  id, a credential-bearing DSN, a NANP number, a card number, a dotted quad)
  tripped its own detectors. Fixed by describing the shapes instead of writing
  them out — this module is scanned, so a literal sample is a self-inflicted hit.
  A house rule now says so in the file.
• GitHub's own push protection rejected the test fixtures as a real Stripe key
  and Twilio SID. They are supposed to look real — that is how a detector is
  proven. Rather than clicking "allow the secret", each provider prefix is split
  across two adjacent string literals: Python joins them at compile time so the
  scanner sees the whole value, while the file text holds no credential-shaped
  token.

What was deliberately NOT ported

swarm-map suppresses NANP 555 numbers and RFC 5737 documentation IP ranges
outright. Those relaxations are not here: this repo's own tests assert that a
203.0.113.x address is flagged, and quietly loosening a detector to reduce
noise is how a gate stops meaning anything. Verified documentation placeholders
are handled through the config allowlist instead, where a reviewer can see exactly
what is exempted and why.