Auditable LB changes, evidence export, and console fixes (F1–F3)#1
Merged
Conversation
…it (+ --app-firewall/--template flags) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ply/refine apply --from-scan can now take --finding <id> so the refiner/validator loads the real probe (out/probes.json keyed by finding_id) instead of falling back to the Nimbus negative-pay probe. Threaded through cli.apply -> refine_apply_service_policy (already accepted finding_id) and apply_from_scan -> apply_service_policy. Enables genuine live validation + self-refine of a generated policy with no scan/ledger. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ation (A2-A4) Both controls historically keyed abuse on client IP (user_id_client_ip), which mis-attributes when many agents egress a shared IP behind an edge. - xc.py: user_identification CRUD (get/create/exists), mirroring the service_policy idiom. - apply.py: _user_id_ref (tenant-qualified) + _ensure_user_identification (create a user_identification keyed on an HTTP header if absent). apply_malicious_user and apply_rate_limit take --user-id-header/--user-id-name: when set they create+reference the user_identification and drop user_id_client_ip; otherwise behavior is unchanged. - probe.py: probe_rate_limit takes headers=, so a per-identity limit can be driven from one identity and confirmed against another. - cli.py: apply-maluser and apply-ratelimit expose --user-id-header/--user-id-name; apply-ratelimit also wires the existing --behavioral-path and adds repeatable --burst-header name=value. Note (per the Mode-2 plan A4): whether an LB user_identification makes the aggregate rate_limiter count per-identity is not decidable from source — the two-identity behavioral burst is the arbiter; if a clean agent is also throttled, a keyed rate_limiter_policy is needed. Tests cover the header-keyed vs IP-default paths; 139 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An audit entry could not answer the questions an auditor asks. Of the 15 actions the codebase emits, not one apply_* recorded a finding_id — every apply function had it in scope and dropped it — and nothing recorded the XC namespace, the operator, or which run the change belonged to. - audit.record stamps run_id/actor/host/tool_version centrally, so no mutating path can forget it and a caller cannot override it (reserved keys are stripped from **detail). - finding_id + namespace on all 19 record sites: the 7 apply_* and 3 create_* in apply.py, the 5 refine_apply in refiner.py, rollback_failed in engine.py, and retire. apply_waf / apply_data_guard / apply_api_schema also record `kept` — "was it rolled back" alone cannot say whether a change is still live. - engine.ApplyContext carries finding_id, so a rollback failure raised deep in the spine is still attributable; pr.py writes finding_id alongside the legacy `finding` key. - runmeta.py: <out>/run.json provenance manifest (repo commit/branch, config, per-agent models, caps, counts, actor/host) keyed by a run_id every entry joins back to. Written fail-soft from run_pipeline — provenance is evidence, not a gate on a completed scan. - export.py normalizes the heterogeneous append-only log into one reviewable row per event (joined to ledger + findings) and packages a .zip whose manifest SHA-256s every member. It keeps EVERY entry on purpose: report.py's impact table filters to before_after or behavioral, which would silently drop retire, open_pr, create_* and config-only applies. - `vpcopilot export [--out] [--output] [--all]` is the CLI twin of the console endpoint. Additive: every existing reader uses .get(), so report.py and bench_model are unaffected. Dry runs stay unrecorded by design, and the bundle manifest states that in-band so an export can never imply more coverage than it has. 30 new tests; 186 pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… Retire (F1-F3)
Three operator-facing gaps, all in the console.
F1 — the scan log was truncated server-side to the last 40 lines (/api/action: 60), so the
full transcript never reached the client at all; and #scanLog was a bare <pre> with no
height cap, which made the global `pre { overflow:auto }` inert — it grew and pushed the
page down. Both endpoints now serve the whole log with a `?since=<n>` incremental tail and
a `log_total` cursor, behind a bounded sink (LOG_MAX) that says in-band when it caps. The
page appends a text node instead of rewriting, so scroll position and selection survive
each poll; it sticks to the bottom only when you were already there, and a "follow" chip
plus a line count appear when you scroll up to read.
F2 — /api/report already rebuilt from the current run dir on every request, so "latest run"
was already true; only the affordance was missing. Added Open HTML report / Download to
Review (in the section markup — renderReview clobbers #reviewBox) and the Download to
Setup, both captioned with the run dir they rebuild from. `?download=1` sends a stamped
attachment via _stamped_name().
F3 — /api/audit existed but nothing in the UI ever called it. Retire gains an Audit trail
card: when, action, the finding that justified it, control + XC object, LB + namespace,
outcome (with the self-heal badge and the 200 -> 403 proof), and who ran it; filterable,
with a per-row raw-JSON expander. The trail is SHOWN before it can be exported, so you can
check what leaves the machine. New read endpoints: /api/audit-events, /api/runs, and
/api/audit-export?scope=run|all streaming the bundle export.py builds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ixture
docs/AUDIT.md (new) is the reference a security/GRC reader gets sent: what is and is not
recorded, all 15 actions with their detail fields, the run.json schema, the normalized
event model and how `outcome` is derived, the bundle contents with a runnable verification
snippet, and worked examples ("which vuln justified this change?", "what is still live?").
It is explicit about the limits — the log is not tamper-evident, `actor` is attribution
rather than authentication, and a bundle is evidence for a human reviewer, not a compliance
certification.
README rewritten around three pillars instead of two; auditability is now first-class
rather than absent. USAGE §7 rewritten — it described Dashboard/Workflow/Ledger/Admin tabs
that no longer exist. DESIGN gains the audit/provenance decisions (identity stamped in one
place; write heterogeneous, normalize at export; fail-soft provenance). QUALITY_PLAN gains
Phase 5 so the F1-F3 tags cited in the source resolve.
Screenshots regenerated — the set predated the Benchmark step and the model switcher, so
all five were stale beyond this change. Captured headless at 1200px/2x against a
credential-free .env: with XC creds loaded the hero renders a deep link carrying the tenant
hostname and namespace, which would have shipped in the image. DEMO.md documents that.
demo/build_demo_out.py writes the finding_id + namespace shape a real run now writes, plus
run.json, so the Retire audit trail is fully populated in the offline dataset instead of
rendering half-blank columns. It also spaces the timestamps (the builder wrote all 13 in
one burst) and curates actor/host/out_dir — no real machine identity or home directory in
a fixture that is committed and screenshotted. That curation lives in the builder, not in
runmeta: the audit trail depends on runmeta being honest.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an end-to-end audit trail for every change made to a load balancer, an evidence
export, and two console fixes. Also carries four earlier
applycommits that were sittingunmerged on this branch — see Also in this PR at the bottom.
Why
An audit entry could not answer the questions an auditor asks. Of the 15 actions the
codebase emits, not one
apply_*recorded afinding_id— every apply function had itin scope and dropped it — and nothing recorded the XC namespace, the operator, or which run
a change belonged to.
/api/auditexisted but nothing in the UI ever called it.What changed
F3 — auditable LB changes (
76e29f9)audit.recordstampsrun_id/actor/host/tool_versioncentrally, so nomutating path can forget it and a caller cannot override it.
finding_id+namespaceon all 19 record sites.apply_waf/apply_data_guard/apply_api_schemaalso recordkept— "was it rolled back" alone cannot say whether achange is still live.
ApplyContextcarriesfinding_id, so a rollback failure raised deep in the spine isstill attributable.
runmeta.py:<out>/run.jsonprovenance manifest (repo commit/branch, per-agentmodels, caps, counts) keyed by a
run_idevery entry joins back to. Fail-soft — it neverfails a completed scan.
export.py: normalizes the heterogeneous append-only log into one reviewable row perevent, and packages a
.zipwhose manifest SHA-256s every member.vpcopilot exportisthe CLI twin.
F1 + F2 + F3 console (
f5017a7)/api/scanwas truncating to the last 40 lines (/api/action: 60) — the full transcriptnever reached the client. Both now serve the whole log with a
?since=tail, in a boundedscrollable box that sticks to the tail unless you scroll up to read.
?download=1.the machine.
Docs + fixture (
806717d) — newdocs/AUDIT.md, README rewrite, USAGE §7 rewritten (itdescribed tabs that no longer exist), all five screenshots regenerated.
Verification
ruff check src tests democlean; 186 tests pass, coverage 73% (floor 65%).test_export.py,test_audit_provenance.py,test_console_*.py.docs/AUDIT.mdwas run verbatim against a realbundle: 18 members, 0 hash mismatches.
Honest limits (stated in the docs, not just here)
tamper-evident. The manifest's SHA-256s prove a bundle was not altered after export;
they say nothing about the log before it.
actoris self-asserted — attribution, not authentication.apply_timingonly exists forconsole-driven live applies. The bundle manifest states both in-band.
the legacy
findingkey and the policy index, and leaves cells blank rather than guessing.audit.logfiles. Tracked inBACKLOG.mdalong with bundlesigning and
export --verify.Review notes
.get(), soreport.pyandbench_modelare unaffected.demo/outis regenerated.build_demo_out.pynow curatesactor/host/out_dirso noreal machine identity or home directory lands in a committed, screenshotted fixture — that
curation lives in the builder, deliberately not in
runmeta, which the audit traildepends on being honest.
.env: with XC creds loaded the heroband renders a deep link carrying the tenant hostname and namespace, which would otherwise
ship in the image.
docs/DEMO.mddocuments that for whoever regenerates them next.Also in this PR
Four commits that predate this work and were already unmerged on the branch:
4c7aadbper-agentmalicious_user+rate_limitviauser_identification(A2–A4) ·eea6c78api_schemavalidates headers + query params ·0f506fb--findingthreads aprobe id into service-policy apply/refine ·
38eeebcdata_guardreuses the attached WAF.Happy to split them onto their own branch if you'd rather review them separately.
🤖 Generated with Claude Code