Auditable LB changes, evidence export, and console fixes (F1–F3)

.env.example

@@ -22,6 +22,7 @@ GITHUB_TOKEN=        # repo-scoped PAT for opening PRs (or use `gh auth login`)
 # VPCOPILOT_PROTECTED_LBS=nimbus-www   # comma-separated LBs that refuse mutation without an override
 # VPCOPILOT_REFINE_ATTEMPTS=3          # max self-heal attempts when a policy doesn't block the exploit
 # VPCOPILOT_REQUIRE_PROBE=1            # fail-closed: don't validate a finding that has no derived probe
+# VPCOPILOT_ACTOR=jane.doe             # audit-log attribution — default: the OS user; set it in CI
 # --- Validation auth (auth-protected targets: let the probe log in so it can demonstrate the exploit) ---
 # VPCOPILOT_PROBE_USER=                # username the validation probe logs in with (cookie or token app)
 # VPCOPILOT_PROBE_PASS=                # its password

BACKLOG.md

@@ -2,11 +2,16 @@
 
 Ideas captured for later; not yet scheduled.
 
-- **HTML results dashboard.** Have an agent generate a self-contained HTML dashboard of a
-  run — findings, triage (band-aids + residual risk), generated XC configs, and code-fix
-  PRs, plus the benchmark scorecard. A shareable artifact for stakeholders. NOTE: the
-  interactive console dashboard now exists — this is the *standalone, static, shareable*
-  export (a single self-contained .html file). _(Requested 2026-07-01.)_
+- ~~**HTML results dashboard.**~~ ✅ **Done** — `report.py` writes a single self-contained
+  `<out>/report.html` (inline CSS, no external assets, native `<details>`) at the end of every
+  `run_pipeline`; `vpcopilot report` rebuilds it from any existing out dir. It carries the hero,
+  at-a-glance severity/coverage bars, model independence, pipeline metrics, findings + band-aid
+  coverage, the generated XC policies, the `found → mitigated → remediated → retired` ledger and
+  the band-aid impact table. Reachable from ② Review and Setup — **Open HTML report ↗** (rebuilt
+  from the current out dir on every request, so it is never stale) and **Download** for a stamped
+  `vpcopilot-report-<run>-<UTC>.html`. Not included: the benchmark scorecard — that still lives
+  only in the console's ⑥ Benchmark step. Original ask: a standalone, static, shareable export of
+  a run for stakeholders. _(Requested 2026-07-01.)_
 - ~~**Ops console admin panel (localhost).**~~ ✅ **Done** in the console MVP — the Admin tab
   reads/writes the local `.env` (XC creds + model API keys), redacting secrets.
 - **Benchmark: bonus-vuln handling.** Add a `bonus:` section to `bench/answer_key.yaml` so
@@ -16,3 +21,20 @@ Ideas captured for later; not yet scheduled.
   rate) and discovery duplicates, not just discovery + triage.
 - **Finding correlation as a first-class step.** The model already remarks "band-aid for A
   covers B" — make it explicit so overlapping band-aids are deduped/linked in the output.
+- **Sign the evidence bundle.** `manifest.json` SHA-256s every member, so tampering with a
+  *member* is detectable — but nothing binds the manifest to a signer. A detached signature
+  (sigstore or a plain GPG/minisign detached sig next to the manifest) would make a bundle
+  attributable after it leaves the machine, not just internally consistent. _(Requested 2026-07-26.)_
+- **`vpcopilot export --verify`.** The other half of the above: re-read a bundle, recompute each
+  member's digest against the manifest, and print pass/fail. Cheap, and it means the reviewer
+  doesn't have to trust the sender's word. _(Requested 2026-07-26.)_
+- **Stream audit events to a SIEM.** `audit.record` is the single choke point every mutating path
+  goes through — an optional sink (syslog / HTTP webhook / stdout JSON) would put the trail
+  somewhere other than the box that made the change. Fail-soft: a dead collector must never fail
+  an apply. _(Requested 2026-07-26.)_
+- **Backfill attribution onto historic audit logs.** Entries written before identity was stamped
+  in `audit.record` have no `run_id` / `actor`, and older `apply_*` entries no `finding_id`; the
+  export leaves those cells blank rather than guessing. A one-shot `vpcopilot audit backfill`
+  could fill only what is provably derivable (policy name → finding, via `policies.json`), mark
+  the rest `unknown`, and record that it did so — a backfill that silently invents an actor is
+  worse than a blank. _(Requested 2026-07-26.)_

CONTRIBUTING.md

@@ -29,6 +29,12 @@ the nightly job.
   roll back on failure, and honor the protected-LB / protected-policy guardrails.
 - New band-aid controls are added in one place: the `controls.py` registry (attach/detach, LB-wide,
   validation kind, refine strategy). Wire the handler through the engine, not a new bespoke function.
+- Every mutating path must leave an audit record. Call `audit.record(out, "<action>", ...)` with at
+  least `finding_id` and `namespace` — an entry that can't say *why* an LB changed, or *in which
+  tenant*, isn't an audit record. Identity (`run_id` / `actor` / `host` / `tool_version`) is stamped
+  inside `audit.record`, never at the call site, and those keys are stripped from `**detail` so a
+  caller can't override them. Register the new action in `export.CATEGORY` / `export.CONTROL` so the
+  evidence bundle names it instead of showing `other`. Dry runs stay unrecorded, by design.
 - Match the surrounding style; `ruff` enforces the important bits.
 
 ## Scope + safety

DESIGN.md

@@ -90,6 +90,11 @@ abstraction and handle differences with config.
 - **Self-validation + auto-rollback:** after applying, fire the exploit + a legit request;
   if the exploit isn't blocked or legit traffic breaks → auto-revert and flag. Validation
   target is the **live LB** (more demo-dramatic; the snapshot/rollback makes it safe).
+- **Audit trail:** every mutating action in the finding lifecycle appends one JSON line to `<out>/audit.log`, stamped
+  centrally with `run_id` / `actor` / `host` / `tool_version` and carrying `finding_id`,
+  `namespace`, the LB, the XC object it touched, `rolled_back` **and** `kept` (rolled-back alone
+  can't say whether the change is still live), plus the before/after exploit proof. Dry runs are
+  not recorded — nothing changed, so there is nothing to answer for. See *Audit + provenance*.
 - **Secrets:** scoped XC token + provider keys in env / secret store, never in git.
 
 ## XC integration (`xc/`, next increment)
@@ -107,18 +112,96 @@ API. Each PR notes it permanently remediates an issue currently held closed by a
 XC virtual patch (retire the policy on merge). A **ledger** tracks finding state so
 band-aids don't silently become permanent.
 
+## Audit + provenance (`audit.py`, `runmeta.py`, `export.py`)
+A band-aid is a change to production infrastructure. The question a change board asks is not "did
+it work" but **"why was this load balancer changed, by whom, in which tenant, and is it still
+live?"** These three modules exist to answer exactly that — and nothing more: they are evidence for
+a human reviewer, not a compliance certification.
+
+**What one entry carries.** Every mutating action appends a JSON line to `<out>/audit.log`:
+
+| stamped centrally (`audit.record`) | supplied by the action |
+|---|---|
+| `ts` · `run_id` · `actor` · `host` · `tool_version` | `finding_id` · `namespace` · `lb` · the XC object (`policy` / `app_firewall` / `apidef` / …) · `passed` \| `config_enabled` \| `enabled` · `rolled_back` · `kept` · `attempts` · `before_after` |
+
+15 actions emit records: the 7 `apply_*`, the 3 `create_*`, `refine_apply`, `apply_timing`,
+`open_pr`, `retire`, `rollback_failed`.
+
+### The decisions
+1. **Identity is stamped inside `audit.record`, never at the call sites.** Nineteen `record()` call
+   sites write the trail — `apply.py`, `refiner.py`, `engine.py`, `pr.py`, `retire.py`, the console.
+   Asking each to remember `run_id`/`actor`/`host`/`tool_version` guarantees one eventually forgets
+   — and an entry that can't say who made the change, from where, as part of which run, is not an
+   audit record. `record()` also **strips** those keys out of `**detail`, so a caller cannot
+   override them. Only the per-action facts (`finding_id`, `namespace`, the object, the outcome)
+   stay at the call site, because only the call site knows them. `engine.ApplyContext` carries
+   `finding_id` for the same reason: a rollback failure raised deep in the spine must still be
+   attributable to the vulnerability that justified the change.
+2. **Run identity lives in `<out>/run.json`, keyed by `run_id`.** `runmeta.run_id(out)` mints an id
+   on first use and persists it (atomic replace), so a scan and a `vpcopilot apply` an hour later
+   **in a separate process** against the same out dir stamp the same run — the join key is the
+   directory, not process memory. `write_manifest` never clobbers an existing `run_id`, so a
+   re-scan keeps the identity the entries already on disk join to. Provenance sits beside it:
+   scanned repo + commit/branch/dirty, config path, per-agent models, caps, counts,
+   started/finished, actor/host/tool_version. Because it lives *in the run dir*, an exported bundle
+   is self-describing — it doesn't need this machine, this checkout, or this console to be read.
+3. **Write heterogeneous → normalize at export.** The log stays append-only and per-action: each
+   record keeps exactly the fields that mattered for that action (a WAF's `config_enabled` is not a
+   service policy's `passed`). That is right for a log and wrong for a reviewer, who needs one
+   sortable shape. So `export.build_audit_events` does the flattening at **read** time — coalescing
+   `passed ?? config_enabled ?? enabled` into one `outcome`, resolving `finding` → `finding_id`,
+   recovering a finding from the `policies.json` index when an older entry names none, and joining
+   the ledger + `findings.json` for title/class/severity/state. Normalizing at write time would
+   have cost per-action fidelity permanently and frozen the schema; normalizing at export keeps
+   both, and lets old logs improve as the normalizer learns their shapes. It keeps **every** entry:
+   the report's impact table filters to entries with `before_after` or `behavioral`, which would
+   silently drop `retire`, `open_pr`, every `create_*` and every config-only apply — an export that
+   quietly loses rows is worse than no export.
+4. **The export is read-only and stdlib-only.** `zipfile` / `csv` / `hashlib`, no new dependencies;
+   it touches neither XC, GitHub, nor the run's artifacts. Evidence gathering must never be able to
+   change the thing it is evidence of. The bundle carries the normalized events (`audit.csv` +
+   `audit-events.json`, with `detail` keeping the raw JSON so flattening loses nothing), the raw
+   `audit.log` **verbatim**, `run.json`, the scan artifacts, the exact XC configs pushed
+   (`policies/*`), the pre-change LB snapshots (`snapshots/*`), and a `manifest.json` that SHA-256s
+   every member — plus an explicit `caveats` list stating what the trail does *not* cover.
+5. **Provenance writing is fail-soft.** `git_provenance` on a non-git target contributes nothing;
+   a `write_manifest` failure logs a warning and returns. Provenance is evidence, not a gate — it
+   must never fail a scan that already succeeded. Same instinct as the ledger: a half-written
+   manifest reads as `{}` rather than raising.
+
+**Surfaces** (repo convention: console and CLI call the same module function) —
+`GET /api/audit-events` (the Retire step *shows* the trail before anyone exports it, so you can
+check what leaves the machine), `GET /api/audit-export?scope=run|all`, `GET /api/runs`; and
+`vpcopilot export [--out DIR] [--output PATH] [--all] [--root DIR]`.
+
+**Honest limits.** Dry runs are unrecorded by design. `apply_timing` exists only for
+console-driven live applies. Entries written by older builds lack `finding_id`/`namespace`/`actor`
+and export as blank cells rather than inferred ones. `VPCOPILOT_ACTOR` overrides the OS user — set
+it in CI or on a shared jump host so changes name the engineer, not the service account.
+
 ## Repo layout
 ```
 src/vpcopilot/
   schemas.py        typed agent I/O (the cross-model contract)
-  config.py         per-agent model registry
+  config.py         per-agent model registry (+ AGENT_NAMES)
   harness.py        LiteLLM + instructor (model independence)
   repo_scan.py      collect candidate source files
-  agents/           discover, verify, triage, generate, remediate
-  pipeline.py       deterministic orchestration (read-only today)
-  cli.py            `vpcopilot scan`
-config/agents.yaml  model-per-agent
-tests/              schema/config smoke tests (no API needed)
+  agents/           discover, verify, triage, generate, remediate, probe, refine
+  pipeline.py       deterministic orchestration (scan → artifacts + run.json)
+  engine.py         SafeApply spine: snapshot → self-test → attach → validate → keep/rollback
+  controls.py       registry of the 7 XC controls (attach/detach inverse, validation kind)
+  apply.py          per-control apply entry points (CLI + console both call these)
+  refiner.py        validate → refine → retry until the band-aid blocks (or gives up honestly)
+  xc.py / probe.py  XC API client · executable exploit + legit request per finding
+  ledger.py         found → mitigated → remediated → retired
+  audit.py          append-only audit log (stamps run_id/actor/host/tool_version)
+  runmeta.py        run identity + provenance (<out>/run.json, VPCOPILOT_ACTOR)
+  export.py         evidence bundle (.zip): normalized events + raw artifacts + manifest
+  report.py         standalone HTML report · retire.py detach · pr.py the cure
+  console/          FastAPI ops console (app.py + static/index.html)
+  cli.py            scan · apply · export · console · bench-model · …
+config/agents*.yaml model-per-agent, one config per model
+tests/              offline tests against fakes (no API needed)
 ```
 
 ## Roadmap
@@ -142,6 +225,13 @@ tests/              schema/config smoke tests (no API needed)
    and an Admin panel that reads/writes the local `.env`. TODO: remediation ledger, richer
    before/after panel. _(superseded original bullet below)_
    ~~review → approve → apply → undo, with a live before/after panel and the remediation ledger.~~
+6. **Audit + evidence export (done):** ✅ every LB/object-mutating record now carries `finding_id`
+   + `namespace` (and `kept`), with `run_id`/`actor`/`host`/`tool_version` stamped centrally in
+   `audit.record`; `<out>/run.json` (`runmeta.py`) gives a run its identity and provenance so a
+   scan and a later apply join up; `export.py` normalizes the log and zips the run's evidence
+   (`vpcopilot export [--all]`, `GET /api/audit-export?scope=run|all`); the console's Retire step
+   shows the trail — when · action · justified by · control · LB · outcome · by — before you export
+   it. `VPCOPILOT_ACTOR` names who a change is attributed to.
 
 ## Open decisions
 - Remediation output starts as **GitHub PRs** (confirmed).