Functional client-side sandbox comparison site#40
Closed
chrisns wants to merge 2 commits into
Closed
Conversation
Static GitHub Pages report over the probe's data: a baseline-normalized three-state comparison matrix, an exposure-over-time chart with version-release markers, a per-capability heatmap, and an attributed flip-log — all derived client-side from one concatenated reports file (dev fixture: sample-data.json). Design intentionally left plain; this is the functional pass for a design system to build on. Records the domain glossary (CONTEXT.md), the pipeline decision (ADR 0001), and the build checklist (docs/reporting-site-plan.md). Claude-Session: https://claude.ai/code/session_01PtFjYK8MdnDMmFYVWPDRrn
chrisns
marked this pull request as draft
July 13, 2026 15:40
A probe task can run and find nothing, emitting the finding type with an empty value (e.g. DNS blocked -> external_host_dns_resolution: []). The renderer counted a present finding as a leak regardless, so a blocked capability rendered as leaked. Caught against real Seatbelt data, where strict confinement zeroes the network findings rather than dropping them. Gate on a non-empty value. Claude-Session: https://claude.ai/code/session_01PtFjYK8MdnDMmFYVWPDRrn
Collaborator
Author
|
Validated against real probe data, not just the fixture: seeded a throwaway $HOME, ran a real `direct` baseline and real Seatbelt-confined scans (`sandbox-exec`), and pointed the page at the resulting `all-reports.json`. Seatbelt cut sensitive reads 31→5 and (in the strict config) zeroed network — the page rendered the real baseline-vs-sandbox diff correctly. That exercise caught a renderer bug now fixed here (bf1451e): a task that runs and finds nothing emits the finding type with an empty value (e.g. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A static, client-side GitHub Pages page that presents the probe's reports so someone can (a) compare AI-agent sandboxes to make a trust decision, (b) track degradations/improvements as harness and supporting-tech versions move, and (c) absorb new harnesses with no code change. Functional pass only — styling is deliberately plain, ready to hand to design.
What's here
site/—index.html+ vanillaapp.js+ placeholderstyle.css. All derivation (fingerprint-collapse, baseline-normalization, categorization, flips, exposure) is client-side from one concatenated reports file; falls back tosample-data.jsonin dev.scripts/gen-sample-data.mjs— generates the dev fixture (history with real regressions/improvements to exercise every view).CONTEXT.md(glossary),docs/adr/0001-*(pipeline decision + why not Actions artifacts),docs/reporting-site-plan.md(build checklist).Extendability (holds by construction)
Not in this PR (tracked in the plan doc)
list-targets, the soft-seeder, and thescan-matrixaggregate/publish job. The site is developed against the fixture until those land.https://claude.ai/code/session_01PtFjYK8MdnDMmFYVWPDRrn