Wire the Static Web Bundle producer into the authoring build lane - #1252
Open
Koh0920 wants to merge 30 commits into
Open
Wire the Static Web Bundle producer into the authoring build lane#1252Koh0920 wants to merge 30 commits into
Koh0920 wants to merge 30 commits into
Conversation
- Add additive optional [outputs] / [outputs.static_web] to CapsuleManifestV1. - Validate root (non-empty relative, no absolute/./../backslash), entry_path (normalized relative), and connect_src (public https/wss origins, ASCII sorted, unique) — path existence is a Build Attempt fact, not a parse fact. - Absent [outputs] keeps snapshot-only behavior; a dist/ directory never selects the static web lane by itself.
…ring PR 4 — connect the pure producer (PR 1) and the API registry (PR 3) through the builder claim, with an explicit no-op for un-declared output: - authoring_runtime: accept the Build Config Revision plan fields the API claim already sends (build_config_revision_id, plan_digest, effective_build_plan, …) — the claim previously carried them but the deny_unknown_fields struct refused them. Add AuthoringWork::static_web_output_plan to surface the declared [outputs.static_web] section. - static_web_output: StaticWebOutputPlan::from_effective_build_plan_json — a declared section is authoritative (unsafe root etc. fails the build); absence is a snapshot-only no-op. A dist/ directory never selects the lane. - static_web_transport: prepare → batch upload-authorizations → parallel PUT (fresh URL per retry, URL never logged/persisted, digest-derived keys asserted) → verify → complete, mirroring source_archive_upload's security model. Builder holds no storage credential. - static_web_emit: the only place the three outcomes are decided (skip / STATIC_WEB_OUTPUT_MISSING / STATIC_WEB_BUNDLE_FAILED) and the bundle is uploaded. - 15 new tests: transport no-reuse/redaction/conflict, claim plan parsing, emit no-op/missing-root/upload. The Firecracker/Docker image-export call site (feeding the built image tree to emit_static_web) is the remaining rollout seam; the plan parsing, transport, and decision logic are fully covered here.
…SP origins Review fixes on the pure producer + contracts: - extract_static_web_output: an intermediate symlink (e.g. image_root/srv -> outside) could escape the image root because only the final component was checked. Every component of image_output_root is now verified as a real directory, the canonical source must stay strictly beneath the canonical image root, and copy_tree_no_links re-canonicalizes each entry (TOCTOU guard). Tests: intermediate/absolute/relative-.. symlink escapes and an in-tree symlink all refused. - validate_relative_path: ANY literal `%` is now refused, matching the worker's multi-decode behavior (percent-encoded separators/traversal can no longer pass the producer and fail the consumer). New invalid fixture cases. - manifest validation: one blob digest may back many paths but every occurrence must agree on size (the R2 object has one size); receipt validate_for_manifest no longer hides a digest-size disagreement via or_insert. - produce_static_web_bundle: files are opened once, read bounded at MAX_FILE_SIZE+1 (no unbounded builder memory), and re-stated through the SAME descriptor after the read (fstat inode+size on unix) so the manifest size, receipt size, and hashed bytes are one fact. - frame_ancestors: the fixed v1 trust set is now the REAL embedding origins: https://ato.run, https://app.ato.run, https://staging.store.ato.run, https://stg-app.ato.run. staging.ato.run has no DNS record (ato-api staging config), so it is removed; a frame-ancestor that can never frame is a false promise in the policy. All golden JCS fixtures, expected digests, and p/s host labels regenerated in lockstep.
The API only emits the build-plan extension fields (effective_build_plan, plan_digest, authored TOML, revision binding) to a builder that advertised 'static-web-bundle-v1'; a legacy builder keeps receiving the byte-compatible claim it can parse (deny_unknown_fields would hard-fail otherwise). Rollout is builder-first, API-second. - ClaimRequest now carries supported_features; claim() advertises static-web-bundle-v1, claim_with_features() is the test seam. - Tests: the legacy claim shape (no plan fields) still parses, and the capability is advertised on the wire.
…uild (B1) The vertical slice's production path is now wired end to end: - build_v1's clean_replay exports the assembled guest filesystem to <work_root>/guest-rootfs (existing v1 seam). After the rootfs persists, the builder calls static_web_output_plan() and — when the saved Build Config Revision declared [outputs.static_web] — runs emit_static_web_if_declared against the EXPORTED TREE with the real HTTP transport (prepare → batch upload → verify → complete). Absent declaration = complete no-op; a DECLARED output that fails = build failure (never a silent snapshot fallback). - rootfs_pack_script gains an optional export_to that retains a tar copy of the exported tree before cleanup (used by future lanes; legacy builds pass None and stay byte-identical). - Test: the call-site shape (guest-rootfs/app/dist tree + declared plan in the work JSON + revision id/plan digest) emits + uploads + completes. This is the seam the real Docker/Firecracker build exercises; the rollout E2E (Hextris: build -> dist -> bundle -> upload -> s-* URL) completes the acceptance from here.
…l site Blocker 1 + Major 1 from re-review: - ProducedBuild now carries exported_guest_rootfs, set by the v1 lane to the ACTUAL tree the producer exported before packing (`<work_root>/guest-rootfs`). The clean_replay static-web call site consumes that threaded path instead of reconstructing a magic `<jobdir>/v1-work/guest-rootfs` literal — a declared output on a lane that produced no exported tree now fails closed with a clear error instead of silently finding nothing. - Removed the dead `export_to` param added to rootfs_pack_script in the prior round: it was never wired to any caller (the v1 lane exports natively as a required producer step), so it only suggested a mechanism that did not exist. rootfs_builder + docker_import pack scripts are back to byte-identical pre-round output. - extract_static_web_output component walk now accumulates the path IN ORDER (image_root/a -> image_root/a/a -> ...), so duplicate component names like a/a/dist can no longer skip a symlinked later component (the previous take_while stopped at the first match). New test pins the duplicate-name escape case.
- manifest_v1: outputs.static_web.root, entry_path, source paths, and asset paths now reject '%' — the producer's contract validate_relative_path already refused it, so a manifest accepted by the authoring surface could previously fail at build time. API plan validation matches (Major 3). - static_web_transport: a 412 Precondition Failed from the create-only blob PUT means the digest key ALREADY EXISTS (a crashed earlier run may have landed the bytes before verify). The uploader now treats 412 as success and proceeds to VERIFY instead of burning the retry budget (Major 2). Test: a transport that always 412s converges through verify to complete.
…claim on every call, closure-safe verify
…unt/unknown/duplicate skew) (review round 3)
…ation validator (review round 4)
- manifest v1: optional [outputs.static_web] (root/entry_path/spa_fallback/ connect_src) matching the wizard API's staticWebOutputSchema constraints, omitted from serialization when absent so existing manifests round-trip byte-identically; validation reuses the static-web delivery contract's path/origin validators - detector: infer_static_web_outputs emits the declaration only for high-confidence static repos (root index.html with no server entrypoint; the plain vite build/preview production shape with a readable outDir), fails closed everywhere else; run + outputs coexist - clean-build lane: when the claim's Build Config Revision resolved to static_web, extract the DECLARED output root from the exported guest rootfs, produce the Static Web Bundle, scan it, and register it via the idempotent prepare -> upload -> verify -> complete registry flow; any producer failure fails the build attempt with the contract's structured static_web_* code - claim: advertise the static-web-bundle-v1 capability and parse the plan-extension + publication claim fields Claude-Session: https://claude.ai/code/session_013NV566T4RuF8QPR37mHEgh
Use effective_build_plan.static_web_output.materialization_id verbatim when it is registry-shaped, so the registry row and the effective build plan name the same materialization; keep the local bcrev-keyed derivation as the fallback for plans that carry none. Claude-Session: https://claude.ai/code/session_013NV566T4RuF8QPR37mHEgh
…lder The staging builder binary was already built from feat/static-web-publication-pr2 (unmerged): a battle-tested static web producer — lease-fenced transport, bundle emit, declared build-step execution, asset media-type coverage — that this branch had independently re-implemented against main. pr2's machinery is the production path and is kept whole. What survives from this branch is the Publication Lane contract layered on top: - AuthoringWork carries the server-resolved publication block (tolerant parse; absent or unknown means snapshot_compute) plus is_static_web_lane(). The LANE is server-owned: a declared outputs section alone never selects it. pr2's static_web_output_plan() continues to supply the plan itself. - The detector's outputs inference and the structured static_web_* failure codes ride on pr2's producer paths. - The manifest keeps pr2's Option<OutputsV1> shape; this branch's call sites were adapted rather than the type changed. cargo check clean; static_web 30, capsule manifest 355, authoring_runtime 36, capsule authoring_intent 8 — all green.
The reconciliation merge took the shipped branch's manifest validator
wholesale, which dropped this branch's acceptance of the whole-string
root = "." — the form the detector emits for dependency-free static repos
(a root index.html with no build). The API's zod schema accepts it, so the
two sides disagreed and every plain-static build failed at detect with
"outputs.static_web.root is invalid" (caught on staging by the S02-A
fixture, not by unit tests).
"." as a path SEGMENT ("./dist", "dist/.", "a/../b") stays rejected. The
new test pins both halves so a future merge cannot silently drop it again.
The #443 precedence rule — a package.json always wins over a root index.html — is right for bundler-served apps, but it also swallowed every static site that uses npm purely for tooling. Those repos declare no start or dev script, so package-managed inference dead-ends on "requires scripts.start or scripts.dev" and the app can never be published. Measured on the real publication backlog: gridgarden, flexboxfroggy and clumsy-bird ship an empty scripts object, and jspaint declares only electron packaging and release scripts. All four are static sites; all four failed detection. The rule now yields only for the exact decidable combination "root index.html AND no serving script at all" (start / dev / serve / preview). Every app #443 was written for still declares one of those, so it keeps the package-managed path; a malformed or unreadable package.json fails closed to package-managed.
Two failures made `root = "."` — the dependency-free static case the detector emits for a plain index.html repo — impossible to publish. Both were found on staging with 2048: "STATIC_WEB_BUNDLE_FAILED: unsupported static web media type: .gitignore". Repository housekeeping is now excluded rather than served. Any dot-prefixed path segment (.gitignore, .github/, .DS_Store) and the extension-less legal files every repo ships (LICENSE, COPYING, NOTICE, AUTHORS, CHANGELOG, Makefile) are dropped from the bundle. A hidden entry has no place on a public edge surface, and the exclusion is deliberately narrow so an unknown media type on a REAL asset still fails loudly instead of vanishing from the site. The media-type table also lacked things ordinary static sites ship: docs served next to the app (md, csv, xml, pdf) and the audio/video a browser game needs (mp3, ogg, wav, m4a, flac, mp4, webm) plus glTF models. Every addition is inert; executable and unknown types keep failing. Tests: 3 for the housekeeping predicate (dotfiles excluded, extension-less metadata excluded, real content — including README.md and favicon.ico — kept); existing static_web suites 33 + 8 green.
Enumerating repository metadata names fell over on the second real repository: 2048 ships a Rakefile. Web content is addressed by extension and the edge has no media type for an extension-less file, so treat the absence of an extension as the signal — LICENSE, Makefile, Rakefile, Dockerfile, Procfile, CNAME all drop out by the same rule. Assets with a real but unrecognized extension still fail loudly.
The bundle rule was enumerate-and-fail: any file whose extension had no media type killed the whole publish. That is right for a build output directory and wrong for root = ".", where the tree also holds everything the site was built from. 2048 alone disproved it three times in a row — .gitignore, then Rakefile, then style/helpers.scss. The media-type table is now the definition of publishable content: no entry means no Content-Type, which means the edge could not serve the file even if it were bundled, so it is excluded rather than fatal. Hidden entries stay excluded outright. Source maps remain a hard failure — they ARE servable, which is exactly why publishing one by accident must not be silent.
Adding markdown, audio, video and PDF to the emitter produced bundles the
v1 manifest validator then rejected outright ("media type is not in the
v1 allowlist: text/markdown"). is_allowed_media_type is a frozen contract
shared with ato-api, ato-edge and ato-usercontent-static, so the emitter
has to stay a subset of it; widening it is a coordinated v2 change.
Consequence worth stating plainly: a static site's audio, video, markdown
and PDF assets are excluded from a v1 bundle rather than published. Games
that need sound cannot be served through the static lane today.
…pping them A visually rich page (WebGL games — first hit: HexGL) produces PNGs over the 500KB ack cap at full size; the capture was skipped entirely, and the Ready-State Seal's REQUIRED post-restore screenshot then failed the whole seal. Oversized captures are now downscaled in x0.8 steps (floor 640px wide) and re-encoded until they fit; only a floor-width still-oversized capture keeps the old skip behavior.
…ead of refusing Real repositories carry internal links (first hit: jspaint's lib/tracky-mouse/website/core, a directory link to a sibling that is also walked via its real path); the extractor refused ANY link and failed the whole Build Attempt. Links whose canonical target stays inside the image root are now materialized as real content; links escaping the root stay fail-closed, and a link back to a directory on the current recursion stack (a true cycle) fails closed instead of recursing forever. The extracted tree still never contains a link.
…output Dependency provisioning installs node_modules INTO the image, so a root='.' static output swept it up — thousands of package files plus tool-generated dangling links (node_modules/.vite) that failed canonicalization and the whole Build Attempt. node_modules is now a system exclusion of the static output walk (same spirit as the source filter's .git), and a dangling link is skipped with a log line — it materializes nothing and can leak nothing. Cycles and escaping links stay fail-closed.
…he build With root='.' the walked tree is the repository itself, and repos ship committed .map files (jspaint: lib/98.css/98.custom-build.css.map). A source map is simply not publishable output — skip it like every other non-web file rather than failing the whole Build Attempt.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 3/3 of the Static/Compute publication platform (contract: ato-api#502, PWA: ato-pwa#278).
[outputs.static_web](root / entry_path / spa_fallback / connect_src), deny_unknown_fields, TOML round-trip, validation shared with the static-web contract;root="."= source root (aligned with the API zod schema).outputs.static_webfor high-confidence static repos (root index.html without server entrypoints → root="."; plain vite-build shape → literalbuild.outDiror dist, refusing unparseable overrides / server frameworks). Authored manifests and overlays untouched; dist/-existence alone never flips static.publication.resolved_publication_lane == "static_web"(inside the existingstatic-web-bundle-v1capability), the clean_replay lane extracts the DECLARED output root from the exported guest rootfs, produces the Static Web Bundle (deterministicswm_<base32-sha256>materialization id per Build Config Revision — idempotent re-runs converge on the registry), and drives prepare → upload → verify → complete with the authoring lease. Producer failure fails the build attempt with structured codes (static_web_output_missing / _outside_workspace / _entry_missing / _bundle_invalid / _secret_detected / _prepare_failed / _upload_failed / _finalize_failed). Secret scan = canary scan + PEM/AKIA text scan.supported_features: ["static-web-bundle-v1"].Tests: capsule manifest 355 green (4 new), snapshot-builder static_web 11 green, authoring_runtime 31 green; cargo check clean.
🤖 Generated with Claude Code
https://claude.ai/code/session_013NV566T4RuF8QPR37mHEgh