[cloister-667ea6] build-cache/v1 spec + conformance harness (stacked on #83)#84
Closed
jamestexas wants to merge 2 commits into
Closed
[cloister-667ea6] build-cache/v1 spec + conformance harness (stacked on #83)#84jamestexas wants to merge 2 commits into
jamestexas wants to merge 2 commits into
Conversation
… (build-cache/v1) The OCI Distribution v1.1 push surface verifies every body against the client's claimed `sha256:<hex>` digest. The cloister-spec build-cache/v1 capability deliberately reuses that `sha256:` prefix to carry BLAKE3 bytes — per Σ §3.4 the LLO substrate is BLAKE3- locked, but ORAS / cosign / `oras-go` clients (which mache cache push --remote will use) only know how to mint `sha256:` digests. Verify-on-write now tries SHA-256 first (OCI-native fast path — Docker, ORAS, cosign hit this; no @noble/hashes work) and falls back to BLAKE3 if the SHA-256 check fails (build-cache/v1 clients). A collision in either algorithm is infeasible, so accepting the union does not weaken integrity. Failure responses name both computed hashes so client diagnostics show what cloister actually saw. Three verify-on-write call sites refactored through a single `verifyClaimedDigest()` helper (monolithic upload, chunked-upload finalize, manifest-by-digest PUT). Test: oci-registry-push.test.ts now covers the build-cache/v1 round-trip — push body with BLAKE3-digested claim, expect 201. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ss + caller-provided storage key Closes the conformance loop opened by cloister-4d376e (PR #83). Three pieces: 1. **cloister-spec/build-cache/v1/** — vendor-neutral spec authored per cloister-spec/LAYOUT.md. README.md covers wire summary, digest-encoding decision (BLAKE3-in-sha256:), media-type namespace (application/vnd.cloister.build-cache.v1.*), and the conformance contract ("round-trip byte-equality on every vector in vectors/"). Vectors generated deterministically by LLO's gen_build_cache_vectors example (rs/ll-core/schema-capnp/examples/) — chunk-001.bin, chunk-002.bin, lockfile-config.bin, manifest.json, plus digests.json (per-file BLAKE3 + SHA-256) and VECTORS.sha256 (integrity sidechannel). 2. **Caller-provided storage key on BlobStore.put** — fixes a latent bug in PR #83. The OCI route was verifying body-against-BLAKE3-claim and returning 201, but `blob.put(body)` always stored under SHA-256 internally. A subsequent `GET /v2/.../blobs/sha256:<BLAKE3-hex>` then 404'd because the lookup key didn't match the storage key. `BlobStore.put` now accepts an optional `key: Digest` that overrides the default content-addressed key, used by the OCI route to honor the build-cache/v1 wire (the route has already verified the body matches the key under either SHA-256 or BLAKE3, so the substrate's content-addressed invariant holds under the union). All three verify-on-write sites updated (monolithic upload, chunked finalize, manifest PUT by digest) — the manifest PUT was also still on the SHA-256-only path in PR #83, which this PR also migrates through verifyClaimedDigest(). 3. **test/routes/oci-build-cache-conformance.test.ts** — the conformance harness. Loads each vector (base64-inlined; cloudflare:test pool has no fs), asserts BLAKE3 integrity against the committed digests.json, then runs the v1 round-trip contract: push every blob under sha256:<blake3-hex>, pull back, assert byte-equal. Plus an end-to-end consumer walk: push everything, then fetch manifest → parse → fetch config + each layer like a real `mache cache fetch` consumer would. 9 tests, all green. Verification: task lint → 1136/1136 tests pass (was 1127, +9 conformance) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced May 24, 2026
jamestexas
changed the base branch from
feat/build-cache-v1-server-4d376e
to
main
Open
2 tasks
Contributor
Author
|
Superseded by #90 (consolidated build-cache/v1 storage-key + hardening). Closing to reduce review load. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #83. Merge that one first.
Closes cloister-667ea6 — completes the conformance loop for the build-cache/v1 capability.
Summary
Three changes:
cloister-spec/build-cache/v1/— vendor-neutral spec authored percloister-spec/LAYOUT.md. Covers wire summary, the BLAKE3-in-sha256:digest-encoding decision, media-type namespace, and the conformance contract. Vectors generated deterministically by LLO'sgen_build_cache_vectorsexample.Caller-provided storage key on
BlobStore.put— fixes a latent bug in [cloister-4d376e] BLAKE3-in-sha256 digest fallback for build-cache/v1 push #83. The OCI route verified BLAKE3 claims and returned 201, butblob.put(body)always stored under SHA-256 internally → subsequent pulls bysha256:<BLAKE3>404'd.put()now accepts an optionalkeyso the OCI route can honor the build-cache/v1 wire end-to-end. All three verify-on-write sites updated (monolithic upload, chunked finalize, manifest PUT). The manifest PUT was also still on the SHA-256-only path in [cloister-4d376e] BLAKE3-in-sha256 digest fallback for build-cache/v1 push #83; this PR migrates it throughverifyClaimedDigest().test/routes/oci-build-cache-conformance.test.ts— the conformance harness. Loads each vector (base64-inlined; cloudflare:test has no fs), asserts BLAKE3 integrity against committeddigests.json, then runs push → pull → byte-equal for every blob, plus an end-to-end consumer walk (manifest → config → layers).Why this matters
PR #83 proved cloister could accept the wire shape. This PR proves cloister implements the build-cache/v1 capability — the difference between "shape is right" and "behavior is interoperable with the spec's reference producer (LLO)". The conformance test is the falsifiable acceptance criterion: it walks the same path
mache cache push --remotewill walk once mache-aeb262 Phase 3 lands.Test plan
task lint→ 1136/1136 tests pass (was 1127, +9 conformance)cloister-spec/build-cache/v1/vectors/round-trips byte-equal through cloisterBlobStore.putdefault behavior unchanged whenkeyomitted🤖 Generated with Claude Code