Add file-hash and document-ID recompute conformance track (B1b-3a) - #101
Merged
Conversation
Tier 3 part 1 of the document/part layer: the "declared vs computed"
half of State Machine section 5.4.2 row 319 ("File hash or document-ID
mismatch"). Until now the document mapper loaded and type-classified
parts but never checked that manifest.content.hash matched the stored
content bytes, nor that manifest.id matched the recomputed canonical
document ID, so a tampered or stale document loaded CLEAN.
The mapper now, inside the parseable-content branch:
- verifies the file-level content.hash against the exact stored bytes
(Document Hashing section 5.1), via a new centralized hashBytes helper
that reuses the same algorithm allowlist as computeDocumentId; and
- for a document carrying a real (non-pending) id, recomputes the
canonical document ID and compares it to manifest.id.
Both are WARNING in draft/review; the frozen/published INTEGRITY-ERROR
escalation is state-keyed (section 6.3) and, needing a projection-covering
signature to trust the state (section 5.4.2 note 3), is deferred to B3.
An unresolvable Dublin Core reference or an uncomputable id algorithm
(blake3) is treated as indeterminate, not a mismatch. Asset-hash (row
320), presentation-file hash, and the reference/missing-part rows are
deferred to B1b-3b; full content validity to B1b-3c.
Adds two errors.json codes (CDX-E-FILE-HASH-MISMATCH,
CDX-E-DOCUMENT-ID-MISMATCH; vocabulary v0.7), exposes raw part bytes on
a clean load in part-loader, adds 4 document fixtures plus independent
shared-nothing oracle confirmers (re-deriving the file hash and, for
transform-trivial content, the canonical id under a whitelist guard).
check:conformance 274/274; all 27 gates green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tier 3 part 1 of the document/part conformance layer — the "declared vs computed" half of State Machine §5.4.2 row 319 ("File
hashor document-ID mismatch"). Previously the document mapper loaded and type-classified parts but never verified thatmanifest.content.hashmatched the stored content bytes, nor thatmanifest.idmatched the recomputed canonical document ID, so a tampered or stale document loaded CLEAN.The mapper now, inside the parseable-content branch:
content.hashagainst the exact stored content bytes (Document Hashing §5.1), via a new centralizedhashByteshelper reusing the same algorithm allowlist ascomputeDocumentId.pending) id, recomputes the canonical document ID and compares it tomanifest.id.Both are WARNING in draft/review. The frozen/published INTEGRITY-ERROR escalation is state-keyed (§6.3) and, needing a projection-covering signature to trust the state (§5.4.2 note 3), is deferred to B3. An unresolvable Dublin Core reference or an uncomputable id algorithm (blake3) is treated as indeterminate, not a mismatch.
Also:
errors.jsoncodes —CDX-E-FILE-HASH-MISMATCH,CDX-E-DOCUMENT-ID-MISMATCH(vocabulary v0.7).part-loaderexposes the raw decompressed bytes on a clean load (hashing the exact stored bytes, not the re-encoded text).hashlib, and — for transform-trivial content, under a whitelist guard — the canonical id).Scope split: asset-hash (row 320) + presentation-file hash → 3b (shares fixture infra with the dangling-asset / missing-part rows); full content validity → 3c; frozen/published escalation → B3; MIME advisory (row 321) deferred.
Test plan
check:conformance— 274/274 (199 vectors + 75 fixtures)check:document-oracle— 47 injected defects confirmed + 5 clean docs (independent, shared-nothing)check:*,generate:template)/review-fixrun to completion (2 cycles → PASS; one IMPORTANT false-positive fixed and re-verified)