rvf: ADR-009 declares a 4096-byte root manifest normative, but the shipped writer never emits one

[ruvnet]

Found while building an independent RVF parser for rvQR against ADR-009 as the normative spec.

What I observed

A container produced by the shipped CLI (rvf create -d 16 -m cosine + rvf ingest, 24 vectors, 2304 bytes) contains:

• segment magic 53 46 56 52 at offsets 0, 162, 1960, 2092 (4 segments: MANIFEST, VEC, WITNESS, MANIFEST)
• root manifest magic 30 4D 56 52: zero occurrences
• the final 16 bytes are all zero

Why this matters

ADR-009 §2.1 states the Level-0 root manifest "is exactly 4096 bytes and occupies the final 4096 bytes of the latest manifest payload," and acceptance criterion 3 is "Tail scanning locates a valid manifest without reading a fixed header at offset zero." An independent implementor reading ADR-009 as normative — which §2.4 explicitly instructs — will write a reader that finds nothing in a real CLI-produced file.

The cause looks structural: there are two writer paths in the tree.

• rvf-wire + rvf-manifest own the 4096-byte Level0Root codec (manifest_codec.rs:172 write_root_manifest), tail discovery, and the golden vectors added in docs+feat(rvf): ADR-009 — RVF v1 wire contract, exact magic bytes, golden vectors, CI gate #769.
• rvf-runtime — the crate behind rvf-cli, which produces actual files — has its own write_manifest (store.rs:3582) emitting MANIFEST segments, and its Cargo.toml does not depend on rvf-wire or rvf-manifest at all. It has its own find_latest_manifest (read_path.rs:49) that scans for manifest segments, not a Level-0 root.

Both paths agree on the segment magic and 64-byte alignment, so they aren't in conflict there — but only one produces the root manifest ADR-009 describes as the source of truth.

Note a 2304-byte container is smaller than a single 4096-byte root manifest, so small files can't carry one by construction. That itself deserves to be stated in the spec.

Suggested resolution

Either:

1. Amend ADR-009 to describe both layouts — say plainly that the Level-0 root manifest is optional/absent in runtime-produced containers, that rvf-runtime is also a normative source, and how a reader should behave when no root manifest exists (fall back to the newest MANIFEST segment); or
2. Make rvf-runtime emit a root manifest so the shipped writer matches the declared contract.

(1) is cheaper and reflects reality; (2) makes the format uniform. Either way the ADR and the shipped writer should stop disagreeing — that inconsistency is precisely what ADR-009 was written to eliminate.

Golden vectors from #769 are unaffected: they test rvf-wire's codec, which is correct on its own terms.

🤖 Generated with claude-flow

[ruvnet]

Correction and escalation

Two things in my original report need fixing, and the real failure mode is worse than "finds nothing."

1. I was wrong that the paths agree on 64-byte alignment

Segment headers are 64 bytes. The segments are not 64-byte aligned in runtime containers:

offset	type	hdr+payload	next	offset % 64
0	Manifest	64+98	162	0
162	Vec	64+1734	1960	34
1960	Witness	64+68	2092	40
2092	Manifest	64+148	2304	44

The arithmetic settles it: (64+98)+(64+1734)+(64+68)+(64+148) = 2304 — exactly the file size, zero padding. Summing to the file size is proof of the opposite of alignment. rvf-wire/src/writer.rs pads each segment to a 64-byte boundary and records alignment_pad; rvf-runtime/src/write_path.rs:461 writes header and payload back to back and leaves alignment_pad at 0.

2. The failure is a silent wrong answer, not an error

rvf_wire::find_latest_manifest run against a real CLI container returns Ok at offset 0 — the stale epoch-0 manifest reporting 0 vectors — instead of the newest manifest at 2092 (epoch 1, 24 vectors).

The mechanism follows directly from the correction above: a 64-byte-stride backward scan from EOF visits 2304, 2240, 2176, 2112, 2048 … and never 2092, because 2092 % 64 = 44. It does land on offset 0, which is both aligned and a genuine MANIFEST segment. So an implementor following ADR-009 gets a reader that opens the file, reports an empty store, and raises no error. Silent data loss, not a visible failure.

3. ADR-009 §5 was actively harmful

It instructed readers to validate "the 64-byte alignment of the segment start" — which rejects every runtime segment. That has been removed in the amendment.

4. A third divergence neither of us listed: content-hash algorithms

rvf-runtime uses a legacy CRC32-rotation hash labelled checksum_algo = 0; rvf-wire's registry reads 0 as XXH3-128. Cross-path hash verification therefore fails. rvf-runtime/src/hashing.rs documents this in its own module docs.

Status

The ADR amendment now describes both container shapes as first-class, specifies a two-path reader (tail root manifest, else byte-wise backward scan for the newest MANIFEST segment), marks the root manifest optional, adds rvf-runtime as a normative source, and splits the acceptance criteria per shape. Verified implementable: the amended algorithm was coded literally from the ADR text and returns offset 2092, epoch 1, 24 vectors on the real container.

Two related items found but not fixed, both needing a decision rather than a guess:

• rvf-runtime/src/read_path.rs:1-8 module doc still describes a boot sequence ("seek to EOF - 4096, parse Level 0 root manifest") the module does not implement.
• docs/adr/ADR-026-rvcow-branching-and-real-cognitive-containers.md mandates a double-root scheme at EOF-8192/EOF-4096 with cross-hashes and a generation counter; rvf-runtime implements none of it. Same class of divergence — needs a call on whether RVCOW is live or abandoned.

🤖 Generated with claude-flow