Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions docs/support-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ scheme, metadata field, or producer quirk is accepted.
| ZIP/ZIP64 | seek or streaming | see [ZIP compression methods](#zip-compression-methods) grid | see grid | optional WinZip AES-256 AE-2; ZipCrypto not enabled by default | descriptors, ZIP64, Unicode path/comment, and extended/NTFS/Info-ZIP-UX timestamp extras are interpreted as typed metadata; Info-ZIP Unix uid/gid are surfaced when recorded in the central directory (as arca writes them) and synthesized on write; unknown extras are preserved verbatim |
| 7z | seek | LZMA/LZMA2, encoded headers, solid single-folder archives | yes | none (AES unsupported) | optional `sevenz`; multiple folders and general coder graphs are unsupported |
| ISO 9660 | seek | ISO 9660, Rock Ridge, Joliet | yes | none | UDF and continuation-area coverage are not complete |
| CAB | seek | read-only (MSCF): Store and MSZIP folders | no (read-only) | none | QUANTUM/LZX folders and cross-cabinet spanning are structured `Unsupported`; the MSZIP window is carried across a folder's `CFDATA` blocks |
| XAR | seek | read-only: stored and zlib (`x-gzip`) data | no (read-only) | none | zlib-XML TOC; `x-bzip2` and other data encodings are structured `Unsupported` |

### ZIP compression methods

Expand Down Expand Up @@ -62,14 +64,14 @@ read+write on portable.
| Portable **streaming** zstd encode | ZIP write method 93 on `portable-codecs` → structured `Unsupported`; `native-codecs` write works | `ruzstd` ships only a one-shot whole-buffer encoder (`ruzstd::encoding::compress_to_vec`, used for outer-filter frames and `create --zstd`). It cannot emit a single ZIP member as a bounded stream without buffering the whole member, which would break the core bounded-memory guarantee. The engine refuses the path rather than weaken the guarantee. | A streaming, single-stream pure-Rust zstd encoder — upstream to `ruzstd` or a dedicated crate the engine consumes. | RM-307 → follow-on codec initiative |
| Deflate64 (method 9) | ZIP method 9 read/write → structured `Unsupported` (not yet implemented) | Read: a mature pure-Rust decoder (`deflate64`) exists but is not yet wired. Write: no pure-Rust encoder exists and demand is effectively nil (matches libarchive). | Read: adopt the external `deflate64` decoder behind the codec-provider boundary in a follow-on slice. Write: **won't-do**, retired per [ADR-0013](adr/0013-rar5-udf-deflate64-feasibility.md). | RM-306 / ADR-0013 |

RAR5 and UDF read scope is resolved by
CAB and XAR are implemented as read-only seek-native providers (see the archive
containers table above). RAR5 and UDF read scope is resolved by
[ADR-0013](adr/0013-rar5-udf-deflate64-feasibility.md): UDF is a scoped read-only
go (revisions 1.02/1.50/2.01 first, activated from the shared Volume Recognition
Sequence), pending its implementation slice; RAR5 read is feasible in principle,
but the provider is deferred in its entirety until a clean-room pure-Rust
decompressor exists, because `.rar` archives in the wild are almost always
compressed. CAB and XAR remain read-only targets tracked by RM-305. All four are
not yet implemented; each cell flips only when its provider lands.
compressed.

## Outer compression filters

Expand Down
59 changes: 59 additions & 0 deletions docs/tracking/campaign-3-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ change, or versioned release candidate is part of this snapshot.
| ZIP LZMA (method 14) write | RM-302 | `libarchive_oxide/src/zip_stream.rs`, `libarchive_oxide/src/provider.rs` | `StreamZipMethod::Lzma` + `lzma_rust2::LzmaWriter::new_no_header` (raw LZMA1, EOS marker) drained through an in-crate `VecSink` (no trait object, `#![forbid(unsafe_code)]` intact); pinned preset 6 (props 93, 8 MiB dict); emits the 9-byte ZIP-LZMA header once at entry start; general-purpose bit 1 (`0x0002`) set in local+central flags outside the `0x0809` cross-check mask; version-needed 63 |
| ZIP LZMA interop + adversarial evidence | RM-302 | `libarchive_oxide/tests/interop_zip_lzma.rs`, `libarchive_oxide/tests/seek_stream_v2.rs`, `libarchive_oxide/tests/fixtures/zip/python-lzma/` | three producers (arca + first-party raw-LZMA1 builder, both `lzma-rust2`; + committed CPython 3.14.6/liblzma fixture, independent codec) × two consumers (arca, `zip@8.6.0` with `lzma`); WRITE evidence = the `zip` crate decodes arca's method-14 output byte-identically; round-trip, empty-member, truncation, bad-property-size, bomb, and feature-off Unsupported tests; the committed fixture + `generate.py` are byte-reproducible (SHA-256 recorded in `PROVENANCE.md`) |
| Support-matrix + PROVENANCE ZIP LZMA update | RM-302 | `docs/support-matrix.md`, `libarchive_oxide/tests/fixtures/zip/PROVENANCE.md` | ZIP row adds LZMA to Read and Write; the not-yet-implemented note now lists ONLY Deflate64; PROVENANCE records the committed-fixture escape hatch and the two-independent-codecs honesty note |
| CAB read-only provider | RM-305 | `libarchive_oxide/src/cab.rs`, `libarchive_oxide-core/src/format.rs`, `libarchive_oxide/src/provider.rs`, `libarchive_oxide/src/seek_stream.rs` | `CabSeekReader`: MSCF header, CFFOLDER/CFFILE/CFDATA tables, Store + MSZIP (the 32 KiB LZ window is carried across a folder's `CFDATA` blocks via a miniz_oxide wrapping ring); QUANTUM/LZX/cross-cabinet/spanning are structured `Unsupported`; registered as a read-only seek-native provider (`FormatId::Cab`, capability decode-only) |
| XAR read-only provider | RM-305 | `libarchive_oxide/src/xar.rs`, `.../format.rs`, `.../provider.rs`, `.../seek_stream.rs` | `XarSeekReader`: big-endian header, zlib TOC bounded by `metadata_bytes`, a hand-rolled bounded XML pull-scanner over the `<file>` tree, stored + zlib (`x-gzip`) heap data; `x-bzip2`/unknown encodings are structured `Unsupported`; `FormatId::Xar`, decode-only |
| CAB/XAR interop + adversarial evidence | RM-305 | `libarchive_oxide/tests/interop_cab_meta.rs`, `libarchive_oxide/tests/interop_xar_meta.rs`, `libarchive_oxide/tests/fixtures/{cab,xar}/PROVENANCE.md` | first-party in-code raw builders (independent DEFLATE/zlib via `flat2`) read back through the RM-301 harness; multi-file/nested/empty round trips plus unsupported-method and truncated-header structured-error negatives; a three-lens adversarial review with a verification pass |
| Support-matrix CAB/XAR read-only rows | RM-305 | `docs/support-matrix.md` | CAB and XAR added to the archive-containers table as read-only seek providers; removed from the not-implemented list (RAR5/UDF remain, tracked by RM-306) |
| RAR5/UDF/Deflate64 feasibility ADR | RM-306 | `docs/adr/0013-rar5-udf-deflate64-feasibility.md`, `docs/support-matrix.md` | Deflate64 read = go (adopt external pure-Rust `deflate64` behind the codec-provider boundary, follow-on slice) / write = won't-do; UDF read-only go (rev 1.02/1.50/2.01, follow-on); RAR5 deferred in its entirety (no clean-room pure-Rust decompressor); codec-deficit ledger + not-yet-implemented prose updated; no `src/` change, no dependency added |
| Metadata-fidelity harness extension | RM-304 | `libarchive_oxide/tests/common/mod.rs` | additive `read_seq_with_arca` (sequential `ArchiveReader`), `MetaShape` (REAL kind + mode/uid/gid/mtime/link_target, no kind folding), `read_meta_seq_with_arca` / `read_meta_seek_with_arca`, `assert_producers_agree_seq`; the content-only `EntryShape` path is unchanged |
| tar producer corpus + metadata round trip | RM-304 | `libarchive_oxide/tests/interop_tar_meta.rs`, `libarchive_oxide/tests/fixtures/tar/PROVENANCE.md` | 3 producers (arca, `tar@0.4`, first-party raw ustar builder) × 2 consumers (arca sequential reader, `tar@0.4`); mode/uid/gid/mtime and symlink-target fidelity |
Expand Down Expand Up @@ -329,6 +333,61 @@ change, or versioned release candidate is part of this snapshot.
architecture-and-documentation slice establishing the contract that RM-302..306
and future codec work inherit.

## RM-305

- CAB and XAR are arca's FIRST read-only built-in seek-native providers: every
prior built-in format (ZIP, 7z, ISO, tar, cpio, ar) is read+write, so RM-305
also proves the read-only registration path. Both decode through the always-on
`miniz_oxide` codec, so they need no feature gate and stay pure-Rust / C-free on
the portable profile.
- Registration (five points, all in the scaffold): `FormatId::Cab` / `FormatId::Xar`
(`libarchive_oxide-core/src/format.rs`) with `MSCF` / `xar!` probe signatures;
`format_capability` → `FormatCapabilities::new(true, false, true)` (decode yes,
encode NO, seek yes) and `format_name` in `provider.rs`; a `SeekDispatch::Cab` /
`SeekDispatch::Xar` arm plus signature detection in `SeekArchiveReader::open`; and
`mod cab` / `mod xar` in `lib.rs`. Each reader implements the same
`new`/`next_event`/`skip_entry`/`into_inner`/`source_ref` contract as the 7z
reader and drives the identical `Idle → Entry → Data(≤64 KiB) → EndEntry → Done`
phase machine.
- **CAB** (`cab.rs`): a bounded MSCF parser. The `CFHEADER` (with optional reserve
and prev/next-cabinet strings), the `CFFOLDER` and `CFFILE` tables, and the
per-folder `CFDATA` blocks are all walked within the `Limits` budget. A folder is
a solid unit decoded one `CFDATA` block at a time — no whole folder is
materialized. Store copies bytes through; MSZIP verifies the `CK` block magic and
inflates each block's raw DEFLATE while carrying the folder's 32 KiB LZ77 window
across blocks via a fresh `DecompressorOxide` over a preserved power-of-two
wrapping ring (so a back-reference in block N resolves into block N−1's output).
QUANTUM/LZX methods, cross-cabinet continuation files (`iFolder` sentinels), and
spanning blocks (`cbUncomp == 0`) list their metadata then surface a structured
`Unsupported` on the payload.
- **XAR** (`xar.rs`): the 28-byte big-endian header locates the zlib-compressed TOC
(inflated with `DataFormat::Zlib`, capped at `min(toc_length_uncompressed,
metadata_bytes)` and length-verified) and the heap (`heap_start = header.size +
toc_length_compressed`, never a hard-coded 28). A hand-rolled bounded XML
pull-scanner (an explicit DFS `<file>`-frame stack, no DOM, no XML crate) resolves
each entry's `/`-joined path, emitting a directory before its children, bounded by
`entries` / `path_bytes` / a nesting-depth cap. Regular files stream their heap
blob per entry (seek-per-blob for the unordered/shared-heap model) as stored
(`octet-stream`, `S == L` enforced) or zlib (`x-gzip`, RFC-1950 — decoded as zlib,
NOT gzip), with the decoded length verified against `<length>`. `x-bzip2` and any
unknown encoding are structured `Unsupported`; version ≠ 1 is `Unsupported`.
- Write is rejected without a code path of its own: the decode-only capability makes
`format_encoder` return a typed `Capability` error, and `SeekArchiveWriter::with_format`
falls through its default arm to `Unsupported` for `FormatId::Cab`/`Xar`.
- `#![forbid(unsafe_code)]`, the `no-dyn` gate, and bounded memory hold: both readers
are plain generic structs over `R: Read + Seek` with concrete enums (no trait
objects), stream payload in ≤ 64 KiB chunks, and return a structured `StreamError`
for every malformed/truncated/unsupported/limit case (never a panic/unwrap).
- Evidence: `tests/interop_cab_meta.rs` and `tests/interop_xar_meta.rs` build valid
archives with first-party in-code raw byte builders (independent DEFLATE/zlib via
`flat2`, which arca re-inflates with `miniz_oxide` — codec independence), read
them back through the RM-301 harness, and assert content round trips over
multi-file/nested/empty corpora, plus negatives for an unsupported compression
method and a truncated/out-of-range header. A three-lens adversarial review
(panic/bounds, spec-correctness, malformed-input) with an independent verification
pass was run over both modules before commit. Per-format `PROVENANCE.md` registers
the in-code raw builder and documents the external independent producers
(makecab/gcab/cabextract for CAB; the `xar` CLI / `bsdtar --format=xar` for XAR).
## RM-304

- RM-304 lifts the RM-301 interoperability harness from content-only evidence to
Expand Down
8 changes: 8 additions & 0 deletions libarchive_oxide-core/src/format.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ pub enum FormatId {
SevenZip,
/// ISO 9660.
Iso9660,
/// Microsoft Cabinet (read-only).
Cab,
/// XAR extensible archive (read-only).
Xar,
}

impl FormatId {
Expand All @@ -42,6 +46,8 @@ impl FormatId {
Self::SevenZip,
[0x37, 0x7a, 0xbc, 0xaf, 0x27, 0x1c].as_slice(),
),
(Self::Cab, b"MSCF".as_slice()),
(Self::Xar, b"xar!".as_slice()),
] {
if prefix.len() >= signature.len() && prefix.starts_with(signature) {
return ProbeResult::Match(identifier);
Expand Down Expand Up @@ -69,6 +75,8 @@ impl FormatId {
b"PK\x03\x04".as_slice(),
b"PK\x05\x06".as_slice(),
[0x37, 0x7a, 0xbc, 0xaf, 0x27, 0x1c].as_slice(),
b"MSCF".as_slice(),
b"xar!".as_slice(),
] {
if prefix.len() < signature.len() && signature.starts_with(prefix) {
minimum = minimum.min(signature.len());
Expand Down
Loading