diff --git a/docs/support-matrix.md b/docs/support-matrix.md index 1209ff5..ce90285 100644 --- a/docs/support-matrix.md +++ b/docs/support-matrix.md @@ -14,6 +14,8 @@ scheme, metadata field, or producer quirk is accepted. | ZIP/ZIP64 | seek or streaming | see [ZIP compression methods](#zip-compression-methods) grid | see grid | optional WinZip AES-256 AE-2; ZipCrypto not enabled by default | descriptors, ZIP64, Unicode path/comment, and extended/NTFS/Info-ZIP-UX timestamp extras are interpreted as typed metadata; Info-ZIP Unix uid/gid are surfaced when recorded in the central directory (as arca writes them) and synthesized on write; unknown extras are preserved verbatim | | 7z | seek | LZMA/LZMA2, encoded headers, solid single-folder archives | yes | none (AES unsupported) | optional `sevenz`; multiple folders and general coder graphs are unsupported | | ISO 9660 | seek | ISO 9660, Rock Ridge, Joliet | yes | none | UDF and continuation-area coverage are not complete | +| CAB | seek | read-only (MSCF): Store and MSZIP folders | no (read-only) | none | QUANTUM/LZX folders and cross-cabinet spanning are structured `Unsupported`; the MSZIP window is carried across a folder's `CFDATA` blocks | +| XAR | seek | read-only: stored and zlib (`x-gzip`) data | no (read-only) | none | zlib-XML TOC; `x-bzip2` and other data encodings are structured `Unsupported` | ### ZIP compression methods @@ -62,14 +64,14 @@ read+write on portable. | Portable **streaming** zstd encode | ZIP write method 93 on `portable-codecs` → structured `Unsupported`; `native-codecs` write works | `ruzstd` ships only a one-shot whole-buffer encoder (`ruzstd::encoding::compress_to_vec`, used for outer-filter frames and `create --zstd`). It cannot emit a single ZIP member as a bounded stream without buffering the whole member, which would break the core bounded-memory guarantee. The engine refuses the path rather than weaken the guarantee. | A streaming, single-stream pure-Rust zstd encoder — upstream to `ruzstd` or a dedicated crate the engine consumes. | RM-307 → follow-on codec initiative | | Deflate64 (method 9) | ZIP method 9 read/write → structured `Unsupported` (not yet implemented) | Read: a mature pure-Rust decoder (`deflate64`) exists but is not yet wired. Write: no pure-Rust encoder exists and demand is effectively nil (matches libarchive). | Read: adopt the external `deflate64` decoder behind the codec-provider boundary in a follow-on slice. Write: **won't-do**, retired per [ADR-0013](adr/0013-rar5-udf-deflate64-feasibility.md). | RM-306 / ADR-0013 | -RAR5 and UDF read scope is resolved by +CAB and XAR are implemented as read-only seek-native providers (see the archive +containers table above). RAR5 and UDF read scope is resolved by [ADR-0013](adr/0013-rar5-udf-deflate64-feasibility.md): UDF is a scoped read-only go (revisions 1.02/1.50/2.01 first, activated from the shared Volume Recognition Sequence), pending its implementation slice; RAR5 read is feasible in principle, but the provider is deferred in its entirety until a clean-room pure-Rust decompressor exists, because `.rar` archives in the wild are almost always -compressed. CAB and XAR remain read-only targets tracked by RM-305. All four are -not yet implemented; each cell flips only when its provider lands. +compressed. ## Outer compression filters diff --git a/docs/tracking/campaign-3-evidence.md b/docs/tracking/campaign-3-evidence.md index 883b813..b30f267 100644 --- a/docs/tracking/campaign-3-evidence.md +++ b/docs/tracking/campaign-3-evidence.md @@ -39,6 +39,10 @@ change, or versioned release candidate is part of this snapshot. | ZIP LZMA (method 14) write | RM-302 | `libarchive_oxide/src/zip_stream.rs`, `libarchive_oxide/src/provider.rs` | `StreamZipMethod::Lzma` + `lzma_rust2::LzmaWriter::new_no_header` (raw LZMA1, EOS marker) drained through an in-crate `VecSink` (no trait object, `#![forbid(unsafe_code)]` intact); pinned preset 6 (props 93, 8 MiB dict); emits the 9-byte ZIP-LZMA header once at entry start; general-purpose bit 1 (`0x0002`) set in local+central flags outside the `0x0809` cross-check mask; version-needed 63 | | ZIP LZMA interop + adversarial evidence | RM-302 | `libarchive_oxide/tests/interop_zip_lzma.rs`, `libarchive_oxide/tests/seek_stream_v2.rs`, `libarchive_oxide/tests/fixtures/zip/python-lzma/` | three producers (arca + first-party raw-LZMA1 builder, both `lzma-rust2`; + committed CPython 3.14.6/liblzma fixture, independent codec) × two consumers (arca, `zip@8.6.0` with `lzma`); WRITE evidence = the `zip` crate decodes arca's method-14 output byte-identically; round-trip, empty-member, truncation, bad-property-size, bomb, and feature-off Unsupported tests; the committed fixture + `generate.py` are byte-reproducible (SHA-256 recorded in `PROVENANCE.md`) | | Support-matrix + PROVENANCE ZIP LZMA update | RM-302 | `docs/support-matrix.md`, `libarchive_oxide/tests/fixtures/zip/PROVENANCE.md` | ZIP row adds LZMA to Read and Write; the not-yet-implemented note now lists ONLY Deflate64; PROVENANCE records the committed-fixture escape hatch and the two-independent-codecs honesty note | +| CAB read-only provider | RM-305 | `libarchive_oxide/src/cab.rs`, `libarchive_oxide-core/src/format.rs`, `libarchive_oxide/src/provider.rs`, `libarchive_oxide/src/seek_stream.rs` | `CabSeekReader`: MSCF header, CFFOLDER/CFFILE/CFDATA tables, Store + MSZIP (the 32 KiB LZ window is carried across a folder's `CFDATA` blocks via a miniz_oxide wrapping ring); QUANTUM/LZX/cross-cabinet/spanning are structured `Unsupported`; registered as a read-only seek-native provider (`FormatId::Cab`, capability decode-only) | +| XAR read-only provider | RM-305 | `libarchive_oxide/src/xar.rs`, `.../format.rs`, `.../provider.rs`, `.../seek_stream.rs` | `XarSeekReader`: big-endian header, zlib TOC bounded by `metadata_bytes`, a hand-rolled bounded XML pull-scanner over the `` tree, stored + zlib (`x-gzip`) heap data; `x-bzip2`/unknown encodings are structured `Unsupported`; `FormatId::Xar`, decode-only | +| CAB/XAR interop + adversarial evidence | RM-305 | `libarchive_oxide/tests/interop_cab_meta.rs`, `libarchive_oxide/tests/interop_xar_meta.rs`, `libarchive_oxide/tests/fixtures/{cab,xar}/PROVENANCE.md` | first-party in-code raw builders (independent DEFLATE/zlib via `flat2`) read back through the RM-301 harness; multi-file/nested/empty round trips plus unsupported-method and truncated-header structured-error negatives; a three-lens adversarial review with a verification pass | +| Support-matrix CAB/XAR read-only rows | RM-305 | `docs/support-matrix.md` | CAB and XAR added to the archive-containers table as read-only seek providers; removed from the not-implemented list (RAR5/UDF remain, tracked by RM-306) | | RAR5/UDF/Deflate64 feasibility ADR | RM-306 | `docs/adr/0013-rar5-udf-deflate64-feasibility.md`, `docs/support-matrix.md` | Deflate64 read = go (adopt external pure-Rust `deflate64` behind the codec-provider boundary, follow-on slice) / write = won't-do; UDF read-only go (rev 1.02/1.50/2.01, follow-on); RAR5 deferred in its entirety (no clean-room pure-Rust decompressor); codec-deficit ledger + not-yet-implemented prose updated; no `src/` change, no dependency added | | Metadata-fidelity harness extension | RM-304 | `libarchive_oxide/tests/common/mod.rs` | additive `read_seq_with_arca` (sequential `ArchiveReader`), `MetaShape` (REAL kind + mode/uid/gid/mtime/link_target, no kind folding), `read_meta_seq_with_arca` / `read_meta_seek_with_arca`, `assert_producers_agree_seq`; the content-only `EntryShape` path is unchanged | | tar producer corpus + metadata round trip | RM-304 | `libarchive_oxide/tests/interop_tar_meta.rs`, `libarchive_oxide/tests/fixtures/tar/PROVENANCE.md` | 3 producers (arca, `tar@0.4`, first-party raw ustar builder) × 2 consumers (arca sequential reader, `tar@0.4`); mode/uid/gid/mtime and symlink-target fidelity | @@ -329,6 +333,61 @@ change, or versioned release candidate is part of this snapshot. architecture-and-documentation slice establishing the contract that RM-302..306 and future codec work inherit. +## RM-305 + +- CAB and XAR are arca's FIRST read-only built-in seek-native providers: every + prior built-in format (ZIP, 7z, ISO, tar, cpio, ar) is read+write, so RM-305 + also proves the read-only registration path. Both decode through the always-on + `miniz_oxide` codec, so they need no feature gate and stay pure-Rust / C-free on + the portable profile. +- Registration (five points, all in the scaffold): `FormatId::Cab` / `FormatId::Xar` + (`libarchive_oxide-core/src/format.rs`) with `MSCF` / `xar!` probe signatures; + `format_capability` → `FormatCapabilities::new(true, false, true)` (decode yes, + encode NO, seek yes) and `format_name` in `provider.rs`; a `SeekDispatch::Cab` / + `SeekDispatch::Xar` arm plus signature detection in `SeekArchiveReader::open`; and + `mod cab` / `mod xar` in `lib.rs`. Each reader implements the same + `new`/`next_event`/`skip_entry`/`into_inner`/`source_ref` contract as the 7z + reader and drives the identical `Idle → Entry → Data(≤64 KiB) → EndEntry → Done` + phase machine. +- **CAB** (`cab.rs`): a bounded MSCF parser. The `CFHEADER` (with optional reserve + and prev/next-cabinet strings), the `CFFOLDER` and `CFFILE` tables, and the + per-folder `CFDATA` blocks are all walked within the `Limits` budget. A folder is + a solid unit decoded one `CFDATA` block at a time — no whole folder is + materialized. Store copies bytes through; MSZIP verifies the `CK` block magic and + inflates each block's raw DEFLATE while carrying the folder's 32 KiB LZ77 window + across blocks via a fresh `DecompressorOxide` over a preserved power-of-two + wrapping ring (so a back-reference in block N resolves into block N−1's output). + QUANTUM/LZX methods, cross-cabinet continuation files (`iFolder` sentinels), and + spanning blocks (`cbUncomp == 0`) list their metadata then surface a structured + `Unsupported` on the payload. +- **XAR** (`xar.rs`): the 28-byte big-endian header locates the zlib-compressed TOC + (inflated with `DataFormat::Zlib`, capped at `min(toc_length_uncompressed, + metadata_bytes)` and length-verified) and the heap (`heap_start = header.size + + toc_length_compressed`, never a hard-coded 28). A hand-rolled bounded XML + pull-scanner (an explicit DFS ``-frame stack, no DOM, no XML crate) resolves + each entry's `/`-joined path, emitting a directory before its children, bounded by + `entries` / `path_bytes` / a nesting-depth cap. Regular files stream their heap + blob per entry (seek-per-blob for the unordered/shared-heap model) as stored + (`octet-stream`, `S == L` enforced) or zlib (`x-gzip`, RFC-1950 — decoded as zlib, + NOT gzip), with the decoded length verified against ``. `x-bzip2` and any + unknown encoding are structured `Unsupported`; version ≠ 1 is `Unsupported`. +- Write is rejected without a code path of its own: the decode-only capability makes + `format_encoder` return a typed `Capability` error, and `SeekArchiveWriter::with_format` + falls through its default arm to `Unsupported` for `FormatId::Cab`/`Xar`. +- `#![forbid(unsafe_code)]`, the `no-dyn` gate, and bounded memory hold: both readers + are plain generic structs over `R: Read + Seek` with concrete enums (no trait + objects), stream payload in ≤ 64 KiB chunks, and return a structured `StreamError` + for every malformed/truncated/unsupported/limit case (never a panic/unwrap). +- Evidence: `tests/interop_cab_meta.rs` and `tests/interop_xar_meta.rs` build valid + archives with first-party in-code raw byte builders (independent DEFLATE/zlib via + `flat2`, which arca re-inflates with `miniz_oxide` — codec independence), read + them back through the RM-301 harness, and assert content round trips over + multi-file/nested/empty corpora, plus negatives for an unsupported compression + method and a truncated/out-of-range header. A three-lens adversarial review + (panic/bounds, spec-correctness, malformed-input) with an independent verification + pass was run over both modules before commit. Per-format `PROVENANCE.md` registers + the in-code raw builder and documents the external independent producers + (makecab/gcab/cabextract for CAB; the `xar` CLI / `bsdtar --format=xar` for XAR). ## RM-304 - RM-304 lifts the RM-301 interoperability harness from content-only evidence to diff --git a/libarchive_oxide-core/src/format.rs b/libarchive_oxide-core/src/format.rs index 8af10b5..bb5027a 100644 --- a/libarchive_oxide-core/src/format.rs +++ b/libarchive_oxide-core/src/format.rs @@ -26,6 +26,10 @@ pub enum FormatId { SevenZip, /// ISO 9660. Iso9660, + /// Microsoft Cabinet (read-only). + Cab, + /// XAR extensible archive (read-only). + Xar, } impl FormatId { @@ -42,6 +46,8 @@ impl FormatId { Self::SevenZip, [0x37, 0x7a, 0xbc, 0xaf, 0x27, 0x1c].as_slice(), ), + (Self::Cab, b"MSCF".as_slice()), + (Self::Xar, b"xar!".as_slice()), ] { if prefix.len() >= signature.len() && prefix.starts_with(signature) { return ProbeResult::Match(identifier); @@ -69,6 +75,8 @@ impl FormatId { b"PK\x03\x04".as_slice(), b"PK\x05\x06".as_slice(), [0x37, 0x7a, 0xbc, 0xaf, 0x27, 0x1c].as_slice(), + b"MSCF".as_slice(), + b"xar!".as_slice(), ] { if prefix.len() < signature.len() && signature.starts_with(prefix) { minimum = minimum.min(signature.len()); diff --git a/libarchive_oxide/src/cab.rs b/libarchive_oxide/src/cab.rs new file mode 100644 index 0000000..3bf5770 --- /dev/null +++ b/libarchive_oxide/src/cab.rs @@ -0,0 +1,847 @@ +// SPDX-FileCopyrightText: 2026 libarchive_oxide contributors +// +// SPDX-License-Identifier: MIT OR Apache-2.0 + +//! Microsoft Cabinet (`.cab`) read-only, seek-native provider (RM-305). +//! +//! A bounded parser for the MSCF container: `CFHEADER`, the `CFFOLDER` table, +//! the `CFFILE` table, and the per-folder `CFDATA` blocks. Two folder +//! compression methods are decoded — `NONE` (stored) and `MSZIP` (a `'CK'` +//! prefix followed by a raw-DEFLATE stream whose LZ77 window is carried across +//! the folder's blocks). `QUANTUM` and `LZX` are surfaced as structured +//! `Unsupported`, as are cross-cabinet continuation files. +//! +//! A folder is a solid unit: the decompressed output of its `CFDATA` blocks is +//! concatenated and each file is sliced from `uoffFolderStart`. The decoder is +//! streamed one `CFDATA` block at a time (each `<= 32 KiB` uncompressed), so no +//! whole folder is ever materialized and every emitted chunk stays within the +//! 64 KiB event budget. +//! +//! Known limitation: because the MSZIP history is a 32 KiB wrapping ring, a +//! *malformed* block whose back-reference distance reaches before the folder's +//! start resolves against the zero-initialized window instead of erroring. Every +//! spec-conforming cabinet stays within the valid window, so this only affects +//! deliberately corrupt input, and the output stays bounded and panic-free. + +use std::io::{Read, Seek, SeekFrom}; + +use libarchive_oxide_core::{ + ArchiveError, ArchiveMetadata, ArchivePath, EntryKind, EntryMetadata, EntryTimes, ErrorKind, + Limits, Owner, PathEncoding, Timestamp, +}; +use miniz_oxide::inflate::TINFLStatus; +use miniz_oxide::inflate::core::{DecompressorOxide, decompress}; + +use crate::{ReaderEvent, StreamError}; + +/// Maximum size of a streamed payload chunk. +const BUFFER: usize = 64 * 1024; +/// The MSZIP LZ77 window size; also the maximum `CFDATA` uncompressed size. +const MSZIP_WINDOW: usize = 0x8000; +/// The `CFHEADER` `RESERVE_PRESENT` flag. +const FLAG_RESERVE_PRESENT: u16 = 0x0004; +/// The `CFHEADER` `PREV_CABINET` flag. +const FLAG_PREV: u16 = 0x0001; +/// The `CFHEADER` `NEXT_CABINET` flag. +const FLAG_NEXT: u16 = 0x0002; +/// The `CFFILE` "name is UTF-8" attribute bit. +const ATTR_NAME_UTF8: u16 = 0x80; + +/// The compression method of a folder (`typeCompress & 0x000F`). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Method { + /// Stored: each `CFDATA` payload is the literal decompressed bytes. + Store, + /// MSZIP: a `'CK'` prefix then a raw-DEFLATE stream, window carried across blocks. + Mszip, + /// Quantum, LZX, or an unknown method — metadata is listed, payload is unsupported. + Unsupported(u16), +} + +/// A parsed `CFFOLDER` record. +#[derive(Debug, Clone, Copy)] +struct CabFolder { + /// Absolute offset of this folder's first `CFDATA` block. + data_offset: u64, + /// Number of `CFDATA` blocks in the folder. + num_data: u16, + /// The folder's compression method. + method: Method, +} + +/// A parsed `CFFILE` record. +#[derive(Debug, Clone)] +struct CabFile { + /// Path with backslashes normalized to `/`. + name: Vec, + /// Uncompressed file size (`cbFile`). + size: u64, + /// Byte offset of the file within its folder's decompressed stream. + folder_offset: u64, + /// Index into the folder table. + folder_index: usize, + /// Whether `iFolder` was a cross-cabinet continuation sentinel. + continuation: bool, + /// Modification time from the DOS date/time fields. + mtime: Option, + /// Whether the name is UTF-8 (else code-page bytes preserved verbatim). + is_utf8: bool, +} + +/// The streaming decoder for one solid folder. +struct FolderStream { + /// Which folder this stream decodes. + folder_index: usize, + /// The folder's compression method. + method: Method, + /// Number of `CFDATA` blocks in the folder. + num_blocks: u16, + /// Index of the next `CFDATA` block to decode. + next_block: u16, + /// Absolute file offset of the next `CFDATA` block header. + block_cursor: u64, + /// Per-`CFDATA` reserved-field size from the header. + reserve_data: u8, + /// Total decompressed bytes already consumed by the reader. + produced: u64, + /// Decompressed bytes of the current block awaiting consumption. + buf: Vec, + /// Consumption cursor within [`FolderStream::buf`]. + buf_pos: usize, + /// The MSZIP sliding window (empty for stored folders). + ring: Vec, + /// Write position within [`FolderStream::ring`]. + ring_pos: usize, +} + +/// The reader's payload state machine (mirrors the 7z reader's phases). +#[derive(Debug, Clone, Copy)] +enum CabPhase { + /// Between entries. + Idle, + /// Streaming a file payload with this many bytes still to emit. + Data { remaining: u64 }, + /// The open entry lives in an unsupported / cross-cabinet folder. + Unsupported, + /// The open entry's payload is exhausted. + EndEntry, + /// The archive is fully consumed. + Done, +} + +/// Seek-capable read-only Microsoft Cabinet reader. +pub(crate) struct CabSeekReader { + input: R, + limits: Limits, + image_length: u64, + archive_metadata: Option, + folders: Vec, + files: Vec, + next_file: usize, + phase: CabPhase, + folder_stream: Option, + reserve_data: u8, + event_data: Vec, + decoded_total: u64, +} + +impl std::fmt::Debug for CabSeekReader { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter + .debug_struct("CabSeekReader") + .field("folders", &self.folders.len()) + .field("files", &self.files.len()) + .field("next_file", &self.next_file) + .field("phase", &self.phase) + .finish_non_exhaustive() + } +} + +impl CabSeekReader { + pub(crate) fn new(mut input: R, limits: Limits) -> core::result::Result { + let image_length = input.seek(SeekFrom::End(0)).map_err(StreamError::io)?; + input.seek(SeekFrom::Start(0)).map_err(StreamError::io)?; + + let mut header = [0_u8; 36]; + input.read_exact(&mut header).map_err(StreamError::io)?; + if &header[0..4] != b"MSCF" { + return Err(cab_error(ErrorKind::Malformed, "bad MSCF signature")); + } + let coff_files = u64::from(u32::from_le_bytes([ + header[16], header[17], header[18], header[19], + ])); + let num_folders = u16::from_le_bytes([header[26], header[27]]); + let num_files = u16::from_le_bytes([header[28], header[29]]); + let flags = u16::from_le_bytes([header[30], header[31]]); + + if coff_files == 0 || coff_files >= image_length { + return Err(cab_error( + ErrorKind::Malformed, + "CFFILE table offset is outside the cabinet", + )); + } + if limits + .entries() + .is_some_and(|maximum| u64::from(num_files) > maximum) + { + return Err(cab_error( + ErrorKind::Limit, + "file count exceeds configured limit", + )); + } + + let (reserve_folder, reserve_data) = if flags & FLAG_RESERVE_PRESENT != 0 { + let mut reserve = [0_u8; 4]; + input.read_exact(&mut reserve).map_err(StreamError::io)?; + let cb_header = u16::from_le_bytes([reserve[0], reserve[1]]); + if cb_header != 0 { + input + .seek(SeekFrom::Current(i64::from(cb_header))) + .map_err(StreamError::io)?; + } + (reserve[2], reserve[3]) + } else { + (0_u8, 0_u8) + }; + + if flags & FLAG_PREV != 0 { + skip_cstring(&mut input, limits)?; + skip_cstring(&mut input, limits)?; + } + if flags & FLAG_NEXT != 0 { + skip_cstring(&mut input, limits)?; + skip_cstring(&mut input, limits)?; + } + + let folder_table = input.stream_position().map_err(StreamError::io)?; + let folders = read_folders( + &mut input, + folder_table, + num_folders, + reserve_folder, + image_length, + )?; + let files = read_files(&mut input, coff_files, num_files, &folders, limits)?; + + Ok(Self { + input, + limits, + image_length, + archive_metadata: Some(ArchiveMetadata::new()), + folders, + files, + next_file: 0, + phase: CabPhase::Idle, + folder_stream: None, + reserve_data, + event_data: Vec::with_capacity(BUFFER), + decoded_total: 0, + }) + } + + pub(crate) fn next_event(&mut self) -> core::result::Result, StreamError> { + self.event_data.clear(); + if let Some(metadata) = self.archive_metadata.take() { + return Ok(ReaderEvent::ArchiveMetadata(metadata)); + } + loop { + match self.phase { + CabPhase::Idle => { + let Some(file) = self.files.get(self.next_file).cloned() else { + self.phase = CabPhase::Done; + return Ok(ReaderEvent::Done); + }; + self.next_file += 1; + let metadata = self.prepare_file(&file)?; + return Ok(ReaderEvent::Entry(metadata)); + }, + CabPhase::Data { remaining: 0 } => { + self.phase = CabPhase::EndEntry; + }, + CabPhase::Data { remaining } => { + let want = usize::try_from(remaining.min(BUFFER as u64)).map_err(|_| { + cab_error(ErrorKind::Limit, "payload chunk exceeds address space") + })?; + let count = self.pull_into_event(want)?; + if count == 0 { + return Err(cab_error( + ErrorKind::Malformed, + "folder ended before the declared file size", + )); + } + self.phase = CabPhase::Data { + remaining: remaining - count as u64, + }; + return Ok(ReaderEvent::Data(&self.event_data)); + }, + CabPhase::Unsupported => { + return Err(cab_error( + ErrorKind::Unsupported, + "folder compression method is unsupported", + )); + }, + CabPhase::EndEntry => { + self.phase = CabPhase::Idle; + return Ok(ReaderEvent::EndEntry); + }, + CabPhase::Done => return Ok(ReaderEvent::Done), + } + } + } + + pub(crate) fn skip_entry(&mut self) -> core::result::Result<(), StreamError> { + match self.phase { + CabPhase::Data { mut remaining } => { + while remaining != 0 { + let step = self.discard_decoded(remaining)?; + if step == 0 { + return Err(cab_error( + ErrorKind::Malformed, + "folder ended while skipping a file", + )); + } + remaining -= step; + } + self.phase = CabPhase::EndEntry; + Ok(()) + }, + CabPhase::Unsupported => { + self.phase = CabPhase::EndEntry; + Ok(()) + }, + CabPhase::EndEntry => Ok(()), + CabPhase::Idle | CabPhase::Done => Err(cab_error( + ErrorKind::Protocol, + "skip_entry called without an open entry", + )), + } + } + + pub(crate) fn into_inner(self) -> R { + self.input + } + + pub(crate) fn source_ref(&self) -> &R { + &self.input + } + + /// Positions the folder decoder and payload phase for `file`, then builds its metadata. + fn prepare_file(&mut self, file: &CabFile) -> core::result::Result { + if file.continuation { + self.phase = CabPhase::Unsupported; + } else { + let folder = self + .folders + .get(file.folder_index) + .copied() + .ok_or_else(|| { + cab_error(ErrorKind::Malformed, "file references a missing folder") + })?; + match folder.method { + Method::Unsupported(_) => self.phase = CabPhase::Unsupported, + Method::Store | Method::Mszip => { + self.ensure_folder_stream(file.folder_index, folder); + self.drain_to(file.folder_offset)?; + self.phase = CabPhase::Data { + remaining: file.size, + }; + }, + } + } + + let path = if file.is_utf8 { + ArchivePath::from_encoded(file.name.clone(), PathEncoding::Utf8) + } else { + ArchivePath::from_bytes(file.name.clone()) + }; + let times = EntryTimes { + modified: file.mtime, + ..EntryTimes::default() + }; + Ok(EntryMetadata::builder(EntryKind::File, path) + .size(Some(file.size)) + .mode(Some(0o644)) + .owner(Owner::default()) + .times(times) + .build()) + } + + /// Installs a fresh [`FolderStream`] for `index` unless the current one already decodes it. + fn ensure_folder_stream(&mut self, index: usize, folder: CabFolder) { + if self + .folder_stream + .as_ref() + .is_some_and(|stream| stream.folder_index == index) + { + return; + } + let ring = if folder.method == Method::Mszip { + vec![0_u8; MSZIP_WINDOW] + } else { + Vec::new() + }; + self.folder_stream = Some(FolderStream { + folder_index: index, + method: folder.method, + num_blocks: folder.num_data, + next_block: 0, + block_cursor: folder.data_offset, + reserve_data: self.reserve_data, + produced: 0, + buf: Vec::new(), + buf_pos: 0, + ring, + ring_pos: 0, + }); + } + + /// Advances the folder decoder to `offset`, discarding intervening bytes. + fn drain_to(&mut self, offset: u64) -> core::result::Result<(), StreamError> { + let produced = self + .folder_stream + .as_ref() + .map_or(0, |stream| stream.produced); + if offset < produced { + return Err(cab_error( + ErrorKind::Malformed, + "file offsets are not monotonic within a folder", + )); + } + let mut remaining = offset - produced; + while remaining != 0 { + let step = self.discard_decoded(remaining)?; + if step == 0 { + return Err(cab_error( + ErrorKind::Malformed, + "folder ended before a file offset", + )); + } + remaining -= step; + } + Ok(()) + } + + /// Copies up to `max` decoded bytes into `event_data`, returning the count. + fn pull_into_event(&mut self, max: usize) -> core::result::Result { + if !self.ensure_block()? { + return Ok(0); + } + let stream = self + .folder_stream + .as_mut() + .ok_or_else(|| cab_error(ErrorKind::Protocol, "folder stream disappeared"))?; + let available = stream.buf.len() - stream.buf_pos; + let count = available.min(max); + self.event_data + .extend_from_slice(&stream.buf[stream.buf_pos..stream.buf_pos + count]); + stream.buf_pos += count; + stream.produced += count as u64; + Ok(count) + } + + /// Advances the folder decoder by up to `max` bytes without copying, returning the count. + fn discard_decoded(&mut self, max: u64) -> core::result::Result { + if !self.ensure_block()? { + return Ok(0); + } + let stream = self + .folder_stream + .as_mut() + .ok_or_else(|| cab_error(ErrorKind::Protocol, "folder stream disappeared"))?; + let available = (stream.buf.len() - stream.buf_pos) as u64; + let step = available.min(max); + let advance = usize::try_from(step) + .map_err(|_| cab_error(ErrorKind::Limit, "skip step exceeds address space"))?; + stream.buf_pos += advance; + stream.produced += step; + Ok(step) + } + + /// Ensures the current folder block buffer has unconsumed bytes, decoding the next + /// `CFDATA` block when needed. Returns `false` once the folder is fully decoded. + fn ensure_block(&mut self) -> core::result::Result { + loop { + { + let stream = self + .folder_stream + .as_ref() + .ok_or_else(|| cab_error(ErrorKind::Protocol, "no folder stream is open"))?; + if stream.buf_pos < stream.buf.len() { + return Ok(true); + } + if stream.next_block >= stream.num_blocks { + return Ok(false); + } + } + let Self { + input, + folder_stream, + image_length, + limits, + decoded_total, + .. + } = self; + let stream = folder_stream + .as_mut() + .ok_or_else(|| cab_error(ErrorKind::Protocol, "no folder stream is open"))?; + let produced = decode_next_block(input, stream, *image_length)?; + *decoded_total = decoded_total + .checked_add(produced as u64) + .ok_or_else(|| cab_error(ErrorKind::Limit, "decoded total overflow"))?; + if limits + .decoded_total() + .is_some_and(|maximum| *decoded_total > maximum) + { + return Err(cab_error( + ErrorKind::Limit, + "decoded total exceeds configured limit", + )); + } + } + } +} + +/// Reads the `CFFOLDER` table starting at `offset`. +fn read_folders( + input: &mut R, + offset: u64, + count: u16, + reserve_folder: u8, + image_length: u64, +) -> core::result::Result, StreamError> { + input + .seek(SeekFrom::Start(offset)) + .map_err(StreamError::io)?; + let mut folders = Vec::new(); + for _ in 0..count { + let mut record = [0_u8; 8]; + input.read_exact(&mut record).map_err(StreamError::io)?; + let data_offset = u64::from(u32::from_le_bytes([ + record[0], record[1], record[2], record[3], + ])); + let num_data = u16::from_le_bytes([record[4], record[5]]); + let type_compress = u16::from_le_bytes([record[6], record[7]]); + if data_offset >= image_length { + return Err(cab_error( + ErrorKind::Malformed, + "folder data offset is outside the cabinet", + )); + } + let method = match type_compress & 0x000F { + 0 => Method::Store, + 1 => Method::Mszip, + other => Method::Unsupported(other), + }; + folders.push(CabFolder { + data_offset, + num_data, + method, + }); + if reserve_folder != 0 { + input + .seek(SeekFrom::Current(i64::from(reserve_folder))) + .map_err(StreamError::io)?; + } + } + Ok(folders) +} + +/// Reads the `CFFILE` table starting at `offset`, bounding names and metadata by `limits`. +fn read_files( + input: &mut R, + offset: u64, + count: u16, + folders: &[CabFolder], + limits: Limits, +) -> core::result::Result, StreamError> { + input + .seek(SeekFrom::Start(offset)) + .map_err(StreamError::io)?; + let mut files = Vec::new(); + let mut metadata_used = 0_usize; + for _ in 0..count { + let mut record = [0_u8; 16]; + input.read_exact(&mut record).map_err(StreamError::io)?; + let size = u64::from(u32::from_le_bytes([ + record[0], record[1], record[2], record[3], + ])); + let folder_offset = u64::from(u32::from_le_bytes([ + record[4], record[5], record[6], record[7], + ])); + let i_folder = u16::from_le_bytes([record[8], record[9]]); + let date = u16::from_le_bytes([record[10], record[11]]); + let time = u16::from_le_bytes([record[12], record[13]]); + let attribs = u16::from_le_bytes([record[14], record[15]]); + + let raw_name = read_cstring(input, limits)?; + let name: Vec = raw_name + .iter() + .map(|&byte| if byte == b'\\' { b'/' } else { byte }) + .collect(); + + if limits.entry_bytes().is_some_and(|maximum| size > maximum) { + return Err(cab_error( + ErrorKind::Limit, + "file size exceeds configured limit", + )); + } + metadata_used = metadata_used + .checked_add(name.len()) + .and_then(|value| value.checked_add(core::mem::size_of::())) + .ok_or_else(|| cab_error(ErrorKind::Limit, "metadata accounting overflow"))?; + if limits + .metadata_bytes() + .is_some_and(|maximum| metadata_used > maximum) + { + return Err(cab_error( + ErrorKind::Limit, + "file metadata exceeds configured limit", + )); + } + + let continuation = matches!(i_folder, 0xFFFD..=0xFFFF); + let folder_index = if continuation { + 0 + } else { + usize::from(i_folder) + }; + if !continuation && folder_index >= folders.len() { + return Err(cab_error( + ErrorKind::Malformed, + "file references a folder index out of range", + )); + } + + files.push(CabFile { + name, + size, + folder_offset, + folder_index, + continuation, + mtime: dos_datetime_to_timestamp(date, time), + is_utf8: attribs & ATTR_NAME_UTF8 != 0, + }); + } + Ok(files) +} + +/// Decodes the next `CFDATA` block of `stream`, replacing its buffer. Returns the block's +/// decompressed byte count. +fn decode_next_block( + input: &mut R, + stream: &mut FolderStream, + image_length: u64, +) -> core::result::Result { + let start = stream.block_cursor; + input + .seek(SeekFrom::Start(start)) + .map_err(StreamError::io)?; + let mut header = [0_u8; 8]; + input.read_exact(&mut header).map_err(StreamError::io)?; + let cb_data = usize::from(u16::from_le_bytes([header[4], header[5]])); + let cb_uncomp = usize::from(u16::from_le_bytes([header[6], header[7]])); + + if cb_uncomp == 0 { + return Err(cab_error( + ErrorKind::Unsupported, + "spanning CFDATA block continues in another cabinet", + )); + } + if cb_uncomp > MSZIP_WINDOW { + return Err(cab_error( + ErrorKind::Malformed, + "CFDATA uncompressed size exceeds 32 KiB", + )); + } + + let reserve = u64::from(stream.reserve_data); + let payload_offset = start + .checked_add(8) + .and_then(|value| value.checked_add(reserve)) + .ok_or_else(|| cab_error(ErrorKind::Malformed, "CFDATA offset overflow"))?; + let next_cursor = payload_offset + .checked_add(cb_data as u64) + .ok_or_else(|| cab_error(ErrorKind::Malformed, "CFDATA extent overflow"))?; + if next_cursor > image_length { + return Err(cab_error( + ErrorKind::Malformed, + "CFDATA block extends past the cabinet", + )); + } + + input + .seek(SeekFrom::Start(payload_offset)) + .map_err(StreamError::io)?; + let mut payload = vec![0_u8; cb_data]; + input.read_exact(&mut payload).map_err(StreamError::io)?; + stream.block_cursor = next_cursor; + stream.next_block += 1; + + let decoded = match stream.method { + Method::Store => { + if cb_data != cb_uncomp { + return Err(cab_error( + ErrorKind::Malformed, + "stored CFDATA compressed and uncompressed sizes disagree", + )); + } + payload + }, + Method::Mszip => { + if payload.len() < 2 || &payload[0..2] != b"CK" { + return Err(cab_error( + ErrorKind::Malformed, + "MSZIP CFDATA block missing 'CK' signature", + )); + } + mszip_inflate_block( + &mut stream.ring, + &mut stream.ring_pos, + &payload[2..], + cb_uncomp, + )? + }, + Method::Unsupported(_) => { + return Err(cab_error( + ErrorKind::Unsupported, + "folder compression method is unsupported", + )); + }, + }; + + let produced = decoded.len(); + stream.buf = decoded; + stream.buf_pos = 0; + Ok(produced) +} + +/// Inflates one MSZIP block's raw-DEFLATE stream into the folder's wrapping window `ring`, +/// carrying the LZ77 history across blocks. Returns exactly `expected` decompressed bytes. +fn mszip_inflate_block( + ring: &mut [u8], + ring_pos: &mut usize, + deflate: &[u8], + expected: usize, +) -> core::result::Result, StreamError> { + let mask = ring.len() - 1; + let mut decompressor = DecompressorOxide::new(); + let mut output = Vec::with_capacity(expected); + let mut input = deflate; + // flags == 0: raw DEFLATE (no zlib header), wrapping output buffer, all input present. + loop { + let start = *ring_pos; + let (status, consumed, written) = decompress(&mut decompressor, input, ring, start, 0); + output.extend_from_slice(&ring[start..start + written]); + input = input + .get(consumed..) + .ok_or_else(|| cab_error(ErrorKind::Malformed, "MSZIP consumed count overflow"))?; + *ring_pos = (start + written) & mask; + if output.len() > expected { + return Err(cab_error( + ErrorKind::Malformed, + "MSZIP block produced more than its declared size", + )); + } + match status { + TINFLStatus::Done => break, + TINFLStatus::HasMoreOutput => { + if written == 0 && consumed == 0 { + return Err(cab_error( + ErrorKind::Malformed, + "MSZIP decoder made no progress", + )); + } + }, + TINFLStatus::NeedsMoreInput | TINFLStatus::FailedCannotMakeProgress => { + return Err(cab_error( + ErrorKind::Malformed, + "truncated MSZIP DEFLATE stream", + )); + }, + _ => { + return Err(cab_error( + ErrorKind::Malformed, + "invalid MSZIP DEFLATE stream", + )); + }, + } + } + if output.len() != expected { + return Err(cab_error( + ErrorKind::Malformed, + "MSZIP block size does not match its header", + )); + } + Ok(output) +} + +/// Reads a NUL-terminated name from the stream, bounding its length by `path_bytes`. +fn read_cstring( + input: &mut R, + limits: Limits, +) -> core::result::Result, StreamError> { + let mut out = Vec::new(); + loop { + let mut byte = [0_u8; 1]; + input.read_exact(&mut byte).map_err(StreamError::io)?; + if byte[0] == 0 { + return Ok(out); + } + if limits + .path_bytes() + .is_some_and(|maximum| out.len() >= maximum) + { + return Err(cab_error( + ErrorKind::Limit, + "name exceeds configured path limit", + )); + } + out.push(byte[0]); + } +} + +/// Skips a NUL-terminated cabinet header string, bounded by `path_bytes`. +fn skip_cstring(input: &mut R, limits: Limits) -> core::result::Result<(), StreamError> { + read_cstring(input, limits).map(|_| ()) +} + +/// Converts DOS date and time fields to a Unix [`Timestamp`], or `None` when unset/invalid. +fn dos_datetime_to_timestamp(date: u16, time: u16) -> Option { + if date == 0 && time == 0 { + return None; + } + let year = 1980 + i64::from(date >> 9); + let month = i64::from((date >> 5) & 0x0F); + let day = i64::from(date & 0x1F); + let hour = i64::from((time >> 11) & 0x1F); + let minute = i64::from((time >> 5) & 0x3F); + let second = i64::from(time & 0x1F) * 2; + if !(1..=12).contains(&month) || !(1..=31).contains(&day) || hour > 23 || minute > 59 { + return None; + } + let days = days_from_civil(year, month, day); + let secs = days + .checked_mul(86_400)? + .checked_add(hour * 3_600 + minute * 60 + second)?; + Some(Timestamp { secs, nanos: 0 }) +} + +/// Days from the Unix epoch to the given civil date (Howard Hinnant's algorithm). +fn days_from_civil(year: i64, month: i64, day: i64) -> i64 { + let year = if month <= 2 { year - 1 } else { year }; + let era = if year >= 0 { year } else { year - 399 } / 400; + let year_of_era = year - era * 400; + let month_shift = if month > 2 { month - 3 } else { month + 9 }; + let day_of_year = (153 * month_shift + 2) / 5 + day - 1; + let day_of_era = year_of_era * 365 + year_of_era / 4 - year_of_era / 100 + day_of_year; + era * 146_097 + day_of_era - 719_468 +} + +/// Builds a `cab`-tagged structured error. +fn cab_error(kind: ErrorKind, context: &'static str) -> StreamError { + StreamError::archive( + ArchiveError::new(kind) + .with_format("cab") + .with_context(context), + ) +} diff --git a/libarchive_oxide/src/lib.rs b/libarchive_oxide/src/lib.rs index 93b8f1a..4f22b54 100644 --- a/libarchive_oxide/src/lib.rs +++ b/libarchive_oxide/src/lib.rs @@ -31,6 +31,7 @@ pub mod async_range; pub mod async_seek; #[cfg(feature = "async")] pub mod async_stream; +mod cab; pub mod create; pub mod engine; pub mod extractor; @@ -55,6 +56,7 @@ pub mod spool; pub mod stream; #[cfg(feature = "tokio")] pub mod tokio_stream; +mod xar; mod zip; mod zip_stream; diff --git a/libarchive_oxide/src/provider.rs b/libarchive_oxide/src/provider.rs index 52ef652..820821f 100644 --- a/libarchive_oxide/src/provider.rs +++ b/libarchive_oxide/src/provider.rs @@ -1025,6 +1025,10 @@ impl StaticFormatProviders for BuiltinFormatProviders { ProviderCapability::Available(FormatCapabilities::new(true, true, true)) }, FormatId::SevenZip => ProviderCapability::Disabled, + // CAB and XAR are seek-native READ-ONLY providers: decode yes, encode no. + FormatId::Cab | FormatId::Xar => { + ProviderCapability::Available(FormatCapabilities::new(true, false, true)) + }, _ => ProviderCapability::Unknown, } } @@ -1345,6 +1349,8 @@ pub(crate) const fn format_name(format: FormatId) -> &'static str { FormatId::Zip => "zip", FormatId::SevenZip => "7z", FormatId::Iso9660 => "iso9660", + FormatId::Cab => "cab", + FormatId::Xar => "xar", _ => "unknown", } } diff --git a/libarchive_oxide/src/seek_stream.rs b/libarchive_oxide/src/seek_stream.rs index be88679..e777ef6 100644 --- a/libarchive_oxide/src/seek_stream.rs +++ b/libarchive_oxide/src/seek_stream.rs @@ -254,6 +254,8 @@ enum SeekDispatch { Indexed(Box>), #[cfg(feature = "sevenz")] SevenZ(Box>), + Cab(Box>), + Xar(Box>), } /// Seek-capable archive reader. @@ -299,6 +301,16 @@ impl SeekArchiveReader { let mut signature = [0_u8; 6]; let read = read_prefix(&mut input, &mut signature)?; input.seek(SeekFrom::Start(0)).map_err(StreamError::io)?; + if read >= 4 && &signature[..4] == b"MSCF" { + return Ok(Self { + inner: SeekDispatch::Cab(Box::new(crate::cab::CabSeekReader::new(input, limits)?)), + }); + } + if read >= 4 && &signature[..4] == b"xar!" { + return Ok(Self { + inner: SeekDispatch::Xar(Box::new(crate::xar::XarSeekReader::new(input, limits)?)), + }); + } if read == signature.len() && signature == [0x37, 0x7a, 0xbc, 0xaf, 0x27, 0x1c] { #[cfg(feature = "sevenz")] { @@ -330,6 +342,8 @@ impl SeekArchiveReader { SeekDispatch::Indexed(reader) => reader.next_event(), #[cfg(feature = "sevenz")] SeekDispatch::SevenZ(reader) => reader.next_event(), + SeekDispatch::Cab(reader) => reader.next_event(), + SeekDispatch::Xar(reader) => reader.next_event(), } } @@ -339,6 +353,8 @@ impl SeekArchiveReader { SeekDispatch::Indexed(reader) => reader.skip_entry(), #[cfg(feature = "sevenz")] SeekDispatch::SevenZ(reader) => reader.skip_entry(), + SeekDispatch::Cab(reader) => reader.skip_entry(), + SeekDispatch::Xar(reader) => reader.skip_entry(), } } @@ -349,6 +365,8 @@ impl SeekArchiveReader { SeekDispatch::Indexed(reader) => reader.format(), #[cfg(feature = "sevenz")] SeekDispatch::SevenZ(_) => FormatId::SevenZip, + SeekDispatch::Cab(_) => FormatId::Cab, + SeekDispatch::Xar(_) => FormatId::Xar, } } @@ -359,6 +377,8 @@ impl SeekArchiveReader { SeekDispatch::Indexed(reader) => reader.into_inner(), #[cfg(feature = "sevenz")] SeekDispatch::SevenZ(reader) => reader.into_inner(), + SeekDispatch::Cab(reader) => reader.into_inner(), + SeekDispatch::Xar(reader) => reader.into_inner(), } } @@ -367,6 +387,8 @@ impl SeekArchiveReader { SeekDispatch::Indexed(reader) => reader.source_ref(), #[cfg(feature = "sevenz")] SeekDispatch::SevenZ(reader) => reader.source_ref(), + SeekDispatch::Cab(reader) => reader.source_ref(), + SeekDispatch::Xar(reader) => reader.source_ref(), } } } diff --git a/libarchive_oxide/src/xar.rs b/libarchive_oxide/src/xar.rs new file mode 100644 index 0000000..191aa9d --- /dev/null +++ b/libarchive_oxide/src/xar.rs @@ -0,0 +1,1223 @@ +// SPDX-FileCopyrightText: 2026 libarchive_oxide contributors +// +// SPDX-License-Identifier: MIT OR Apache-2.0 + +//! `eXtensible` `ARchive` (`.xar`) read-only, seek-native provider (RM-305). +//! +//! Parses the 28-byte big-endian header, inflates the zlib (RFC-1950) compressed +//! TOC into bounded UTF-8 XML, walks the `` tree with a hand-rolled bounded +//! pull-scanner (no DOM, no XML crate), and streams heap payloads per file in +//! `<= 64 KiB` chunks. Supported data encodings: `application/octet-stream` +//! (stored) and `application/x-gzip` (zlib RFC-1950). Every other encoding, and +//! `application/x-bzip2`, surface a structured `Unsupported` error at read time. + +use std::io::{Read, Seek, SeekFrom}; + +use libarchive_oxide_core::{ + ArchiveError, ArchiveMetadata, ArchivePath, EntryKind, EntryMetadata, EntryTimes, ErrorKind, + Limits, Owner, PathEncoding, Timestamp, +}; +use miniz_oxide::inflate::stream::{InflateState, inflate}; +use miniz_oxide::{DataFormat, MZError, MZFlush, MZStatus}; + +use crate::{ReaderEvent, StreamError}; + +/// The 4-byte XAR magic (`'x' 'a' 'r' '!'`), big-endian `0x7861_7221`. +const MAGIC: u32 = 0x7861_7221; +/// The minimal (and canonical) header length. +const MIN_HEADER: u16 = 28; +/// Streaming chunk size for both TOC inflate and heap payload decode. +const BUFFER: usize = 64 * 1024; +/// Cap on `` element nesting depth. +const MAX_DEPTH: usize = 256; + +// ════════════════════════════════════════════════════════════════════════════ +// Error helper +// ════════════════════════════════════════════════════════════════════════════ + +fn xar_error(kind: ErrorKind, context: &'static str) -> StreamError { + StreamError::archive( + ArchiveError::new(kind) + .with_format("xar") + .with_context(context), + ) +} + +// ════════════════════════════════════════════════════════════════════════════ +// Parsed structures +// ════════════════════════════════════════════════════════════════════════════ + +/// Heap data-blob encoding classified at TOC-parse time. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum XarEncoding { + /// `application/octet-stream`: raw copy, stored size equals decoded length. + Stored, + /// `application/x-gzip`: a zlib (RFC-1950) stream — the XAR default. + Zlib, + /// A recognized-but-unsupported or unknown encoding style. + Unsupported, +} + +/// The heap window and decode plan for a regular file. +#[derive(Debug, Clone, Copy)] +struct XarData { + encoding: XarEncoding, + /// Heap-relative offset of the stored blob. + offset: u64, + /// Stored (on-heap) size in bytes. + stored_size: u64, + /// Decoded length in bytes. + length: u64, +} + +/// A fully-resolved TOC entry with its `/`-joined path. +#[derive(Debug, Clone)] +struct XarFile { + path: Vec, + kind: EntryKind, + mode: Option, + uid: Option, + gid: Option, + mtime: Option, + link_target: Option>, + data: Option, +} + +// ════════════════════════════════════════════════════════════════════════════ +// Reader +// ════════════════════════════════════════════════════════════════════════════ + +/// Streaming heap-payload decoder for the currently-open entry. +enum Payload { + /// Copy `remaining` bytes straight from the heap. + Stored { remaining: u64 }, + /// Inflate a zlib stream, `comp_remaining` compressed bytes left on the heap. + Zlib { + comp_remaining: u64, + state: Box, + in_buf: Vec, + in_pos: usize, + in_len: usize, + finished: bool, + }, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum XarPhase { + Idle, + Data { remaining: u64 }, + Unsupported, + EndEntry, + Done, +} + +/// Seek-capable read-only `eXtensible` `ARchive` reader. +pub(crate) struct XarSeekReader { + input: R, + limits: Limits, + archive_metadata: Option, + heap_start: u64, + file_length: u64, + files: Vec, + next_file: usize, + phase: XarPhase, + payload: Option, + event_data: Vec, + decoded_total: u64, +} + +impl std::fmt::Debug for XarSeekReader { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter + .debug_struct("XarSeekReader") + .field("files", &self.files.len()) + .field("next_file", &self.next_file) + .field("phase", &self.phase) + .finish_non_exhaustive() + } +} + +impl XarSeekReader { + pub(crate) fn new(mut input: R, limits: Limits) -> core::result::Result { + let file_length = input.seek(SeekFrom::End(0)).map_err(StreamError::io)?; + input.seek(SeekFrom::Start(0)).map_err(StreamError::io)?; + + let mut header = [0_u8; MIN_HEADER as usize]; + input.read_exact(&mut header).map_err(StreamError::io)?; + if be32(&header, 0) != MAGIC { + return Err(xar_error(ErrorKind::Malformed, "bad xar magic")); + } + let size = be16(&header, 4); + if size < MIN_HEADER { + return Err(xar_error(ErrorKind::Malformed, "xar header size too small")); + } + if be16(&header, 6) != 1 { + return Err(xar_error(ErrorKind::Unsupported, "unsupported xar version")); + } + let toc_comp = be64(&header, 8); + let toc_uncomp = be64(&header, 16); + + let toc_start = u64::from(size); + if toc_start > file_length { + return Err(xar_error( + ErrorKind::Malformed, + "xar header size beyond end of file", + )); + } + let heap_start = toc_start + .checked_add(toc_comp) + .filter(|&end| end <= file_length) + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar TOC region beyond end of file"))?; + + // Bound the decoded TOC by the metadata budget before inflating. + if let Some(maximum) = limits.metadata_bytes() { + if toc_uncomp > maximum as u64 { + return Err(xar_error( + ErrorKind::Limit, + "xar TOC exceeds metadata limit", + )); + } + } + let toc_uncomp_usize = usize::try_from(toc_uncomp) + .map_err(|_| xar_error(ErrorKind::Limit, "xar TOC exceeds address space"))?; + + input + .seek(SeekFrom::Start(toc_start)) + .map_err(StreamError::io)?; + let xml = inflate_toc(&mut input, toc_comp, toc_uncomp_usize)?; + + let files = parse_toc(&xml, limits)?; + + Ok(Self { + input, + limits, + archive_metadata: Some(ArchiveMetadata::new()), + heap_start, + file_length, + files, + next_file: 0, + phase: XarPhase::Idle, + payload: None, + event_data: Vec::with_capacity(BUFFER), + decoded_total: 0, + }) + } + + pub(crate) fn next_event(&mut self) -> core::result::Result, StreamError> { + self.event_data.clear(); + if let Some(metadata) = self.archive_metadata.take() { + return Ok(ReaderEvent::ArchiveMetadata(metadata)); + } + loop { + match self.phase { + XarPhase::Idle => { + let Some(record) = self.files.get(self.next_file).cloned() else { + self.phase = XarPhase::Done; + return Ok(ReaderEvent::Done); + }; + self.next_file += 1; + let metadata = self.prepare_record(&record)?; + return Ok(ReaderEvent::Entry(metadata)); + }, + XarPhase::Data { remaining: 0 } => { + self.payload = None; + self.phase = XarPhase::EndEntry; + }, + XarPhase::Data { remaining } => { + let amount = usize::try_from(remaining.min(BUFFER as u64)) + .map_err(|_| xar_error(ErrorKind::Limit, "chunk exceeds address space"))?; + self.event_data.resize(amount, 0); + let count = self.read_payload_chunk(amount)?; + if count == 0 { + return Err(xar_error( + ErrorKind::Integrity, + "xar heap blob shorter than declared length", + )); + } + self.event_data.truncate(count); + self.phase = XarPhase::Data { + remaining: remaining - count as u64, + }; + return Ok(ReaderEvent::Data(&self.event_data)); + }, + XarPhase::Unsupported => { + return Err(xar_error( + ErrorKind::Unsupported, + "xar data encoding is unsupported", + )); + }, + XarPhase::EndEntry => { + self.phase = XarPhase::Idle; + return Ok(ReaderEvent::EndEntry); + }, + XarPhase::Done => return Ok(ReaderEvent::Done), + } + } + } + + pub(crate) fn skip_entry(&mut self) -> core::result::Result<(), StreamError> { + match self.phase { + XarPhase::Data { mut remaining } => { + let mut scratch = vec![0_u8; BUFFER]; + while remaining != 0 { + let amount = usize::try_from(remaining.min(BUFFER as u64)) + .map_err(|_| xar_error(ErrorKind::Limit, "chunk exceeds address space"))?; + let count = self.read_payload_into(&mut scratch[..amount])?; + if count == 0 { + return Err(xar_error( + ErrorKind::Integrity, + "xar heap blob shorter than declared length", + )); + } + remaining -= count as u64; + } + self.payload = None; + self.phase = XarPhase::EndEntry; + Ok(()) + }, + XarPhase::Unsupported => { + self.phase = XarPhase::EndEntry; + Ok(()) + }, + XarPhase::EndEntry => Ok(()), + XarPhase::Idle | XarPhase::Done => Err(xar_error( + ErrorKind::Protocol, + "skip_entry called without an open xar entry", + )), + } + } + + pub(crate) fn into_inner(self) -> R { + self.input + } + + pub(crate) fn source_ref(&self) -> &R { + &self.input + } + + fn prepare_record( + &mut self, + record: &XarFile, + ) -> core::result::Result { + match record.data { + Some(data) if record.kind == EntryKind::File => { + self.begin_payload(data)?; + }, + _ => { + self.payload = None; + self.phase = XarPhase::EndEntry; + }, + } + + let size = match record.kind { + EntryKind::File => record.data.map(|d| d.length), + EntryKind::Dir => Some(0), + _ => None, + }; + let owner = Owner { + uid: record.uid, + gid: record.gid, + user: None, + group: None, + }; + let link_target = record + .link_target + .clone() + .map(|target| ArchivePath::from_encoded(target, PathEncoding::Utf8)); + let builder = EntryMetadata::builder( + record.kind, + ArchivePath::from_encoded(record.path.clone(), PathEncoding::Utf8), + ) + .size(size) + .mode(record.mode) + .owner(owner) + .times(EntryTimes { + modified: record.mtime, + accessed: None, + changed: None, + created: None, + }) + .link_target(link_target); + Ok(builder.build()) + } + + /// Seeks to the heap blob and installs the decode state for `data`. + fn begin_payload(&mut self, data: XarData) -> core::result::Result<(), StreamError> { + let blob_start = self + .heap_start + .checked_add(data.offset) + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar heap offset overflow"))?; + let blob_end = blob_start + .checked_add(data.stored_size) + .filter(|&end| end <= self.file_length) + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar heap blob beyond end of file"))?; + let _ = blob_end; + + // decoded-total budget. + if let Some(maximum) = self.limits.decoded_total() { + if self + .decoded_total + .checked_add(data.length) + .is_none_or(|total| total > maximum) + { + return Err(xar_error( + ErrorKind::Limit, + "xar decoded total exceeds limit", + )); + } + } + + match data.encoding { + XarEncoding::Stored => { + if data.stored_size != data.length { + return Err(xar_error( + ErrorKind::Malformed, + "xar stored blob size and length disagree", + )); + } + self.input + .seek(SeekFrom::Start(blob_start)) + .map_err(StreamError::io)?; + self.payload = Some(Payload::Stored { + remaining: data.stored_size, + }); + self.phase = if data.length == 0 { + XarPhase::EndEntry + } else { + XarPhase::Data { + remaining: data.length, + } + }; + }, + XarEncoding::Zlib => { + self.input + .seek(SeekFrom::Start(blob_start)) + .map_err(StreamError::io)?; + self.payload = Some(Payload::Zlib { + comp_remaining: data.stored_size, + state: Box::new(InflateState::new(DataFormat::Zlib)), + in_buf: vec![0_u8; BUFFER], + in_pos: 0, + in_len: 0, + finished: false, + }); + self.phase = if data.length == 0 { + XarPhase::EndEntry + } else { + XarPhase::Data { + remaining: data.length, + } + }; + }, + XarEncoding::Unsupported => { + self.payload = None; + self.phase = XarPhase::Unsupported; + }, + } + Ok(()) + } + + /// Fills the first `amount` bytes of `event_data`, returning bytes produced. + fn read_payload_chunk(&mut self, amount: usize) -> core::result::Result { + let mut filled = 0; + while filled < amount { + let produced = read_payload( + &mut self.input, + self.payload.as_mut(), + &mut self.event_data[filled..amount], + )?; + if produced == 0 { + break; + } + filled += produced; + } + self.account_decoded(filled)?; + Ok(filled) + } + + fn read_payload_into(&mut self, out: &mut [u8]) -> core::result::Result { + let mut filled = 0; + while filled < out.len() { + let produced = + read_payload(&mut self.input, self.payload.as_mut(), &mut out[filled..])?; + if produced == 0 { + break; + } + filled += produced; + } + self.account_decoded(filled)?; + Ok(filled) + } + + fn account_decoded(&mut self, count: usize) -> core::result::Result<(), StreamError> { + self.decoded_total = self + .decoded_total + .checked_add(count as u64) + .ok_or_else(|| xar_error(ErrorKind::Limit, "decoded total overflow"))?; + if self + .limits + .decoded_total() + .is_some_and(|maximum| self.decoded_total > maximum) + { + return Err(xar_error( + ErrorKind::Limit, + "xar decoded total exceeds limit", + )); + } + Ok(()) + } +} + +/// Reads up to `out.len()` decoded bytes from the current payload; `0` = blob end. +fn read_payload( + input: &mut R, + payload: Option<&mut Payload>, + out: &mut [u8], +) -> core::result::Result { + let Some(payload) = payload else { + return Ok(0); + }; + if out.is_empty() { + return Ok(0); + } + match payload { + Payload::Stored { remaining } => { + if *remaining == 0 { + return Ok(0); + } + let want = usize::try_from(*remaining) + .unwrap_or(usize::MAX) + .min(out.len()); + let count = input.read(&mut out[..want]).map_err(StreamError::io)?; + if count == 0 { + return Err(xar_error( + ErrorKind::Malformed, + "xar heap ended before the stored blob", + )); + } + *remaining -= count as u64; + Ok(count) + }, + Payload::Zlib { + comp_remaining, + state, + in_buf, + in_pos, + in_len, + finished, + } => { + loop { + if *finished { + return Ok(0); + } + if *in_pos == *in_len && *comp_remaining > 0 { + let want = usize::try_from(*comp_remaining) + .unwrap_or(usize::MAX) + .min(in_buf.len()); + let count = input.read(&mut in_buf[..want]).map_err(StreamError::io)?; + if count == 0 { + return Err(xar_error( + ErrorKind::Malformed, + "xar heap ended before the compressed blob", + )); + } + *in_pos = 0; + *in_len = count; + *comp_remaining -= count as u64; + } + let flush = if *comp_remaining == 0 { + MZFlush::Finish + } else { + MZFlush::None + }; + let result = inflate(state, &in_buf[*in_pos..*in_len], out, flush); + *in_pos += result.bytes_consumed; + match result.status { + Ok(MZStatus::StreamEnd) => { + *finished = true; + return Ok(result.bytes_written); + }, + Ok(_) => { + if result.bytes_written != 0 { + return Ok(result.bytes_written); + } + // No output yet: loop to feed more input. + if *in_pos == *in_len && *comp_remaining == 0 { + return Err(xar_error( + ErrorKind::Malformed, + "xar zlib blob ended without stream end", + )); + } + }, + Err(MZError::Buf) => { + if result.bytes_written != 0 { + return Ok(result.bytes_written); + } + if *in_pos == *in_len && *comp_remaining == 0 { + return Err(xar_error( + ErrorKind::Malformed, + "xar zlib blob ended without stream end", + )); + } + }, + Err(_) => { + return Err(xar_error(ErrorKind::Malformed, "xar zlib decode failed")); + }, + } + } + }, + } +} + +// ════════════════════════════════════════════════════════════════════════════ +// TOC inflate +// ════════════════════════════════════════════════════════════════════════════ + +/// Streams the zlib-compressed TOC into a `Vec` capped at `expected` bytes. +fn inflate_toc( + input: &mut R, + comp_len: u64, + expected: usize, +) -> core::result::Result, StreamError> { + let mut comp_remaining = comp_len; + let mut compressed = vec![0_u8; BUFFER]; + let mut scratch = vec![0_u8; BUFFER]; + let mut state = InflateState::new(DataFormat::Zlib); + let mut output: Vec = Vec::with_capacity(expected.min(BUFFER)); + + loop { + let count = usize::try_from(comp_remaining.min(BUFFER as u64)) + .map_err(|_| xar_error(ErrorKind::Limit, "xar TOC chunk exceeds address space"))?; + if count != 0 { + input + .read_exact(&mut compressed[..count]) + .map_err(StreamError::io)?; + comp_remaining -= count as u64; + } + let mut start = 0; + loop { + let flush = if comp_remaining == 0 { + MZFlush::Finish + } else { + MZFlush::None + }; + let result = inflate(&mut state, &compressed[start..count], &mut scratch, flush); + start += result.bytes_consumed; + if output + .len() + .checked_add(result.bytes_written) + .is_none_or(|size| size > expected) + { + return Err(xar_error( + ErrorKind::Malformed, + "xar TOC inflated beyond its declared length", + )); + } + output.extend_from_slice(&scratch[..result.bytes_written]); + match result.status { + Ok(MZStatus::StreamEnd) => { + if output.len() != expected { + return Err(xar_error( + ErrorKind::Malformed, + "xar TOC length does not match its header", + )); + } + return Ok(output); + }, + Ok(_) if result.bytes_consumed != 0 || result.bytes_written != 0 => {}, + Ok(_) | Err(MZError::Buf) => { + if comp_remaining == 0 && start >= count { + return Err(xar_error( + ErrorKind::Malformed, + "xar TOC ended without stream end", + )); + } + break; + }, + Err(_) => { + return Err(xar_error(ErrorKind::Malformed, "xar TOC decode failed")); + }, + } + if start >= count && comp_remaining != 0 { + break; + } + } + } +} + +// ════════════════════════════════════════════════════════════════════════════ +// Bounded XML pull-scanner +// ════════════════════════════════════════════════════════════════════════════ + +/// The leaf element whose text we are currently accumulating. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Sink { + None, + Name, + Type, + Mode, + Uid, + Gid, + Mtime, + Link, + Length, + Size, + Offset, +} + +/// One open `` element on the DFS stack. +#[derive(Debug, Default)] +struct Frame { + name: Option>, + kind: Option, + mode: Option, + uid: Option, + gid: Option, + mtime: Option, + link_target: Option>, + has_data: bool, + encoding: Option, + offset: Option, + stored_size: Option, + length: Option, + /// Set once the frame has been finalized into `files` (path known). + path: Option>, +} + +/// Parses the inflated TOC XML into a flat, DFS-pre-order list of resolved files. +#[allow(clippy::too_many_lines)] +fn parse_toc(xml: &[u8], limits: Limits) -> core::result::Result, StreamError> { + let mut files: Vec = Vec::new(); + let mut stack: Vec = Vec::new(); + let mut sink = Sink::None; + let mut text: Vec = Vec::new(); + // Inside an `` extended-attribute block, whose children mirror `` + // (name/offset/size/length/encoding). Those must NOT overwrite the file's own + // name or data plan, so all leaf capture is suppressed until ``. + let mut in_ea = false; + let mut i = 0usize; + let len = xml.len(); + + while i < len { + if xml[i] == b'<' { + // Commit any pending leaf text first is handled at end-tag; a new tag + // starting means structured markup begins. + if i + 1 >= len { + return Err(xar_error(ErrorKind::Malformed, "xar TOC truncated tag")); + } + match xml[i + 1] { + b'!' => { + // Comment, CDATA, or DOCTYPE — skip to matching '>' (or '-->'). + if xml[i..].starts_with(b"").map(|p| p + 3).ok_or_else(|| { + xar_error(ErrorKind::Malformed, "xar unterminated comment") + })?; + } else { + i = find_byte(xml, i + 2, b'>').map(|p| p + 1).ok_or_else(|| { + xar_error(ErrorKind::Malformed, "xar unterminated markup") + })?; + } + continue; + }, + b'?' => { + i = find(xml, i + 2, b"?>") + .map(|p| p + 2) + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar unterminated PI"))?; + continue; + }, + b'/' => { + // End tag. + let close = find_byte(xml, i + 2, b'>').ok_or_else(|| { + xar_error(ErrorKind::Malformed, "xar unterminated end tag") + })?; + let tag = trim(&xml[i + 2..close]); + if tag == b"file" { + finalize_frame(&mut stack, &mut files, &limits)?; + stack.pop(); + } else if tag == b"ea" { + in_ea = false; + } else if sink != Sink::None { + if !in_ea { + commit_text(&mut stack, sink, &text)?; + } + sink = Sink::None; + text.clear(); + } + i = close + 1; + continue; + }, + _ => {}, + } + + // Start tag (possibly self-closing). + let close = find_byte(xml, i + 1, b'>') + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar unterminated start tag"))?; + let raw = &xml[i + 1..close]; + let self_closing = raw.last() == Some(&b'/'); + let inner = if self_closing { + &raw[..raw.len() - 1] + } else { + raw + }; + let (name, attrs) = split_tag(inner); + + match name { + b"file" => { + if stack.len() >= MAX_DEPTH { + return Err(xar_error( + ErrorKind::Limit, + "xar file nesting exceeds depth cap", + )); + } + // Emit the parent directory before its first child. + finalize_frame(&mut stack, &mut files, &limits)?; + stack.push(Frame::default()); + if self_closing { + finalize_frame(&mut stack, &mut files, &limits)?; + stack.pop(); + } + }, + b"ea" => { + if !self_closing { + in_ea = true; + } + }, + b"data" if !in_ea => { + if let Some(frame) = stack.last_mut() { + frame.has_data = true; + } + }, + b"encoding" if !in_ea => { + if let Some(style) = attribute(attrs, b"style") { + let enc = classify_encoding(&style); + if let Some(frame) = stack.last_mut() { + frame.encoding = Some(enc); + } + } + }, + b"name" if !self_closing => { + sink = Sink::Name; + text.clear(); + }, + b"type" if !self_closing => { + sink = Sink::Type; + text.clear(); + }, + b"mode" if !self_closing => { + sink = Sink::Mode; + text.clear(); + }, + b"uid" if !self_closing => { + sink = Sink::Uid; + text.clear(); + }, + b"gid" if !self_closing => { + sink = Sink::Gid; + text.clear(); + }, + b"mtime" if !self_closing => { + sink = Sink::Mtime; + text.clear(); + }, + b"link" if !self_closing => { + sink = Sink::Link; + text.clear(); + }, + b"length" if !self_closing => { + sink = Sink::Length; + text.clear(); + }, + b"size" if !self_closing => { + sink = Sink::Size; + text.clear(); + }, + b"offset" if !self_closing => { + sink = Sink::Offset; + text.clear(); + }, + _ => {}, + } + i = close + 1; + } else { + // Character data. + let next = find_byte(xml, i, b'<').unwrap_or(len); + if sink != Sink::None { + text.extend_from_slice(&xml[i..next]); + } + i = next; + } + } + + if !stack.is_empty() { + return Err(xar_error(ErrorKind::Malformed, "xar TOC has unclosed file")); + } + Ok(files) +} + +/// Finalizes the top frame into `files` if it carries name+type and has not been +/// emitted yet. Used both when descending into a child and when closing a file. +fn finalize_frame( + stack: &mut [Frame], + files: &mut Vec, + limits: &Limits, +) -> core::result::Result<(), StreamError> { + let Some((last, parents)) = stack.split_last_mut() else { + return Ok(()); + }; + if last.path.is_some() { + return Ok(()); + } + let name = last + .name + .as_deref() + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar file without a name"))?; + validate_name(name)?; + let kind = last + .kind + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar file without a type"))?; + + let parent_path = parents.iter().rev().find_map(|frame| frame.path.as_deref()); + let mut path = match parent_path { + Some(prefix) if !prefix.is_empty() => { + let mut p = prefix.to_vec(); + p.push(b'/'); + p.extend_from_slice(name); + p + }, + _ => name.to_vec(), + }; + + if let Some(maximum) = limits.path_bytes() { + if path.len() > maximum { + return Err(xar_error( + ErrorKind::Limit, + "xar path exceeds configured limit", + )); + } + } + if let Some(maximum) = limits.entries() { + if files.len() as u64 >= maximum { + return Err(xar_error( + ErrorKind::Limit, + "xar entry count exceeds configured limit", + )); + } + } + + let data = if kind == EntryKind::File && last.has_data { + let length = last.length.unwrap_or(0); + let stored_size = last.stored_size.unwrap_or(length); + let offset = last + .offset + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar data without an offset"))?; + if let Some(maximum) = limits.entry_bytes() { + if length > maximum { + return Err(xar_error( + ErrorKind::Limit, + "xar entry size exceeds configured limit", + )); + } + } + Some(XarData { + encoding: last.encoding.unwrap_or(XarEncoding::Stored), + offset, + stored_size, + length, + }) + } else { + None + }; + + let record = XarFile { + path: path.clone(), + kind, + mode: last.mode, + uid: last.uid, + gid: last.gid, + mtime: last.mtime, + link_target: last.link_target.clone(), + data, + }; + last.path = Some(std::mem::take(&mut path)); + files.push(record); + Ok(()) +} + +/// Commits accumulated leaf text to the top frame's corresponding field. +fn commit_text( + stack: &mut [Frame], + sink: Sink, + text: &[u8], +) -> core::result::Result<(), StreamError> { + let Some(frame) = stack.last_mut() else { + return Ok(()); + }; + match sink { + Sink::Name => frame.name = Some(xml_unescape(trim(text))), + Sink::Type => frame.kind = Some(map_kind(trim(text))), + Sink::Mode => { + frame.mode = parse_octal(trim(text)); + }, + Sink::Uid => frame.uid = parse_u64(trim(text)), + Sink::Gid => frame.gid = parse_u64(trim(text)), + Sink::Mtime => frame.mtime = parse_iso8601(trim(text)), + Sink::Link => frame.link_target = Some(xml_unescape(trim(text))), + Sink::Length => { + frame.length = Some( + parse_u64(trim(text)) + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar bad data length"))?, + ); + }, + Sink::Size => { + frame.stored_size = Some( + parse_u64(trim(text)) + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar bad data size"))?, + ); + }, + Sink::Offset => { + frame.offset = Some( + parse_u64(trim(text)) + .ok_or_else(|| xar_error(ErrorKind::Malformed, "xar bad data offset"))?, + ); + }, + Sink::None => {}, + } + Ok(()) +} + +// ════════════════════════════════════════════════════════════════════════════ +// Small helpers +// ════════════════════════════════════════════════════════════════════════════ + +fn be16(b: &[u8], off: usize) -> u16 { + u16::from_be_bytes([b[off], b[off + 1]]) +} + +fn be32(b: &[u8], off: usize) -> u32 { + u32::from_be_bytes([b[off], b[off + 1], b[off + 2], b[off + 3]]) +} + +fn be64(b: &[u8], off: usize) -> u64 { + u64::from_be_bytes([ + b[off], + b[off + 1], + b[off + 2], + b[off + 3], + b[off + 4], + b[off + 5], + b[off + 6], + b[off + 7], + ]) +} + +fn find(haystack: &[u8], from: usize, needle: &[u8]) -> Option { + if from > haystack.len() || needle.is_empty() { + return None; + } + haystack[from..] + .windows(needle.len()) + .position(|w| w == needle) + .map(|p| p + from) +} + +fn find_byte(haystack: &[u8], from: usize, byte: u8) -> Option { + if from > haystack.len() { + return None; + } + haystack[from..] + .iter() + .position(|&b| b == byte) + .map(|p| p + from) +} + +fn trim(bytes: &[u8]) -> &[u8] { + let start = bytes + .iter() + .position(|b| !b.is_ascii_whitespace()) + .unwrap_or(bytes.len()); + let end = bytes + .iter() + .rposition(|b| !b.is_ascii_whitespace()) + .map_or(start, |p| p + 1); + &bytes[start..end] +} + +/// Splits a start-tag body into `(tag_name, remaining_attrs)`. +fn split_tag(inner: &[u8]) -> (&[u8], &[u8]) { + let inner = trim(inner); + let split = inner + .iter() + .position(u8::is_ascii_whitespace) + .unwrap_or(inner.len()); + (&inner[..split], &inner[split..]) +} + +/// Extracts the quoted value of `key` from an attribute run. +fn attribute(attrs: &[u8], key: &[u8]) -> Option> { + let mut i = 0; + while i < attrs.len() { + while i < attrs.len() && attrs[i].is_ascii_whitespace() { + i += 1; + } + let name_start = i; + while i < attrs.len() && attrs[i] != b'=' && !attrs[i].is_ascii_whitespace() { + i += 1; + } + let name = &attrs[name_start..i]; + while i < attrs.len() && attrs[i].is_ascii_whitespace() { + i += 1; + } + if i >= attrs.len() || attrs[i] != b'=' { + if name.is_empty() { + break; + } + continue; + } + i += 1; // skip '=' + while i < attrs.len() && attrs[i].is_ascii_whitespace() { + i += 1; + } + if i >= attrs.len() { + break; + } + let quote = attrs[i]; + if quote != b'"' && quote != b'\'' { + break; + } + i += 1; + let value_start = i; + while i < attrs.len() && attrs[i] != quote { + i += 1; + } + if i > attrs.len() { + break; + } + let value = &attrs[value_start..i.min(attrs.len())]; + if name == key { + return Some(xml_unescape(value)); + } + i += 1; // skip closing quote + } + None +} + +fn classify_encoding(style: &[u8]) -> XarEncoding { + match style { + b"application/octet-stream" => XarEncoding::Stored, + b"application/x-gzip" => XarEncoding::Zlib, + _ => XarEncoding::Unsupported, + } +} + +fn map_kind(type_bytes: &[u8]) -> EntryKind { + match type_bytes { + b"directory" => EntryKind::Dir, + b"symlink" => EntryKind::Symlink, + _ => EntryKind::File, + } +} + +/// Rejects empty, `.`/`..`, and any `/`-bearing basename. +fn validate_name(name: &[u8]) -> core::result::Result<(), StreamError> { + if name.is_empty() || name == b"." || name == b".." || name.contains(&b'/') { + return Err(xar_error( + ErrorKind::Malformed, + "xar file has an invalid name", + )); + } + Ok(()) +} + +fn parse_u64(bytes: &[u8]) -> Option { + if bytes.is_empty() { + return None; + } + let mut value: u64 = 0; + for &b in bytes { + if !b.is_ascii_digit() { + return None; + } + value = value.checked_mul(10)?.checked_add(u64::from(b - b'0'))?; + } + Some(value) +} + +fn parse_octal(bytes: &[u8]) -> Option { + if bytes.is_empty() { + return None; + } + let mut value: u32 = 0; + for &b in bytes { + if !(b'0'..=b'7').contains(&b) { + return None; + } + value = value.checked_mul(8)?.checked_add(u32::from(b - b'0'))?; + } + Some(value) +} + +/// Best-effort ISO-8601 (`YYYY-MM-DDThh:mm:ssZ`) → UNIX timestamp; `None` on any +/// deviation so a malformed mtime never fails a read. +fn parse_iso8601(bytes: &[u8]) -> Option { + if bytes.len() < 19 || bytes[4] != b'-' || bytes[7] != b'-' || bytes[10] != b'T' { + return None; + } + let year = i64::try_from(parse_u64(&bytes[0..4])?).ok()?; + let month = i64::try_from(parse_u64(&bytes[5..7])?).ok()?; + let day = i64::try_from(parse_u64(&bytes[8..10])?).ok()?; + let hour = i64::try_from(parse_u64(&bytes[11..13])?).ok()?; + let minute = i64::try_from(parse_u64(&bytes[14..16])?).ok()?; + let second = i64::try_from(parse_u64(&bytes[17..19])?).ok()?; + if !(1..=12).contains(&month) || !(1..=31).contains(&day) { + return None; + } + // Howard Hinnant's days_from_civil. + let y = if month <= 2 { year - 1 } else { year }; + let era = if y >= 0 { y } else { y - 399 } / 400; + let yoe = y - era * 400; + let mp = (month + 9) % 12; + let doy = (153 * mp + 2) / 5 + day - 1; + let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; + let days = era * 146_097 + doe - 719_468; + let secs = days + .checked_mul(86_400)? + .checked_add(hour * 3600 + minute * 60 + second)?; + Some(Timestamp { secs, nanos: 0 }) +} + +/// Decodes the five standard XML entities; leaves unknown `&…;` runs literal. +fn xml_unescape(input: &[u8]) -> Vec { + if !input.contains(&b'&') { + return input.to_vec(); + } + let mut out = Vec::with_capacity(input.len()); + let mut i = 0; + while i < input.len() { + if input[i] == b'&' { + if input[i..].starts_with(b"&") { + out.push(b'&'); + i += 5; + continue; + } else if input[i..].starts_with(b"<") { + out.push(b'<'); + i += 4; + continue; + } else if input[i..].starts_with(b">") { + out.push(b'>'); + i += 4; + continue; + } else if input[i..].starts_with(b""") { + out.push(b'"'); + i += 6; + continue; + } else if input[i..].starts_with(b"'") { + out.push(b'\''); + i += 6; + continue; + } + } + out.push(input[i]); + i += 1; + } + out +} diff --git a/libarchive_oxide/tests/fixtures/cab/PROVENANCE.md b/libarchive_oxide/tests/fixtures/cab/PROVENANCE.md new file mode 100644 index 0000000..b8f8439 --- /dev/null +++ b/libarchive_oxide/tests/fixtures/cab/PROVENANCE.md @@ -0,0 +1,111 @@ + + +# CAB fixture provenance + +Provenance registry for the Microsoft Cabinet (`.cab`) producers exercised by the +CAB interoperability test (`libarchive_oxide/tests/interop_cab_meta.rs`, which +reuses the RM-301 harness in `libarchive_oxide/tests/common/mod.rs`). + +## Generation policy + +**No binary fixtures are committed for CAB.** All CAB bytes used by the test are +generated **deterministically, in-code, at test run time** — hermetic, no network, +no committed blobs, nothing that can rot or need re-verification. The raw-DEFLATE +streams inside MSZIP folders are produced by the pinned `flat2` dev-dependency; +everything else (`CFHEADER`, `CFFOLDER`, `CFFILE`, `CFDATA` framing) is hand-laid +first-party bytes. + +This directory therefore contains **only this policy document** today. It exists as +the reserved location for any future byte-exact external-tool artifact (see +"External-tool escape hatch" below). + +## Producer registry + +| Label | Crate / tool | Version | Independent of arca? | Method | Generation | +|-------|--------------|---------|----------------------|--------|------------| +| `raw-cab-builder` | first-party bytes in `tests/interop_cab_meta.rs` | n/a | yes (hand-assembled MSCF container, independent of any CAB library) | Store (NONE) | in-code | +| `raw-cab-builder + flat2` | `flat2` (raw DEFLATE) inside a first-party MSZIP frame | flat2 1.x | yes (independent DEFLATE codec; arca inflates with `miniz_oxide`) | MSZIP | in-code | + +Consumer: `arca` (self, via `read_with_arca` / `SeekArchiveReader`). + +The `flat2` version is pinned in `libarchive_oxide/Cargo.toml`: + +```toml +flat2 = "1" +``` + +## Method coverage this slice + +- **CAB Store (NONE, method 0):** first-party raw builder + arca reader; multi-file + solid folder (small file, empty file, nested-path file), round-tripped + `(path, kind, content)`. +- **CAB MSZIP (method 1):** first-party raw builder driving `flat2` raw DEFLATE + + arca reader (`miniz_oxide` inflate). Covered layouts: + - single-block folder (multi-file solid folder); + - single block with real LZ77 back-references (repetitive payload); + - multi-block folder with a file spanning several `CFDATA` blocks (32-byte + blocks), exercising the folder-stream concatenation and the sliding-window + carry across blocks. +- **Structured-error paths:** an out-of-scope compression method (`QUANTUM`, 2) + yields `Unsupported`; an out-of-range `coffFiles` and a sub-`CFHEADER`-truncated + image each yield a structured error. + +### DEFLATE-codec independence note + +MSZIP is `'CK'` + raw DEFLATE with the LZ77 window carried across a folder's +blocks. The test's producer compresses with `flat2` (which wraps the C `zlib` +/ `miniz` family) while arca decompresses with the pure-Rust `miniz_oxide` +low-level inflate core driven over a 32 KiB power-of-two wrapping ring. These are +independent DEFLATE implementations, so a shared codec bug cannot mask a framing +error. The in-code multi-block fixtures deflate each block independently (a +spec-valid MSZIP producer choice), so they prove correct block concatenation and +window continuation but not a producer that deliberately emits cross-block +back-references; that stricter evidence is deferred to the external escape hatch. + +## Out of scope (structured `Unsupported`) + +QUANTUM (2), LZX (3), and cross-cabinet continuation files (`iFolder` sentinels +`0xFFFD`/`0xFFFE`/`0xFFFF`, and spanning `CFDATA` with `cbUncomp == 0`) are listed +where possible but always report a structured `Unsupported` error on payload +access — never a panic. + +## Spec reference + +- CAB container: Microsoft `[MS-CAB]` Cabinet File Format (`CFHEADER`, + `CFFOLDER`, `CFFILE`, `CFDATA`). +- MSZIP: the `'CK'`-prefixed DEFLATE variant with a window carried across blocks. +- DEFLATE stream: RFC 1951. +- Compression-method codes: NONE = 0, MSZIP = 1, QUANTUM = 2, LZX = 3 + (`typeCompress & 0x000F`). + +## License / origin + +All CAB bytes are produced at test time by first-party code and the pinned +`flat2` dev-dependency; no third-party binary artifact is redistributed. +`flat2` is dual-licensed under `MIT OR Apache-2.0`. First-party generators are +covered by this repository's `MIT OR Apache-2.0` license. + +## How to regenerate + +There is nothing to regenerate: the bytes do not exist on disk. Run the test to +reproduce them deterministically: + +```sh +cargo test -p libarchive_oxide --test interop_cab_meta +``` + +## External-tool escape hatch + +If a future slice needs a byte-exact artifact from an independent external +producer — Microsoft `makecab.exe`, `cabextract`/`libmspack`, or `gcab` — commit +it under `tests/fixtures/cab//.cab` and add a row here plus a +regeneration block recording: tool name, exact version, exact command line, +capture date, SHA-256 of the committed file, and the upstream +license/redistribution note. Regeneration must be byte-reproducible. Such an +artifact is the intended vehicle for cross-block MSZIP back-reference evidence and +for LZX/Quantum corpora once those methods move into scope. +``` diff --git a/libarchive_oxide/tests/fixtures/xar/PROVENANCE.md b/libarchive_oxide/tests/fixtures/xar/PROVENANCE.md new file mode 100644 index 0000000..2c9dc01 --- /dev/null +++ b/libarchive_oxide/tests/fixtures/xar/PROVENANCE.md @@ -0,0 +1,113 @@ + + +# XAR fixture provenance + +Provenance registry for the XAR producers exercised by the interoperability-evidence +harness (`libarchive_oxide/tests/common/mod.rs`, consumed by +`libarchive_oxide/tests/interop_xar_meta.rs`). + +XAR is a **read-only, seek-native** format in arca — there is no XAR writer — so +the interop evidence is producer-driven: independent producers emit `.xar` bytes +and arca's seek reader (`read_with_arca`) reconstructs identical +`(path, kind, content)` shapes. + +## Generation policy + +**No binary fixtures are committed for XAR.** All XAR bytes used by the interop +harness are generated **deterministically, in-code, at test run time** — hermetic, +no network, no committed blobs, nothing that can rot. The pinned +`[dev-dependencies]` in `libarchive_oxide/Cargo.toml` (`flat2` for the zlib TOC +and `x-gzip` heap blobs) are the single source of truth for producer identity; +when a pin changes, the label strings change with it as a deliberate, reviewed +edit. + +This directory therefore contains **only this policy document** today. It exists as +the reserved location for any future byte-exact external-tool artifact (see +"External-tool escape hatch" below). + +## Producer registry + +| Label | Crate / tool | Version | Independent of arca? | Methods | Generation | +|-------|--------------|---------|----------------------|---------|------------| +| `raw-xar-builder` | first-party bytes in `tests/interop_xar_meta.rs` | n/a | yes (hand-written BE header + zlib TOC XML + heap layout, independent of arca's reader) | STORED (`application/octet-stream`) + zlib (`application/x-gzip`) | in-code | + +Consumer: `arca` (self, via `read_with_arca` / `SeekArchiveReader`). + +The zlib codec used by the builder is pinned in `libarchive_oxide/Cargo.toml`: + +```toml +flat2 = "1" # zlib (RFC-1950) TOC + x-gzip heap blobs for the raw-xar-builder producer +``` + +## Layout produced by `raw-xar-builder` + +- **Header (28 bytes, BIG-endian):** magic `0x78617221` (`xar!`), `size = 28`, + `version = 1`, `toc_length_compressed`, `toc_length_uncompressed`, `cksum_alg = 0`. +- **TOC:** a zlib (RFC-1950) stream that inflates to UTF-8 XML: `` with + top-level `` roots and one nested `` carrying a child ``. + Each regular file has a `` child declaring `` (decoded), + `` (heap-relative), `` (stored), and an + ``. A `` element is present and ignored. +- **Heap:** STORED blobs are copied verbatim; `x-gzip` blobs are zlib streams, + placed at the declared offsets. Blobs are addressed only by offset/size (never + positional), matching XAR's unordered/shared-heap model. + +## Case coverage this slice + +- **Positive round trip (`xar_meta_roundtrip`, `xar_payload_bytes_exact`):** + small STORED file, larger zlib (`x-gzip`) file, empty file (no ``), + nested directory + file-in-subdir. arca reads each back to byte-identical + content, dir before its children. +- **Negative — unsupported encoding (`xar_unsupported_encoding_errors`):** a + `` with `style="application/x-lzma"` yields a structured + `ErrorKind::Unsupported` (format `"xar"`) at read time. +- **Negative — truncated header (`xar_truncated_header_errors`):** a 4-byte input + (`xar!` magic only) fails to open with a structured error. +- **Negative — TOC region past EOF (`xar_toc_region_past_eof_errors`):** a header + declaring a compressed-TOC length beyond the file yields structured + `ErrorKind::Malformed`. + +## Spec reference + +- XAR: the `xar` on-disk format — 28-byte big-endian `xar_header`, zlib-compressed + XML table of contents, and an offset-addressed heap. +- TOC compression / `x-gzip` heap blobs: RFC 1950 (zlib) — note that XAR's + `application/x-gzip` is a **zlib** stream, NOT a gzip/RFC-1952 wrapper. + +## License / origin + +All XAR bytes are produced at test time by first-party code in this repository and +the pinned `flat2` dev-dependency; no third-party binary artifact is +redistributed. First-party generators are covered by this repository's +`MIT OR Apache-2.0` license. + +## How to regenerate + +There is nothing to regenerate: the bytes do not exist on disk. Run the harness to +reproduce them deterministically: + +```sh +cargo test -p libarchive_oxide --test interop_xar_meta +``` + +## External-tool / independent-producer escape hatch + +Only ONE fully-independent XAR producer runs in this slice (the first-party +`raw-xar-builder`; arca has no XAR writer, so there is no self-producer). If a +future slice needs additional independent producers for stronger +"≥N producers can be read" evidence, the canonical external references are: + +- **`xar` CLI** (the reference C implementation / libxar), e.g. + `xar -c -f out.xar --compression=gzip `. +- **`libarchive` `bsdtar`**, e.g. `bsdtar -c -f out.xar --format=xar `. + +To commit a byte-exact artifact from either, place it under +`tests/fixtures/xar//.xar` and add a row here plus a regeneration +block recording: tool name, exact version, exact command line, capture date, +SHA-256 of the committed file, and the upstream license/redistribution note. +Regeneration must be byte-reproducible. REUSE is covered by the repo-wide +`REUSE.toml` override for `**/tests/fixtures/**` (no `.license` sidecar). diff --git a/libarchive_oxide/tests/interop_cab_meta.rs b/libarchive_oxide/tests/interop_cab_meta.rs new file mode 100644 index 0000000..93d491c --- /dev/null +++ b/libarchive_oxide/tests/interop_cab_meta.rs @@ -0,0 +1,356 @@ +// SPDX-FileCopyrightText: 2026 libarchive_oxide contributors +// +// SPDX-License-Identifier: MIT OR Apache-2.0 + +//! Microsoft Cabinet (`.cab`) read-only interoperability evidence (RM-305). +//! +//! Every byte is produced by a first-party, deterministic, in-code raw CAB +//! builder (no external tool, no committed blob). The builder hand-assembles a +//! `CFHEADER` (no flags: no reserve area, no prev/next cabinet), the `CFFOLDER` +//! table, the `CFFILE` table at `coffFiles`, and the per-folder `CFDATA` blocks. +//! Stored folders carry the payload verbatim; MSZIP folders carry a `'CK'` +//! prefix plus a raw-DEFLATE stream produced by the independent `flate2` crate. +//! +//! The tests assert the `(path, kind, content)` round trip for a multi-file +//! solid folder (small file, empty file, nested-path file), a single-block and +//! a multi-block MSZIP folder, and that an unsupported compression method and a +//! truncated/inconsistent header each surface a structured error. +#![allow( + clippy::unwrap_used, + clippy::expect_used, + clippy::panic, + clippy::cast_possible_truncation +)] + +use std::io::{Cursor, Write}; + +use flate2::{Compression, write::DeflateEncoder}; +use libarchive_oxide::{ReaderEvent, SeekArchiveReader}; +use libarchive_oxide_core::EntryKind; + +mod common; +use common::*; + +// --------------------------------------------------------------------------- +// First-party raw CAB builder. +// --------------------------------------------------------------------------- + +/// `typeCompress` method code for a stored folder. +const METHOD_NONE: u16 = 0; +/// `typeCompress` method code for an MSZIP folder. +const METHOD_MSZIP: u16 = 1; +/// `typeCompress` method code for a Quantum folder (out of scope -> Unsupported). +const METHOD_QUANTUM: u16 = 2; + +struct FileSpec { + name: Vec, + content: Vec, +} + +struct FolderSpec { + method: u16, + /// Maximum uncompressed bytes per `CFDATA` block (drives multi-block layout). + block_size: usize, + files: Vec, +} + +fn file(name: &[u8], content: &[u8]) -> FileSpec { + FileSpec { + name: name.to_vec(), + content: content.to_vec(), + } +} + +fn raw_deflate(data: &[u8]) -> Vec { + let mut encoder = DeflateEncoder::new(Vec::new(), Compression::best()); + encoder.write_all(data).unwrap(); + encoder.finish().unwrap() +} + +fn push_u16(out: &mut Vec, value: u16) { + out.extend_from_slice(&value.to_le_bytes()); +} + +fn push_u32(out: &mut Vec, value: u32) { + out.extend_from_slice(&value.to_le_bytes()); +} + +/// The assembled `CFDATA` bytes for one folder plus their total length. +fn build_folder_data(folder: &FolderSpec) -> Vec { + let stream: Vec = folder + .files + .iter() + .flat_map(|f| f.content.iter().copied()) + .collect(); + let mut data = Vec::new(); + if stream.is_empty() { + // A folder with only empty files still needs one CFDATA block so the + // decoder has something to open; emit an empty stored block. + push_u32(&mut data, 0); // csum (0 = none) + push_u16(&mut data, 0); // cbData + push_u16(&mut data, 0); // cbUncomp + return data; + } + let block_size = folder.block_size.max(1); + for chunk in stream.chunks(block_size) { + let (payload, cb_uncomp) = match folder.method { + METHOD_MSZIP => { + let mut payload = b"CK".to_vec(); + payload.extend_from_slice(&raw_deflate(chunk)); + (payload, chunk.len()) + }, + // Store and any out-of-scope method carry the raw chunk; an + // out-of-scope method's blocks are never actually decoded. + _ => (chunk.to_vec(), chunk.len()), + }; + push_u32(&mut data, 0); // csum + push_u16(&mut data, payload.len() as u16); // cbData + push_u16(&mut data, cb_uncomp as u16); // cbUncomp + data.extend_from_slice(&payload); + } + data +} + +/// Number of `CFDATA` blocks a folder produces. +fn folder_block_count(folder: &FolderSpec) -> u16 { + let total: usize = folder.files.iter().map(|f| f.content.len()).sum(); + if total == 0 { + 1 + } else { + total.div_ceil(folder.block_size.max(1)) as u16 + } +} + +fn build_cab(folders: &[FolderSpec]) -> Vec { + let total_files: usize = folders.iter().map(|f| f.files.len()).sum(); + + let coff_files = 36 + folders.len() * 8; + let file_table_size: usize = folders + .iter() + .flat_map(|f| f.files.iter()) + .map(|f| 16 + f.name.len() + 1) + .sum(); + let data_start = coff_files + file_table_size; + + // Pre-render each folder's CFDATA and compute its absolute start offset. + let mut folder_data = Vec::new(); + let mut folder_offsets = Vec::new(); + let mut cursor = data_start; + for folder in folders { + folder_offsets.push(cursor as u32); + let data = build_folder_data(folder); + cursor += data.len(); + folder_data.push(data); + } + + let mut out = Vec::new(); + // CFHEADER. + out.extend_from_slice(b"MSCF"); + push_u32(&mut out, 0); // reserved1 + push_u32(&mut out, cursor as u32); // cbCabinet (total size) + push_u32(&mut out, 0); // reserved2 + push_u32(&mut out, coff_files as u32); // coffFiles + push_u32(&mut out, 0); // reserved3 + out.push(3); // versionMinor + out.push(1); // versionMajor + push_u16(&mut out, folders.len() as u16); // cFolders + push_u16(&mut out, total_files as u16); // cFiles + push_u16(&mut out, 0); // flags + push_u16(&mut out, 0); // setID + push_u16(&mut out, 0); // iCabinet + + // CFFOLDER table. + for (folder, offset) in folders.iter().zip(folder_offsets.iter()) { + push_u32(&mut out, *offset); // coffCabStart + push_u16(&mut out, folder_block_count(folder)); // cCFData + push_u16(&mut out, folder.method); // typeCompress + } + + // CFFILE table (files grouped by folder, with per-folder running offsets). + for (index, folder) in folders.iter().enumerate() { + let mut folder_offset = 0u32; + for f in &folder.files { + push_u32(&mut out, f.content.len() as u32); // cbFile + push_u32(&mut out, folder_offset); // uoffFolderStart + push_u16(&mut out, index as u16); // iFolder + push_u16(&mut out, 0); // date + push_u16(&mut out, 0); // time + push_u16(&mut out, 0); // attribs + out.extend_from_slice(&f.name); + out.push(0); + folder_offset += f.content.len() as u32; + } + } + + // CFDATA blocks. + for data in &folder_data { + out.extend_from_slice(data); + } + assert_eq!( + out.len(), + cursor, + "computed layout must match emitted length" + ); + out +} + +/// Drives the reader to completion, returning the first error encountered (if any). +fn drive_to_error(bytes: &[u8]) -> Option { + let mut reader = match SeekArchiveReader::new(Cursor::new(bytes.to_vec())) { + Ok(reader) => reader, + Err(error) => return Some(format!("{error:?}")), + }; + loop { + match reader.next_event() { + Ok(ReaderEvent::Done) => return None, + Ok(_) => {}, + Err(error) => return Some(format!("{error:?}")), + } + } +} + +// --------------------------------------------------------------------------- +// Shared corpus: small file, empty file, nested-path file, one solid folder. +// --------------------------------------------------------------------------- + +fn corpus() -> Vec { + vec![ + file(b"readme.txt", b"hello cab\n"), + file(b"empty.dat", b""), + file(b"docs\\guide\\intro.txt", b"nested payload here\n"), + ] +} + +fn assert_corpus_round_trip(bytes: &[u8]) { + let shapes = read_with_arca(bytes); + assert_eq!(shapes.len(), 3, "expected three entries"); + + // Backslashes are normalized to '/'; every entry is a File. + let expect: [(&[u8], &[u8]); 3] = [ + (b"readme.txt", b"hello cab\n"), + (b"empty.dat", b""), + (b"docs/guide/intro.txt", b"nested payload here\n"), + ]; + for (shape, (path, content)) in shapes.iter().zip(expect.iter()) { + assert_eq!(shape.kind(), EntryKind::File, "kind for {path:?}"); + assert_eq!(shape.path(), *path, "path"); + assert_eq!(shape.content(), *content, "content for {path:?}"); + } +} + +// --------------------------------------------------------------------------- +// Store: multi-file solid folder round trip. +// --------------------------------------------------------------------------- + +#[test] +fn cab_store_multi_file_round_trip() { + let bytes = build_cab(&[FolderSpec { + method: METHOD_NONE, + block_size: 0x8000, + files: corpus(), + }]); + assert_corpus_round_trip(&bytes); +} + +// --------------------------------------------------------------------------- +// MSZIP: single-block folder round trip (raw DEFLATE via flate2). +// --------------------------------------------------------------------------- + +#[test] +fn cab_mszip_single_block_round_trip() { + let bytes = build_cab(&[FolderSpec { + method: METHOD_MSZIP, + block_size: 0x8000, + files: corpus(), + }]); + assert_corpus_round_trip(&bytes); +} + +// --------------------------------------------------------------------------- +// MSZIP: a file whose payload spans several CFDATA blocks (32-byte blocks), +// exercising the block-boundary staging and folder-stream concatenation. +// --------------------------------------------------------------------------- + +#[test] +fn cab_mszip_multi_block_round_trip() { + let payload: Vec = (0..200u32).map(|i| (i % 251) as u8).collect(); + let bytes = build_cab(&[FolderSpec { + method: METHOD_MSZIP, + block_size: 32, + files: vec![ + file(b"a.bin", &payload), + file(b"b.bin", b"tail file after a block boundary\n"), + ], + }]); + + let shapes = read_with_arca(&bytes); + assert_eq!(shapes.len(), 2); + assert_eq!(shapes[0].path(), b"a.bin"); + assert_eq!(shapes[0].content(), payload.as_slice()); + assert_eq!(shapes[1].path(), b"b.bin"); + assert_eq!(shapes[1].content(), b"tail file after a block boundary\n"); +} + +// --------------------------------------------------------------------------- +// MSZIP: highly repetitive content so real DEFLATE distance codes (LZ77 +// back-references) are emitted and resolved against the sliding window. +// --------------------------------------------------------------------------- + +#[test] +fn cab_mszip_backreferences_round_trip() { + let repetitive = b"the quick brown fox jumps over the lazy dog\n".repeat(400); + assert!(repetitive.len() < 0x8000, "must fit one MSZIP block"); + let bytes = build_cab(&[FolderSpec { + method: METHOD_MSZIP, + block_size: 0x8000, + files: vec![file(b"repeat.txt", &repetitive)], + }]); + let shapes = read_with_arca(&bytes); + assert_eq!(shapes.len(), 1); + assert_eq!(shapes[0].content(), repetitive.as_slice()); +} + +// --------------------------------------------------------------------------- +// Unsupported compression method -> structured error while streaming. +// --------------------------------------------------------------------------- + +#[test] +fn cab_unsupported_method_errors() { + let bytes = build_cab(&[FolderSpec { + method: METHOD_QUANTUM, + block_size: 0x8000, + files: vec![file(b"data.bin", b"quantum payload not decodable")], + }]); + let error = drive_to_error(&bytes).expect("unsupported method must error"); + assert!( + error.contains("Unsupported"), + "expected an Unsupported error, got: {error}" + ); +} + +// --------------------------------------------------------------------------- +// Truncated / inconsistent header -> structured error at open. +// --------------------------------------------------------------------------- + +#[test] +fn cab_truncated_header_errors() { + // A cabinet whose coffFiles points past the end of the image. + let mut bytes = build_cab(&[FolderSpec { + method: METHOD_NONE, + block_size: 0x8000, + files: vec![file(b"readme.txt", b"hello")], + }]); + // Corrupt coffFiles (offset 16) to an absurd value beyond the image. + bytes[16..20].copy_from_slice(&0x00FF_FFFFu32.to_le_bytes()); + assert!( + drive_to_error(&bytes).is_some(), + "an out-of-range CFFILE offset must error" + ); + + // A cabinet truncated below the fixed 36-byte CFHEADER. + let stub = b"MSCF\0\0\0\0".to_vec(); + assert!( + drive_to_error(&stub).is_some(), + "a sub-header truncated cabinet must error" + ); +} diff --git a/libarchive_oxide/tests/interop_xar_meta.rs b/libarchive_oxide/tests/interop_xar_meta.rs new file mode 100644 index 0000000..5a7da46 --- /dev/null +++ b/libarchive_oxide/tests/interop_xar_meta.rs @@ -0,0 +1,364 @@ +// SPDX-FileCopyrightText: 2026 libarchive_oxide contributors +// +// SPDX-License-Identifier: MIT OR Apache-2.0 + +//! XAR (`.xar`) read-only metadata + payload interoperability evidence (RM-305). +//! +//! XAR is a read-only, seek-native format in arca (no writer), so the interop +//! evidence is producer-driven: a first-party, deterministic RAW XAR byte builder +//! (`raw_xar`) emits a valid archive with a big-endian header, a zlib-compressed +//! TOC, and a heap carrying both STORED (`application/octet-stream`) and zlib +//! (`application/x-gzip`) blobs. arca reads it back through the RM-301 harness +//! (`read_with_arca`) and every (path, kind, content) must equal the canonical +//! shapes DERIVED from the shared logical corpus. +//! +//! Negative coverage (direct `SeekArchiveReader`): an unknown data-encoding style +//! yields a structured `Unsupported`, and a truncated header yields a structured +//! `Malformed`. +#![allow( + clippy::unwrap_used, + clippy::expect_used, + clippy::panic, + clippy::cast_possible_truncation, + clippy::doc_markdown +)] + +use std::io::{Cursor, Write}; + +use libarchive_oxide::SeekArchiveReader; +use libarchive_oxide_core::{EntryKind, ErrorKind}; + +mod common; +use common::*; + +// --------------------------------------------------------------------------- +// Shared logical corpus: a small file, a compressible file, an empty file, and +// a nested-directory path (dir + file-in-subdir). +// --------------------------------------------------------------------------- + +fn xar_entries() -> Vec { + let big = b"the quick brown fox jumps over the lazy dog\n".repeat(64); + vec![ + LogicalEntry::file(b"readme.txt".to_vec(), b"hello xar world\n".to_vec()), + LogicalEntry::file(b"notes.txt".to_vec(), big), + LogicalEntry::file(b"empty.txt".to_vec(), Vec::new()), + LogicalEntry::dir(b"sub".to_vec()), + LogicalEntry::file(b"sub/nested.txt".to_vec(), b"nested payload\n".to_vec()), + ] +} + +// --------------------------------------------------------------------------- +// First-party RAW XAR builder (independent, deterministic, no external tool). +// Header integers are BIG-endian. The TOC is a hand-written XML document, +// zlib-compressed with flate2 (RFC-1950). Heap blobs are STORED or zlib +// (x-gzip) at the offsets the XML declares. +// --------------------------------------------------------------------------- + +/// How the raw builder should store a given logical file on the heap. +#[derive(Clone, Copy, PartialEq, Eq)] +enum Enc { + Stored, + Gzip, +} + +fn zlib_compress(data: &[u8]) -> Vec { + let mut e = flate2::write::ZlibEncoder::new(Vec::new(), flate2::Compression::default()); + e.write_all(data).unwrap(); + e.finish().unwrap() +} + +struct XmlBuilder { + xml: Vec, + heap: Vec, + id: u64, +} + +impl XmlBuilder { + fn new() -> Self { + Self { + xml: Vec::new(), + heap: Vec::new(), + id: 1, + } + } + + fn push(&mut self, s: &str) { + self.xml.extend_from_slice(s.as_bytes()); + } + + fn next_id(&mut self) -> u64 { + let v = self.id; + self.id += 1; + v + } + + /// Opens a `` element with name + type; caller closes with `close_file`. + fn open_file(&mut self, name: &str, kind: &str) { + let id = self.next_id(); + self.push(&format!( + "{name}{kind}" + )); + self.push("064400"); + } + + fn close_file(&mut self) { + self.push(""); + } + + /// Appends a regular-file `` blob (with the given encoding) to the heap + /// and writes the matching `` element referencing its offset. + fn add_regular(&mut self, name: &str, content: &[u8], enc: Enc) { + self.open_file(name, "file"); + let (style, blob) = match enc { + Enc::Stored => ("application/octet-stream", content.to_vec()), + Enc::Gzip => ("application/x-gzip", zlib_compress(content)), + }; + let offset = self.heap.len() as u64; + let stored_size = blob.len() as u64; + let length = content.len() as u64; + self.heap.extend_from_slice(&blob); + self.push(&format!( + "{length}{offset}{stored_size}\ + " + )); + self.close_file(); + } +} + +/// Assembles the 28-byte BE header + zlib TOC + heap into a valid `.xar`. +fn assemble(toc_xml: &[u8], heap: &[u8]) -> Vec { + let toc_comp = zlib_compress(toc_xml); + let mut out = Vec::new(); + out.extend_from_slice(&0x7861_7221_u32.to_be_bytes()); // magic 'xar!' + out.extend_from_slice(&28_u16.to_be_bytes()); // size (header length) + out.extend_from_slice(&1_u16.to_be_bytes()); // version + out.extend_from_slice(&(toc_comp.len() as u64).to_be_bytes()); // toc_length_compressed + out.extend_from_slice(&(toc_xml.len() as u64).to_be_bytes()); // toc_length_uncompressed + out.extend_from_slice(&0_u32.to_be_bytes()); // cksum_alg = none + debug_assert_eq!(out.len(), 28); + out.extend_from_slice(&toc_comp); + out.extend_from_slice(heap); + out +} + +/// `ProducerCase`-compatible builder: mirrors `xar_entries` layout, choosing +/// STORED for the small file and zlib (x-gzip) for the larger/nested payloads. +fn raw_xar(entries: &[LogicalEntry]) -> Vec { + let mut b = XmlBuilder::new(); + b.push(""); + b.push(""); + b.push("2026-07-23T00:00:00Z"); + for e in entries { + let name = std::str::from_utf8(&e.path).unwrap(); + match e.kind { + EntryKind::Dir => { + // Top-level directory carrying its nested children. + b.open_file(name, "directory"); + for child in entries { + if let Some(base) = child_of(&child.path, &e.path) { + if child.kind == EntryKind::File { + let enc = if child.content.len() > 32 { + Enc::Gzip + } else { + Enc::Stored + }; + b.add_regular(base, &child.content, enc); + } + } + } + b.close_file(); + }, + EntryKind::File => { + if e.path.contains(&b'/') { + // Emitted as a nested child by its parent directory above. + continue; + } + if e.content.is_empty() { + // Empty file: no element at all. + b.open_file(name, "file"); + b.close_file(); + } else { + let enc = if e.content.len() > 32 { + Enc::Gzip + } else { + Enc::Stored + }; + b.add_regular(name, &e.content, enc); + } + }, + _ => {}, + } + } + b.push(""); + assemble(&b.xml, &b.heap) +} + +/// If `path` is a direct child of `dir` (one component deeper), returns its +/// basename component. +fn child_of<'a>(path: &'a [u8], dir: &[u8]) -> Option<&'a str> { + let rest = path.strip_prefix(dir)?; + let rest = rest.strip_prefix(b"/")?; + if rest.is_empty() || rest.contains(&b'/') { + return None; + } + std::str::from_utf8(rest).ok() +} + +// --------------------------------------------------------------------------- +// Positive: raw XAR builder -> arca reads back identical shapes + content. +// --------------------------------------------------------------------------- + +#[test] +fn xar_meta_roundtrip() { + let entries = xar_entries(); + assert_producers_agree( + &entries, + &[ProducerCase { + name: "raw-xar-builder", + encode: raw_xar, + }], + ); +} + +/// A byte-level spot check independent of the harness projection: confirms both +/// STORED and zlib payloads decode to their exact original bytes. +#[test] +fn xar_payload_bytes_exact() { + let entries = xar_entries(); + let shapes = read_with_arca(&raw_xar(&entries)); + let find = |p: &[u8]| { + shapes + .iter() + .find(|s| s.path() == p) + .unwrap_or_else(|| panic!("missing {p:?}")) + }; + assert_eq!(find(b"readme.txt").content(), b"hello xar world\n"); + assert_eq!( + find(b"notes.txt").content(), + &b"the quick brown fox jumps over the lazy dog\n".repeat(64)[..] + ); + assert_eq!(find(b"empty.txt").content(), b""); + assert_eq!(find(b"empty.txt").kind(), EntryKind::File); + assert_eq!(find(b"sub").kind(), EntryKind::Dir); + assert_eq!(find(b"sub/nested.txt").content(), b"nested payload\n"); +} + +// --------------------------------------------------------------------------- +// Negative: unsupported encoding -> structured Unsupported (direct reader). +// --------------------------------------------------------------------------- + +/// Drives `SeekArchiveReader` to completion, returning the first error (if any). +fn read_all(bytes: &[u8]) -> Result { + let mut reader = SeekArchiveReader::new(Cursor::new(bytes.to_vec()))?; + let mut entries = 0usize; + loop { + match reader.next_event()? { + libarchive_oxide::ReaderEvent::Entry(_) => entries += 1, + libarchive_oxide::ReaderEvent::Done => return Ok(entries), + _ => {}, + } + } +} + +#[test] +fn xar_unsupported_encoding_errors() { + // A single file whose encoding is an out-of-scope style. + let content = b"payload bytes that will never decode"; + let mut heap = Vec::new(); + heap.extend_from_slice(content); + let offset = 0u64; + let size = content.len(); + let xml = format!( + "\ + weird.binfile\ + {size}{offset}{size}\ + \ + " + ); + let bytes = assemble(xml.as_bytes(), &heap); + + let err = read_all(&bytes).expect_err("unsupported encoding must error"); + let archive = err + .archive_error() + .expect("expected a structured archive error"); + assert_eq!(archive.kind(), ErrorKind::Unsupported); + assert_eq!(archive.format(), Some("xar")); +} + +// --------------------------------------------------------------------------- +// Negative: truncated header -> structured Malformed (at open time). +// --------------------------------------------------------------------------- + +#[test] +fn xar_truncated_header_errors() { + // Only the 4-byte magic is present; the seek dispatch routes it to the XAR + // reader on the magic, then the header read fails. + let bytes = b"xar!".to_vec(); + let err = SeekArchiveReader::new(Cursor::new(bytes)).err(); + // Either the open errors (header read/parse) — assert it is structured when so. + let err = err.expect("truncated header must fail to open"); + // A short read surfaces as I/O; a bad/short header surfaces as Malformed xar. + if let Some(archive) = err.archive_error() { + assert_eq!(archive.format(), Some("xar")); + assert_eq!(archive.kind(), ErrorKind::Malformed); + } else { + assert!(err.io_error().is_some(), "expected io or archive error"); + } +} + +/// Regression (RM-305 adversarial review): a file carrying an `` +/// extended-attribute block — whose children mirror `` (name / offset / +/// size / length / encoding) — must NOT have that block overwrite the file's own +/// name or data plan. The reader must surface the real filename and the real +/// `` body, never the xattr name or the xattr heap window. +#[test] +fn xar_extended_attribute_does_not_clobber_data() { + let body = b"the real file body bytes"; + let xattr = b"com.apple.quarantine xattr blob payload"; + let mut heap = Vec::new(); + heap.extend_from_slice(body); // offset 0 + heap.extend_from_slice(xattr); // offset body.len() + let xml = format!( + "\ + readme.txtfile\ + {bl}0{bl}\ + \ + com.apple.quarantine{xl}\ + {bl}{xl}\ + \ + ", + bl = body.len(), + xl = xattr.len(), + ); + let bytes = assemble(xml.as_bytes(), &heap); + let shapes = read_with_arca(&bytes); + assert_eq!(shapes.len(), 1, "exactly one file entry"); + assert_eq!( + shapes[0].path(), + b"readme.txt", + "the name must not overwrite the file name" + ); + assert_eq!( + shapes[0].content(), + body, + "the heap window must not overwrite the file body" + ); +} + +/// A header that is well-formed up front but declares a TOC region past EOF must +/// be a structured `Malformed` (not a panic). +#[test] +fn xar_toc_region_past_eof_errors() { + let mut bytes = Vec::new(); + bytes.extend_from_slice(&0x7861_7221_u32.to_be_bytes()); + bytes.extend_from_slice(&28_u16.to_be_bytes()); + bytes.extend_from_slice(&1_u16.to_be_bytes()); + bytes.extend_from_slice(&9999_u64.to_be_bytes()); // toc_length_compressed >> file + bytes.extend_from_slice(&100_u64.to_be_bytes()); + bytes.extend_from_slice(&0_u32.to_be_bytes()); + // No TOC/heap bytes follow. + let err = SeekArchiveReader::new(Cursor::new(bytes)).expect_err("region past EOF must fail"); + let archive = err.archive_error().expect("structured error expected"); + assert_eq!(archive.format(), Some("xar")); + assert_eq!(archive.kind(), ErrorKind::Malformed); +}