packing: replace greedy merge with statistical partitioning

[jlebon]

The old algorithm used a greedy merge approach (BinaryHeap-based,
minimizing TEV loss per merge) which was fundamentally unstable: small
changes in input caused completely different merge decisions, resulting
in poor layer reuse across updates.

Additionally, because of its greediness, it easily fell into a local
optimum with this giant catch-all bucket with low stability and safer
more stable items in the other layers.

Replace it with a two-phase statistical partitioning approach inspired
by rpm-ostree's chunking algorithm:

Phase 1 classifies components by size using median + MAD, giving large
components (linux-firmware, kernel, firefox) their own singleton layers.
Phase 2 assigns all remaining components to bins using stability tiers
(high/mid/low via mean+stddev) and deterministic name-based hashing,
which ensures stable bin membership across builds without needing to
track prior build state.

Also remove the stability fallback that assigned min(known)/2 to
components without stability data (xattr, bigfiles, unclaimed). These
now stay at 0.0 and are naturally handled by the stability tiers.

Benchmarked against rpm-ostree's native chunking on FCOS F43 (10
biweekly stable releases) and Silverblue F43 (10 daily builds):

FCOS:       49.2% reuse, 826 MiB avg download (rpm-ostree: 33.0%, 1.1 GiB)
Silverblue: 89.5% reuse, 543 MiB avg download (rpm-ostree: 71.8%, 1.5 GiB)

Assisted-by: OpenCode (Claude Opus 4.6)

[gemini-code-assist]

Code Review

This pull request replaces the existing greedy clustering algorithm for OCI layer packing with a new statistical partitioning approach inspired by rpm-ostree. The new algorithm classifies components into layers based on size outliers (using median and MAD) and stability tiers, utilizing name-based hashing for deterministic binning. Feedback identifies critical logic errors where components could be silently dropped if the layer budget is exhausted or if a stability tier is allocated zero bins. Additionally, a fix is required for the use of an unstable Rust feature in the median calculation.

[jlebon]

Don't look too much at the code. This is still rough/raw from Opus. I still need to carefully go through it, but the results are encouraging. I made it sweep for optimal parameters for the FCOS and Silverblue sets (which hopefully generalizes well to non-Fedora data sets -- will try to do more validation there).

[cgwalters]

There's plenty of other container images to look at, I think registry.redhat.io/rhelai1/bootc-cuda-rhel9 is another one we should have in our references. IIRC Chris was looking at tweaking its build to use user.component.

[castrojo]

ghcr.io/projectbluefin/dakota:latest is a buildstream based image. We want to just start with chunkah out of the gate, would love to offer a prototype. We at least have a few people on this, and I'm dogfooding on metal! projectbluefin/dakota#212

Happy to turn on whatever is necessary to help!