test(testreport): pin the SLFO packages envelope against a real metadata record - #444
Merged
mimi1vx merged 2 commits intoAug 12, 2026
Merged
Conversation
openSUSE#397) The `packages` map being keyed `standard` was load-bearing and never checked against a real SLFO record. `packages_for_map` routes SLFO through a branch that fires only for a single `standard` key, and if real metadata were keyed anything else — a product version, a product name — seeding would resolve to nothing. The only SLFO end-to-end fixture shipped `"packages": {}`, and the one populated SLFO probe was synthetic, guessing the key as `6.1` and the separator as `_`. Capture the real metadata.json for SUSE:SLFO:1.1:418286 as a fixture and test against it. The map is keyed `standard` and nothing else, and entries are `<name> = <version>` — the same three-token form the Maintenance golden already pins. The synthetic probe, now redundant against a real record, is removed. Two further gaps this closes. The `map.len() == 1` half of the `standard` guard was invisible to the suite: the existing single-key test cannot distinguish it from a bare `contains_key`, and dropping it leaves all 264 lib tests green. And nothing anywhere asserted `base().packages` after `TestReport::read` — no unit test drives `read()` at all — so the on-disk loader could stop parsing envelopes for SL reports without a package-level failure. `get_package_list()` is deliberately only a labelled secondary assertion: it flattens every product and never consults `base_version`, so it stays green under a total failure of the `standard` assumption. Also fixes a probe that could not distinguish version-from-second-token from version-from-third, having used the same value for both. No production behaviour changes. The fixture is field-for-field the captured record apart from `packager`, replaced with the address the lifecycle fixtures already use.
…penSUSE#397) The envelope docs rendered a package entry as `"pkg _ version"`, using Rust's ignored-binding notation for the token the parser throws away. Read as a wire format instead, it says SLFO ships underscore-separated entries — which is how the synthetic test fixtures came to use `_`, while every real record uses `=`. Write the shape as `<name> <op> <version>` with an observed example, and say outright that the middle token is not significant. Comments only.
mimi1vx
approved these changes
Aug 11, 2026
mimi1vx
left a comment
Member
There was a problem hiding this comment.
Test/docs-only; the sole src edit is rustdoc prose, so no production behaviour changes.
Verified on the branch: cargo fmt --all --check, cargo clippy -p mtui-testreport --all-targets -- -D warnings, RUSTDOCFLAGS="-D warnings" cargo doc -p mtui-testreport,
and cargo test -p mtui-testreport (154 integration + 264 lib) all green.
Both new assertions were observed red against broken code, so they pin something:
- Dropping
map.len() == 1 &&from thepackages_for_mapguard (src/testreport.rs:254)
reddens onlypackages_for_map_second_product_key_disables_standard_branch— no
pre-existing test caught that mutation. - The fixture fix
"test_pkg 1.0 1.0"->"1.0 2.0"makes a version/token swap in the
parser detectable; before it, that assertion could not fail. A real fix, not cosmetic.
Also checked: both sides sort, so no HashMap-iteration flake; no fixture-directory
globbing that the new slfo_metadata.json could perturb; no secrets in the fixture.
Non-blocking notes, all wording/placement:
tests/metadata_parsers.rs:586-598—packages_for_map_second_product_key_disables_standard_branch
is a pure unit test (hand-builtHashMap, no I/O) sitting in an integration module.
Every siblingpackages_for_maptest is colocated insrc/testreport.rs. Moving it
would put the probe next to the guard, reuse the existingbase_with_packageshelper,
and drop the public-API import for something reachable in-crate. The two fixture-driven
tests in the same file are correctly placed.- The
#397 T1/T2/T2b/T3prefixes are task-breakdown scaffolding that goes stale once the
issue closes; the bare issue number carries the same provenance. - The 15-20 line doc blocks are largely review argument. The durable parts are the "what
breaks if this changes" lines — thatbase_versionmust not be a map key or the
standardbranch isn't under test, and thatget_package_list()is the weaker
secondary assertion. The rest reads better in the PR description, where it already is. - The fixture is
include_str!'d into two consts in the onetests/it.rsbinary, each
with its own provenance doc to keep in sync by hand; onepub(crate)const would do. - The PR body says "byte-for-byte ... one substitution", while
tests/metadata_parsers.rs:29says it is re-serialized with sorted keys and re-indented.
The in-tree doc is the accurate one — worth fixing the body, since fixture provenance is
the point of the change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #397.
What the evidence says
#397 asked for a real SLFO
metadata.jsonand left two questions open. Both arenow answered from observed records, fetched from TeReGen for two independent
updates —
SUSE:SLFO:1.1:418286(the one #396 named) and a currentSUSE:SLFO:1.2:6766:<name> = <version>— theparser's strict form is right, with
=as the discarded middle token.standard, and nothing else — sopackages_for_map'ssingle-key branch is the live path for SLFO, and seeding does reach hosts.
Both assumptions hold, so this is not a hidden #396 root cause and no
production behaviour changes. It is the coverage the issue asked for, plus the
correction of a belief the tree was encoding.
To be precise about what was and was not already covered: the
=form andversion-is-token[2] were already pinned against real data by the Maintenance
golden, which asserts whole-map equality on a captured record. The genuinely
unpinned things were the
standardkey, themap.len() == 1half of theguard, and any assertion at all on
base().packagesafterTestReport::read.The belief that was wrong
slfo_populated_envelope_parseslanded with #396 as an explicitly syntheticprobe (its own doc said real-fixture coverage remained #397) and guessed both
halves wrong: key
6.1, separator_. So the tree held two irreconcilableclaims —
packages_for_map's doc saying SLFO ⇒standard, and that test sayingSLFO ⇒
6.1. It is retargeted onto the observed shape.The
_came from the envelope rustdoc writing"pkg _ version", which isRust's ignored-binding notation for the discarded token; read as a wire format
it says SLFO ships underscores. Second commit fixes the notation.
What is pinned, and what is not
json_parser_parses_slfo_real_fixturecontains_key) and both real versionsslfo_real_fixture_seeds_packages_for_any_base_versionpackages_for_mapwith abase_versiondeliberately absent from the map; assertsrequired(), not just namespackages_for_map_second_product_key_disables_standard_branchmap.len() == 1half of the guardmake_testreport_slfo_real_metadata_populates_packagesTestReport::readTwo deliberate restraints:
get_package_list()is only a labelled secondary assertion. The issuesuggests asserting it is non-empty; it flattens every product and never
consults
base_version, so it stays green under a total failure of thestandardassumption. Resting the case on it would have been another checkthat cannot fail.
Session::connect_oneis private andTarget::connecthard-codesSshConnection::connectwith no injectable seam,so no test can reach the production seeding call site. The
packages_for_maphalf is re-implemented explicitly instead, and the test doc says so rather
than implying coverage that does not exist.
Mutation evidence
Every new assertion has a mutation applied to production code, observed red, and
reverse-restored. Two worth naming:
map.len() == 1frompackages_for_map's guard leaves the entire264-test lib suite green — the existing single-key test cannot distinguish it,
because its map has one key either way. The new probe is the only thing in the
workspace that catches it.
base().packagesright after the loader'sparse_strreddens the newlifecycle test alone, which is what isolates its added coverage from the
eleven pre-existing SLFO lifecycle tests.
Also fixed along the way: a pre-existing probe in the same file used the same
value for an entry's second and third token, so it could not distinguish
version-from-second-token from version-from-third.
Fixture provenance
crates/mtui-testreport/tests/fixtures/metadata/slfo_metadata.jsonisfield-for-field the captured record with one substitution:
packager→someone@suse.com, the address the lifecycle fixtures already use, so a namedindividual's address is not published. Nothing under test depends on it. Say the
word if you would rather have it verbatim — the existing golden fixture already
ships a real address and a real IBS URL, so either is consistent with precedent.
Field-for-field, not byte-for-byte: the record arrives from TeReGen as a single
~950-byte line, and the fixture is re-serialized with sorted keys and 4-space
indent to match its golden sibling. Every value is byte-identical; the file is
not. So a diff against a fresh re-fetch is reformatting, not tampering. The
in-tree provenance note on the const says the same.
Every other value — RRID, products, packages, testplatform, repository, bugs,
SRCRPMs, product_composer — is the real record, unedited.