Skip to content

Add a reference-fixture export tool for cross-repo test-data sharing #35

Description

@andremun

Summary

The Python port's tests/matlab_reference/ validation fixtures (MATLAB-trained projection
matrix, SVM support vectors, footprint polygons — used to validate Python's explore() against
MATLAB) were produced by a one-off manual MATLAB run, with no recorded provenance and no
repeatable process. As MATLAB continues to evolve, there's no way to tell when these fixtures
have gone stale, and generating updated ones is currently a bespoke manual task.

Also directly relevant, checked directly: every svm_<algo>.csv reference artifact in that
fixture set has kernel_fn = gaussian — there is no reference data at all for a
polynomial-kernel PYTHIA model. This blocks validating the Python port's own "Q1" fix (its
build→explore adapter's missing polynomial-kernel branch) against MATLAB once that's
implemented, since there's nothing to check the fix's output against.

Proposed change

Add a MATLAB export script (new file, e.g. exportReferenceFixtures.m, or an extension to
test_integration.m) that dumps training artifacts and explore-stage outputs in exactly the CSV
interchange format the Python repo's tests/matlab_reference/README.md already documents. Use
it to generate:

  1. A refreshed baseline fixture set reflecting current v0.9.0 behaviour
  2. A new polynomial-kernel PYTHIA case (opts.pythia.ispolykrnl = true), closing the gap above

Also add a small provenance.json (or similar) alongside the exports, recording the exact
MATLAB commit SHA/tag used to generate them — mirroring the Contents.m/CITATION.cff version
fields already established elsewhere in this repo.

Motivation

Turns "regenerate the Python port's validation fixtures" from a manual copy-paste exercise into
one repeatable command, and gives a concrete way to detect drift from current MATLAB behaviour.
Directly unblocks the Python roadmap's Q1 item. Longer-term (a follow-on, not part of this
issue): once #34 (CI) exists, a release-triggered job could run this script automatically and
publish fixtures as a release asset for the Python repo's CI to pull and diff against — turning
staleness detection into something automatic rather than manual.

Acceptance criteria

  • Export script/extension produces training-artifact and explore-output CSVs matching the
    format documented in the Python repo's tests/matlab_reference/README.md
  • Polynomial-kernel case generated and available for the Python port to consume
  • Provenance/version-stamp file included alongside exports
  • Script is repeatable — re-running it against an unchanged MATLAB state reproduces
    numerically-identical output

Part of the v0.9.1 milestone. Source: Python-port testing-infrastructure audit (§7). Independent of batches 1 and 2. Suggested order: after #34 (this issue's core deliverable doesn't depend on it, but its longer-term automated-publishing follow-on does).

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions