add vibe coded regression test suite#40
Merged
Merged
Conversation
joegomes
approved these changes
Jun 18, 2026
joegomes
left a comment
Collaborator
There was a problem hiding this comment.
Tests and example update LGTM!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add golden-output regression tests and raise coverage
Why
The FSM is fully deterministic given the same inputs and the EMT calculator, but
the test suite never validated its output — the existing tests only ran
fsm_example.pyin a subprocess and asserted the exit code. That left two gaps:--cov=src/mlfsm, so coverage was stuck.This PR adds a golden-output regression test that makes it trivial to tell
whether a change is purely efficiency/flags or actually changes numerical output,
and adds targeted unit tests to raise coverage.
What
Golden regression suite
cart,lst,ric), a small deterministic EMTrun produces
fsm.out, compared against a committed golden file.abs_tol=1e-5), not rawbyte equality — CI runs on both
linux-64andosx-arm64, where the lastprinted digit can drift. Structure and text changes remain hard failures, with
line-numbered diffs on mismatch.
Date/Time:) is masked before comparison.pytest tests/test_golden.py --update-goldens(or
UPDATE_GOLDENS=1); review the diff ingit diff tests/data/golden.Coverage
test_script,test_cartesian,test_stepsize) to in-processrun_fsmcalls so they count toward coverage.utils,output,interp, andcos, plus tests for thecomparison helpers themselves.
interp.py100%,output.py93%,utils.py98%).Files
tests/conftest.py,tests/golden_utils.py,tests/test_golden.py,tests/test_golden_utils.py,tests/test_utils.py,tests/test_output.py,tests/test_interp.py,tests/test_cos.pytests/data/reactions/diels_alder/(frozen inputs) andtests/data/golden/diels_alder_{cart,lst,ric}/fsm.outtests/test_script.py,tests/test_cartesian.py,tests/test_stepsize.py(subprocess → in-process)pyproject.toml: per-file ruff ignores for test codeNo
src/mlfsmbehavior changes — the goldens exist precisely to detect such changes.Notes
utils.float_check's docstring claims 0-D ndarrays are supported, but the coderaises
TypeErroron them. Tests assert the actual behavior; aligning the twois a possible follow-up.
opt.py(the in-developmentInternalsOptimizer/
optcoords="ric"path) andcos.py/coords.pyfrontier-growth edge cases.Verification
pytest tests/— 44 passedpytest tests/ --cov=src/mlfsm— ~78%ruff check .,ruff format --check .,mypy .— all clean