Test hardening: property-based tests + mutation-score gaps closed - #8
Merged
Conversation
Property-based tests (new tests/strategies.py for shared strategies): - SVDAlgebra.save_model/load_model round-trip preserves vocabulary + embeddings. - ProcrustesAligner.transform is an isometry (orthogonal map preserves norms). - ProcrustesAligner save/load round-trip preserves rotation and anchors. - PPMIComputer output is non-negative for arbitrary valid count inputs. Mutation testing (MUTATION_TESTING.md): - mutmut v3 is incompatible with this src-layout/editable/Cython setup, so a guided mutation pass (the /mutation-testing skill's method) was used. - Score on a 9-mutation core sample: 5/9 -> 8/9 (89%) after closing gaps. - Added 3 targeted tests killing the worthwhile survivors: - CMS.query returns the row-wise minimum (min vs max was indistinguishable). - get_heavy_hitters threshold is strict (`>` not `>=`). - find_common_words returns the intersection (not union). - One surviving mutation (align_with `>=`/`>`) left as documented brittle float-equality boundary. All tests, ty, ruff, and doctests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Challenges the test suite with property-based tests and a mutation pass.
Property-based tests (Hypothesis)
New
tests/strategies.pyholds shared strategies (trainable corpora, orthogonal matrices, embedding pairs, valid PPMI-kernel inputs). New properties target real contracts, not trivia:SVDAlgebrasave/load round-trip — a reloaded model has identical vocabulary and embeddings.ProcrustesAligner.transformis an isometry — the orthogonal map preserves vector norms (the basis of distance-preserving alignment).ProcrustesAlignersave/load round-trip — rotation and anchor bookkeeping survive a pickle cycle.PPMIComputeroutput is non-negative for arbitrary valid count inputs (a hard precondition for NMF).These complement the 5 existing property tests (distance symmetry, most_similar contract, learned-matrix orthogonality, topic distributions, CMS underestimate/merge).
Mutation testing (
MUTATION_TESTING.md)mutmutv3 was added as instructed but is incompatible with thissrc/-layout + editable-install + Cython setup (it runs from amutants/copy wherechronowords.algebraisn't importable). Rather than commit broken config, the dep/config were removed and a guided mutation pass was run — the method the/mutation-testingskill actually prescribes.CMS.queryreturns the row-wise minimum (min vs max was indistinguishable without forced collisions).get_heavy_hittersthreshold is strict (>, not>=).find_common_wordsreturns the intersection (fit-based tests missed this —fitsilently skips non-shared words).align_with>=/>— an unreachable float-equality boundary), documented.Verification
uv run pytest— 55 passeduv run pytest --doctest-modules src/chronowords— passeduv run ty check src tests— cleanuv run ruff check . && uv run ruff format --check .— cleanPart of the docs/design/tests hardening effort. Do not merge — review queue only.
🤖 Generated with Claude Code