Test hardening: property-based tests + mutation-score gaps closed - #4
Merged
Conversation
Property-based tests targeting real contracts on the algorithmic core: - backbone: disparity-filter math — alpha(0, degree) == 1, monotonic decrease in norm_weight, node strength == sum of incident edge weights, alpha/ptile/ norm_weight all in [0, 1]; plus exact-value unit tests pinning the formula. - cooccurrence: edge weights sum to 1 (normalisation invariant); detect_collocations invariants (results are n-tuples drawn from the text's contiguous n-grams, bounded by top_n); exact-structure + inclusive-min_weight + default-window tests. - graphs: cosine_similarity_matrix diagonal == 1 and values in [-1, 1]. Mutation testing (mutmut 3.6, dev-only, not in CI): - Add [tool.mutmut] scoped to the math core (backbone/cooccurrence/graphs) and its fast, spaCy-free tests; add mutmut to the dev extra. - Env-gated Hypothesis profile in conftest so mutmut's parallel workers don't trip the differing_executors health check. - Score ~82% (362/442 killed). MUTATION-TESTING.md records the score, the gaps closed, and triage of the remaining (largely equivalent) survivors. Finding (left unfixed, see CHANGES_SUMMARY): detect_collocations propagates NLTK exceptions on degenerate corpora — chi_sq raises ZeroDivisionError, likelihood a math-domain ValueError, on all-identical tokens. Only pmi is robust; the property test uses pmi. uv.lock regenerated for the mutmut dev dep (also picks up the stale 0.1.0->0.1.1). make ci green (132 tests, 94% coverage). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 3 of the docs/design/test hardening pass — test hardening. Adds property-based tests on the algorithmic core and introduces mutation testing to measure (and then close) real gaps.
Property-based tests (real contracts, not trivia)
alpha(0, degree) == 1, alpha monotonically decreasing innorm_weight, nodestrength == Σ incident weights, andalpha/alpha_ptile/norm_weightall in[0, 1]. Plus exact-value unit tests pinning the formula.detect_collocationsinvariants (n-tuples drawn from the text's contiguous n-grams, bounded bytop_n); exact-structure, inclusive-min_weight, and default-windowtests.cosine_similarity_matrixdiagonal== 1and values in[-1, 1].Mutation testing (mutmut 3.6, dev-only — not in CI)
[tool.mutmut]scoped to the math core and its fast, spaCy-free tests;mutmutadded to thedevextra.conftest.pyso mutmut's parallel workers don't trip thediffering_executorshealth check.MUTATION-TESTING.mdrecords the score, the gaps closed, and triages the remaining 80 survivors (mostly equivalent mutants — the defensivenp.clip, alphabetical vocab order, unreachable weight/strength fallbacks). The disparity formula now has zero survivors.Finding surfaced (left unfixed — see
CHANGES_SUMMARY.md)detect_collocationspropagates NLTK exceptions on degenerate corpora:chi_sq→ZeroDivisionError,likelihood→ math-domainValueError, on all-identical tokens. Onlypmiis robust. The property test usespmi; the source crash is a behaviour change so it's left for your decision.Verification
make cigreen — 132 tests pass, 94% coverage,ty+ruffclean.uv.lockregenerated for themutmutdev dep (also picks up the stale 0.1.0→0.1.1).Notes
🤖 Generated with Claude Code