Skip to content

Test hardening: property-based tests + mutation-score gaps closed - #4

Merged
zolizoli merged 1 commit into
mainfrom
test-hardening
Jun 29, 2026
Merged

Test hardening: property-based tests + mutation-score gaps closed#4
zolizoli merged 1 commit into
mainfrom
test-hardening

Conversation

@zolizoli

Copy link
Copy Markdown
Member

Summary

Phase 3 of the docs/design/test hardening pass — test hardening. Adds property-based tests on the algorithmic core and introduces mutation testing to measure (and then close) real gaps.

Property-based tests (real contracts, not trivia)

  • backbone — disparity-filter math: alpha(0, degree) == 1, alpha monotonically decreasing in norm_weight, node strength == Σ incident weights, and alpha/alpha_ptile/norm_weight all in [0, 1]. Plus exact-value unit tests pinning the formula.
  • cooccurrence — edge weights sum to 1 (normalisation invariant); detect_collocations invariants (n-tuples drawn from the text's contiguous n-grams, bounded by top_n); exact-structure, inclusive-min_weight, and default-window tests.
  • graphscosine_similarity_matrix diagonal == 1 and values in [-1, 1].

Mutation testing (mutmut 3.6, dev-only — not in CI)

  • [tool.mutmut] scoped to the math core and its fast, spaCy-free tests; mutmut added to the dev extra.
  • Env-gated Hypothesis profile in conftest.py so mutmut's parallel workers don't trip the differing_executors health check.
  • Score ≈ 82% (362 / 442 killed). MUTATION-TESTING.md records the score, the gaps closed, and triages the remaining 80 survivors (mostly equivalent mutants — the defensive np.clip, alphabetical vocab order, unreachable weight/strength fallbacks). The disparity formula now has zero survivors.

Finding surfaced (left unfixed — see CHANGES_SUMMARY.md)

detect_collocations propagates NLTK exceptions on degenerate corpora: chi_sqZeroDivisionError, likelihood → math-domain ValueError, on all-identical tokens. Only pmi is robust. The property test uses pmi; the source crash is a behaviour change so it's left for your decision.

Verification

  • make ci green — 132 tests pass, 94% coverage, ty + ruff clean.
  • uv.lock regenerated for the mutmut dev dep (also picks up the stale 0.1.0→0.1.1).

Notes

  • Do not merge — part of the docs / design / tests review queue.

🤖 Generated with Claude Code

Property-based tests targeting real contracts on the algorithmic core:
- backbone: disparity-filter math — alpha(0, degree) == 1, monotonic decrease in
  norm_weight, node strength == sum of incident edge weights, alpha/ptile/
  norm_weight all in [0, 1]; plus exact-value unit tests pinning the formula.
- cooccurrence: edge weights sum to 1 (normalisation invariant); detect_collocations
  invariants (results are n-tuples drawn from the text's contiguous n-grams,
  bounded by top_n); exact-structure + inclusive-min_weight + default-window tests.
- graphs: cosine_similarity_matrix diagonal == 1 and values in [-1, 1].

Mutation testing (mutmut 3.6, dev-only, not in CI):
- Add [tool.mutmut] scoped to the math core (backbone/cooccurrence/graphs) and its
  fast, spaCy-free tests; add mutmut to the dev extra.
- Env-gated Hypothesis profile in conftest so mutmut's parallel workers don't trip
  the differing_executors health check.
- Score ~82% (362/442 killed). MUTATION-TESTING.md records the score, the gaps
  closed, and triage of the remaining (largely equivalent) survivors.

Finding (left unfixed, see CHANGES_SUMMARY): detect_collocations propagates NLTK
exceptions on degenerate corpora — chi_sq raises ZeroDivisionError, likelihood a
math-domain ValueError, on all-identical tokens. Only pmi is robust; the property
test uses pmi.

uv.lock regenerated for the mutmut dev dep (also picks up the stale 0.1.0->0.1.1).
make ci green (132 tests, 94% coverage).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@zolizoli
zolizoli merged commit 02d143c into main Jun 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant