Skip to content

test: kill the 136 surviving mutants in the path-guard modules - #131

Merged
cdeust merged 8 commits into
mainfrom
test/kill-path-guard-mutants
Aug 10, 2026
Merged

test: kill the 136 surviving mutants in the path-guard modules#131
cdeust merged 8 commits into
mainfrom
test/kill-path-guard-mutants

Conversation

@cdeust

@cdeust cdeust commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Closes #74

Result: 0 unexplained surviving mutants across all three modules

Module Tests Before (issue) Mutants Killed Equivalent Unexplained
cortex_viz/infrastructure/wiki_read.py tests/test_wiki_read.py 67 362 355 7 0
cortex_viz/server/http_standalone_static.py tests/test_static_path_traversal.py 53 152 147 5 0
cortex_viz/server/http_file_diff.py tests/test_git_diff_engine.py, tests/test_file_diff.py 16 120 120 0 0
Total 136 634 622 12 0

Every equivalent mutant carries a written proof in tests/MUTATION_NOTES.md
(new file, format follows the existing tests/js/MUTATION_NOTES.md
precedent) — no survivor was left un-triaged, and none were dismissed by
assertion.

Full local suite: 1310 passed, 10 skipped (was 1279/10 before this PR),
ruff check . and ruff format --check . both clean across the whole repo.

Commands and raw output

Each module was run with scripts/mutation_check.sh's pyproject-repoint
logic, invoking python3 -m mutmut run --max-children 3 directly (the
committed script doesn't expose --max-children; a local wrapper added it,
not committed — the repoint/cleanup logic is identical). Reproduce with:

scripts/mutation_check.sh tests/test_wiki_read.py cortex_viz/infrastructure/wiki_read.py
scripts/mutation_check.sh tests/test_static_path_traversal.py cortex_viz/server/http_standalone_static.py
scripts/mutation_check.sh tests/test_git_diff_engine.py,tests/test_file_diff.py cortex_viz/server/http_file_diff.py

(append --max-children 3 to the mutmut run line inside the script, or
edit pyproject.toml's [tool.mutmut] block by hand and run
python3 -m mutmut run --max-children 3 directly, to reproduce the exact
capped-parallelism runs below.)

wiki_read.py, final verification run:

>>> results:
    cortex_viz.infrastructure.wiki_read.x__page_item__mutmut_11: survived
    cortex_viz.infrastructure.wiki_read.x_read_page__mutmut_25: survived
    cortex_viz.infrastructure.wiki_read.x_list_bibliography__mutmut_18: survived
    cortex_viz.infrastructure.wiki_read.x_list_bibliography__mutmut_20: survived
    cortex_viz.infrastructure.wiki_read.x_list_bibliography__mutmut_23: survived
    cortex_viz.infrastructure.wiki_read.x_read_bibliography__mutmut_25: survived
    cortex_viz.infrastructure.wiki_read.x_save_page__mutmut_24: survived
38.07 mutations/second

362 total mutants generated (355 killed, all 7 shown above are the
documented equivalents). Load/disk before: 5,31 / 5,69 / 6,22, 27Gi free
(coordinator's baseline snapshot). Load/disk after this module's final run:
7,54 / 8,11 / 8,36, 25Gi free.

http_standalone_static.py, final verification run:

>>> results:
    cortex_viz.server.http_standalone_static.x_serve_static__mutmut_4: survived
    cortex_viz.server.http_standalone_static.x_serve_static__mutmut_5: survived
    cortex_viz.server.http_standalone_static.x_serve_static__mutmut_8: survived
    cortex_viz.server.http_standalone_static.x_serve_static__mutmut_9: survived
    cortex_viz.server.http_standalone_static.x_serve_shared_asset__mutmut_10: survived
19.43 mutations/second

152 total mutants generated (147 killed, all 5 shown above are the
documented equivalents; the module's earlier serve_file_diff "no tests"
gap — 4 mutants with zero associated tests — is also closed, now covered
and killed). Load before: 7,06 / 8,09 / 8,17, 24Gi free. Load after:
9,53 / 8,51 / 8,31, 24Gi free.

http_file_diff.py, final verification run:

>>> results:
42.37 mutations/second

120/120 killed, empty results list (no survivors, no equivalents needed).
Load before: 7,64 / 8,41 / 8,31, 24Gi free. Load after: 8,76 / 7,75 / 7,98, 22Gi free.

All three runs stayed under --max-children 3, one module at a time, never
two campaigns in parallel. The 1-minute load average crossed 10 twice
between modules (never mid-run); each time the in-flight run was allowed to
finish and the next was held until the load dropped back under 7, per the
coordinator's stop rule. mutants/ and .mutmut-cache were deleted and
pyproject.toml restored after every run — no leftover artifacts, disk
stayed in the 22–27Gi-free range throughout (never approached the incident
threshold from the 2026-08-09 disk-exhaustion writeup).

What changed (see individual commits for the full per-module rationale)

  • wiki_read.py: added coverage for list_pages/list_projects/
    list_bibliography/_title_from/_page_item/_iter_md/_parse_list
    (previously zero assertions), UTF-8 pinning under a non-UTF-8 process
    locale (locale.setlocale(LC_ALL, "C") in-process — a subprocess-based
    approach was tried first and abandoned because it's invisible to
    mutmut's coverage-based test-to-mutant association; see
    tests/MUTATION_NOTES.md for the full account), invalid-UTF-8-byte
    robustness, exact response-shape assertions, and the suffix-refusal gate
    on save_page/read_bibliography.
  • http_standalone_static.py: response framing headers, the full
    content-type table, the filename-whitelist regex boundary, three
    defense-in-depth segment-rejection payloads that net to a legitimately
    contained file if the pre-check is bypassed, and wiring tests for the
    previously-untested serve_file_diff delegate.
  • http_file_diff.py: store-forwarding proofs at all four call sites
    along the name-resolution chain, an exact-string reason assertion, and a
    full-dict-equality response-shape assertion.

No structural changes to any guard were required — every survivor was a
test gap, not a code defect, and none of the equivalent mutants indicated
dead code (verified per §9's "survivors that mark dead code are removed,
not tested" clause: list_bibliography's two equivalent mutants are
equivalent because its ASCII-only consumers can't observe encoding
differences, not because that code path is unreachable).

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

Comment thread tests/test_file_diff.py Fixed
Comment thread tests/test_file_diff.py Fixed
Comment thread tests/test_file_diff.py Fixed
Comment thread tests/test_file_diff.py Fixed
cdeust and others added 3 commits August 10, 2026 11:43
…hree survivor modules

Issue #74's 136 pre-existing mutants concentrate on branches these suites
never asserted through: wiki_read.py's list/project/bibliography readers,
the static/shared-asset response headers and content-type table, and the
file-diff resolver's exception and activity-store-lookup arms.

Adds behaviour-derived tests for each, reasoned from the contract (not from
mutmut's suggestions, since a shared-machine mutation campaign is on hold
for this commit pending a neighbouring measurement session):

- wiki_read: _parse_list's three raw shapes, _title_from's fallback ladder,
  _page_item's field precedence and OSError arm, _iter_md's bibliography
  skip, list_pages/list_projects/list_bibliography's empty-root and
  grouping/sorting/counting contracts, read_bibliography's content+size,
  save_page's UTF-8 byte accounting (distinct from character length).
- http_standalone_static: response framing headers (Content-Type,
  Content-Length, Cache-Control) on both readers, the shared-asset
  extension-to-content-type table (every mapped extension plus the
  text/plain fallback, case-insensitively), the serve_static filename
  regex's first-character boundary, and the directory-vs-file distinction
  in the whitelist.
- http_file_diff: the activity-store lookup-exception arms for both the
  basename and suffix-search resolvers, the suffix-search fallback and its
  precedence below a known-repo match, and serve_file_diff itself (missing
  name, unresolvable name, successful delegation to the diff engine, and
  the top-level exception-to-500 arm).

139 tests now pass across the three modules (was 116); full suite (1289
tests) is green. Mutation-count verification is queued for when the shared
machine clears (issue #74 comment).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ing tests

CI's lint job caught two E501 line-length violations and one RUF012
(mutable class-attribute default needing ClassVar) introduced by the
previous commit's added tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Discovered while running the issue #74 mutant-killing suite in a fresh
worktree: a bare `uv sync --frozen` leaves pytest/mutmut uninstalled
(they live in the `dev` optional-dependency group) and the failure is
silent — `uv run pytest` falls through PATH to a system interpreter
instead of erroring, so a scoped test run can look green while the full
suite silently drops every DB-backed module's collection. Recorded here
so the next worktree doesn't rediscover it the hard way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@cdeust
cdeust force-pushed the test/kill-path-guard-mutants branch from be6731b to 6030085 Compare August 10, 2026 09:43
cdeust and others added 3 commits August 10, 2026 12:18
…quivalents

Module 1 of issue #74's three-module mutant-killing campaign: mutmut on
cortex_viz/infrastructure/wiki_read.py went from 67 survivors (recorded in
the issue) to 0 unexplained -- 355/362 mutants killed, 7 documented
equivalent (5 are Python codec-name case-aliasing, 2 are list_bibliography's
encoding param having no observable effect since only ASCII bytes are ever
inspected). Full argument for each in tests/MUTATION_NOTES.md.

Notable finding during this module: a subprocess-with-LC_ALL=C approach to
testing the encoding="utf-8" pins is invisible to mutmut's coverage-based
test-to-mutant association (a spawned child process's execution is outside
the parent's coverage trace, so such a test never gets scheduled against
the mutant it targets -- it passes in isolation while the mutant still
reports "survived"). Replaced with in-process locale.setlocale(LC_ALL, "C"),
which Path.read_text/write_text genuinely consult (verified; monkeypatching
locale.getencoding()/getpreferredencoding() does not reach them) and which
mutmut's coverage tracer can see since it runs in the same process.

67 tests total in this file now (was 45), all pass locally in 0.23s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Module 2 of issue #74's three-module mutant-killing campaign: mutmut on
cortex_viz/server/http_standalone_static.py went from 53 survivors
(recorded in the issue) to 0 unexplained -- 147/152 mutants killed, 5
documented equivalent. Full argument for each in tests/MUTATION_NOTES.md.

Two equivalence classes, both provable from the guard's own final regex/
predicate rather than asserted by inspection: serve_static's early clauses
(empty/dot-prefix/null-byte) are each independently implied by its closing
`re.match(r"^[\w][\w.\-]*$", ...)`, verified against the relevant string
classes; serve_shared_asset's explicit ".." segment check is redundant
with its own dot-prefix check since ".." always starts with ".".

Also closes a real gap: serve_file_diff (the thin delegate to
http_file_diff.serve_file_diff) had zero associated tests before this
change ("no tests" in mutmut's own report, not merely "survived") --
added wiring tests proving both the handler and the store cross the
delegation boundary intact.

64 tests total in this file now (was 57), all pass locally in 0.4s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…alents

Module 3 (final) of issue #74's three-module mutant-killing campaign: mutmut
on cortex_viz/server/http_file_diff.py went from 16 survivors (recorded in
the issue) to 0 -- 120/120 mutants killed, no equivalents needed. Details in
tests/MUTATION_NOTES.md.

Every survivor here was a genuine test gap, not an unobservable difference:
four call sites forward `store` through the resolution chain
(_resolve_by_basename -> find_abs_path_by_label,
_resolve_by_relative_fragment -> find_abs_path_by_suffix,
_resolve_name -> _resolve_by_relative_fragment,
serve_file_diff -> _resolve_name) and the existing tests monkeypatched the
lookup functions with lambdas that ignored the `store` argument entirely,
so a dropped/None-swapped store was invisible to them -- fixed by recording
every store value actually received. One reason string's substring
assertion was loosened enough to pass under mutmut's "XX...XX" wrapping;
tightened to exact equality. One response dict's full shape (5 keys) was
only 3/5 asserted; switched to full-dict equality.

Full suite: 1310 passed, 10 skipped (was 1279/10 before this campaign),
ruff check and format both clean across the whole repo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@cdeust
cdeust marked this pull request as ready for review August 10, 2026 10:35
Comment thread tests/test_file_diff.py Fixed
@cdeust

cdeust commented Aug 10, 2026

Copy link
Copy Markdown
Owner Author

ZETETIC-REVIEW: APPROVE

Reviewed against the equivalence proofs and the numbers, not against the report.

The issue's own framing was the thing to reject, and it was. #74's title reads "(pre-existing, outside the changed lines)". Debt in material we touch is ours regardless of who created it; that phrasing appears nowhere in the delivery, and the work treats all 136 survivors as owed rather than inherited.

Result: 634 mutants, 622 killed, 12 equivalent with written proofs, 0 unexplained.

Module Mutants Killed Equivalent Unexplained Survivors before
wiki_read.py 362 355 7 0 67
http_standalone_static.py 152 147 5 0 53
http_file_diff.py 120 120 0 0 16

The equivalence proofs are proofs, not shrugs, which is the part that usually rots. Spot-checked the largest class — five mutants turning encoding="utf-8" into encoding="UTF-8" — against the claim that Python's codec registry normalises case. Ran it here: codecs.lookup("UTF-8") is codecs.lookup("utf-8")True. The mutant resolves to the same codec object; no input can distinguish it. An equivalent mutant is a proof obligation and this one is discharged.

The methodological find is worth more than the count. The first attempt at testing the encoding pins drove locale through a subprocess with LC_ALL=C. That reproduces the real divergence, yet the mutants kept reporting "survived" — a spawned child's execution is invisible to mutmut's coverage-based test-to-mutant scheduling, so the tests passed in isolation while never being scheduled against their targets. Replaced with in-process locale.setlocale, which Path.read_text/write_text actually consult and the tracer can see. A test that cannot be attributed to the mutant it kills is a test that proves nothing about that mutant, and this failure mode is silent in both directions. Recording it in MUTATION_NOTES.md is what stops the next person losing an afternoon to it.

Every survivor was a test gap, not a code defect — missing assertions, loose in (403, 404) and substring checks, untested delegate wiring. No guard needed restructuring, and none was restructured to make a number move.

The shared-machine protocol was honoured under real pressure. Capped at --max-children 3, one module at a time, load and disk logged before and after each run. Two load-crossed-ten events were both handled by finishing the in-flight run and pausing between modules — never mid-run, which would have produced a false result rather than a partial one. No mutants/ or .mutmut-cache left behind, on a machine that hit 100% disk earlier this month.

Debt closed in passing, correctly. The uv sync --frozen trap — pytest and mutmut live in the dev extras, so a bare sync silently falls through to the system Python via PATH — is now in this repo's CLAUDE.md with its symptom, not just the corrected command. The symptom is what saves the next reader; a fix alone leaves them unable to recognise they are in it.

Full suite 1310 passed / 10 skipped, up from the 1279 baseline. ruff check and ruff format --check clean. All 12 checks green at 19519ef, including the four required ones.

No declared violation, no deferral, no "pre-existing" left standing. Merging.

cdeust and others added 2 commits August 10, 2026 13:05
CodeQL flagged five threads (lines 275, 291, 311, 331, 347): this file
imported cortex_viz.server.http_file_diff both as named symbols
(_resolve_by_relative_fragment at module scope; _resolve_by_basename,
_resolve_name imported locally in several tests) and as a module alias
(`import ... as mod_local`, needed to monkeypatch module attributes in
three tests). Two references to the same module let a reader lose track
of which form is live where, and that ambiguity is exactly what makes a
test file fragile to edit later.

Standardized on a single top-level `import cortex_viz.server.http_file_diff
as hfd`, since the module-attribute form is the one the monkeypatching
tests require; every call site that used to import a symbol by name now
calls it through `hfd.` instead. No behavior change -- 17/17 tests in this
file still pass, full suite still 1310 passed/10 skipped.

Verified the other three test files this PR touches (test_wiki_read.py,
test_static_path_traversal.py, test_git_diff_engine.py) do not carry the
same dual-import pattern for any cortex_viz module.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@cdeust

cdeust commented Aug 10, 2026

Copy link
Copy Markdown
Owner Author

ZETETIC-REVIEW: APPROVE

Re-verdict at the current head — the branch was updated onto the main that now carries #132, so the earlier approval is stale and does not speak for this diff.

Everything in that earlier review stands: 634 mutants, 622 killed, 12 equivalent with written proofs, 0 unexplained, against 136 survivors in the issue. I re-ran the largest equivalence class here rather than trusting it — codecs.lookup("UTF-8") is codecs.lookup("utf-8") returns True, so the encoding-case mutants resolve to the same codec object and no input can distinguish them. Every survivor was a test gap, not a code defect, and no guard was restructured to make a number move.

What changed since, and why it was the right fix. Five CodeQL threads blocked the merge — main has required_conversation_resolution enabled, so an open thread stops a merge no matter how green the checks are. The finding was legitimate: tests/test_file_diff.py imported cortex_viz.server.http_file_diff both as a module and by name. Resolving the threads to silence the alert would have been the wrong move; the file now standardises on the single hfd module alias, which is the form three tests genuinely need in order to monkeypatch attributes. One path, as the guideline asks. The threads then went outdated on their own, because the flagged lines no longer exist — which is the correct way for a thread to close.

The propagation check was done rather than assumed: test_wiki_read.py, test_static_path_traversal.py and test_git_diff_engine.py were inspected for the same dual-import pattern and are clean. That pattern spreads by copy-paste between neighbouring test files, so checking was not optional.

The mutation campaign was correctly not re-run for an import-style change in a test file — the counts stand, and a re-run under a machine shared with another session's measurement would have risked two invalid results instead of preserving one valid one.

Full suite 1310 passed / 10 skipped. ruff check and ruff format --check clean. 12/12 checks green after the branch update, no unresolved threads.

No declared violation, no deferral, no "pre-existing" left standing. Merging, and closing #74.

@cdeust
cdeust merged commit b3e14d0 into main Aug 10, 2026
12 checks passed
@cdeust
cdeust deleted the test/kill-path-guard-mutants branch August 10, 2026 11:24
cdeust added a commit that referenced this pull request Aug 10, 2026
…aces (#138)

3.1.0 could not build a graph at all: commit 45d4a80 deleted
graph_event_stream's module-level emit/close/reset forwarders on the
false premise of "no caller in this repository's history", killing
every build with AttributeError on the first statement (#134, fixed by
#136). #135/#137 stops the finally-block terminator from swallowing
that same failure silently, which is why #134 reached a release
unnoticed in the first place. #131 (mutation-test hardening, no
behavior change) and #132 (automates RELEASING.md step 5, MCP Registry
publish) also landed since 3.1.0 with no further breaking change, so
this is a patch release: 3.1.0 -> 3.1.1, not a restatement of 3.1.0.

Promotes CHANGELOG's Unreleased section (#132's entry) to 3.1.1,
carries forward #134/#135's fix descriptions, and reopens an empty
Unreleased. #131 gets no entry: it changes no shipped behavior. Aligns
every version-bearing surface pyproject.toml/cortex_viz/identity.py/
server.json/.claude-plugin/plugin.json/.codex-plugin/plugin.json/
gemini-extension.json/.claude-plugin/marketplace.json/uv.lock (via
`uv lock`) plus the two surfaces check_distribution_artifact does not
cover, the README badge and docs/ROADMAP.md's "current version" line.
Verified: `python -m scripts.check_distribution_artifact` passes
against a built wheel; full suite 1317 passed/10 skipped; ruff check
and format clean. Changelog-vs-tag audit: 3.1.0/2.8.0/2.7.1/2.7.0 all
have matching tags, 3.0.0's "cut in the tree, never tagged or
published" annotation is intact and unchanged, nothing new drifted.

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: 136 surviving mutants in the four path-guard modules (pre-existing, outside the changed lines)

2 participants