Add stage-local contract tests for compiler and query pipelines

[devin-ai-integration]

Objective

Add stage-local contract tests for the compiler and query pipelines. Current test suite (6 files, 1,530 lines) covers full lifecycle end-to-end but cannot pinpoint which internal stage broke when an answer is wrong.

Compiler pipeline tests needed:

Test category	What it validates
Parser resilience	Same semantic IR under whitespace, reorder, and prose-only rewrites
Graph closure	No dangling references, route closure intact, relation typing intact
Index round-trip	Aliases, anchors, IDs, and route labels remain resolvable
Validation coverage	Known-bad inputs produce specific validation findings
Snapshot determinism	Same source → byte-identical snapshot

Query pipeline tests needed:

Test category	What it validates
Trace determinism	Same snapshot + same query → same selected support set
Seed coverage	Known question patterns activate expected candidate categories
Frontier bounds	Expansion respects hop budget and anchor limits
Projection stability	Answer surface may rephrase; support set must not silently drift
Synthesis isolation	Synthesis failure does not alter deterministic answer content

"Done" looks like: Each compiler/query stage has at least one focused contract test that fails precisely when that stage's promise breaks.

Kind

test coverage

Affected area

src/runtime/ (compiler & query engine)

Acceptance criteria

• At least one contract test per compiler stage (parser, graph, index, validation, snapshot)
• At least one contract test per query stage (normalizer, seeder, ranker, frontier, projector, synthesis)
• Tests pinpoint the failing stage — not just "end-to-end answer is wrong"
• Existing end-to-end tests remain untouched

Agent-delegable?

partially — needs human review

Additional context

• Current tests: tests/fpf-spec-runtime.test.ts (464 lines), tests/lm-studio-synthesizer.test.ts (488 lines), tests/mcp-server.test.ts (297 lines), tests/docs-projection.test.ts (189 lines), tests/runtime-path-resolution.test.ts (69 lines), tests/server-config.test.ts (23 lines)
• Best tackled after Split compiler.ts into distinct owner stages #1 (compiler split) and Split query-engine.ts into retrieval stages #2 (query engine split) since stage boundaries will be clearer
• Some tests (graph closure, trace determinism) can be written against the current monolithic modules

FPF grounding:

• B.3 (Trust & Assurance Calculus, Stable): Each stage needs its own evidence of correctness anchored at the stage level. End-to-end tests create a single assurance layer where trust is opaque — when a test fails, you cannot attribute the failure to a specific stage.
• A.15 (Role–Method–Work Alignment, Stable): Each stage's Work must be independently auditable. The current end-to-end-only tests violate A.15's principle that work evidence should be traceable to the specific role/method that produced it.
• B.3.4 (Evidence Decay & Epistemic Debt, Stable): Tests are evidence artifacts with freshness constraints. When stage code changes, stage-local tests provide targeted evidence refresh rather than requiring full end-to-end reruns.

Note: Previous version cited E.19 (Pattern Quality Gates) and F.15 (SCR/RSCR Harness). E.19 governs admission/refresh of FPF spec patterns to the canonical corpus, not software testing. F.15 is the harness for the FPF unification process (Part F), not software test suites. Both were misapplied and have been replaced with B.3, A.15, and B.3.4.

Measurable impact

Metric	Before	After (target)	How to measure
Test files	6	≥8 (new contract test files)	ls tests/*.test.ts | wc -l
Contract tests per compiler stage	0	≥5 (parser, graph, index, validation, snapshot)	grep -Ec 'describe.*parser|describe.*graph|describe.*index|describe.*validation|describe.*snapshot' tests/*.test.ts
Contract tests per query stage	0	≥5 (normalizer, seeder, ranker, frontier, projector)	grep -Ec 'describe.*normalizer|describe.*seeder|describe.*ranker|describe.*frontier|describe.*projector' tests/*.test.ts
Existing end-to-end tests modified	0	0 (untouched)	git diff HEAD -- tests/fpf-spec-runtime.test.ts tests/mcp-server.test.ts | wc -l (should be 0)

[venikman]

Take on the premise

Strongly agree. This is pure leverage — the single highest-value piece of work in the open issue list.

The end-to-end-only test posture is a real problem, and the B.3 / A.15 grounding captures exactly why:

• B.3 trust calculus — when a test fails, you have no stage-attributable evidence of where the failure originated. Debugging burns minutes to hours bisecting through stages.
• A.15 role-method-work — stage evidence must be traceable to the producing role. End-to-end tests collapse all roles' work evidence into a single assurance layer.
• B.3.4 evidence freshness — stage-local tests mean stage-local evidence refresh. A change to the seeder doesn't require re-running the full end-to-end suite to trust the seeder.

Note on the FPF grounding correction: the swap from E.19/F.15 → B.3/A.15/B.3.4 is correct. E.19 is about admitting patterns to the canonical corpus (spec governance), not software test governance. The updated grounding actually fits.

Suggestions to strengthen the acceptance criteria:

• Stage tests must run in <5s collectively, so they can be the default fast-feedback loop (no heavy spec reloads per test)
• At least one test per stage must be a negative test — e.g., corrupted input produces the expected stage-local error, not just a generic pipeline failure
• Tests cite the stage's typed I/O contract (from Split query-engine.ts into retrieval stages #2) explicitly — so that changing the stage's contract fails the test at build time, not just runtime

Ordering concern: this issue says it's best tackled after #2 (query split). Agreed, but some tests (graph closure, determinism, parser resilience) don't need the split and can land now on the monolith. PR #32 actually does both — stage-named tests for query, plus compiler stage tests that don't depend on #31.

Problem is worth solving; PR #32 is a good first cut (see my review there for specific test nits).

[devin-ai-integration]

Merger plan update

Status: PR #32 — part of coordinated batch with #28 + #31

Incorporating venikman's feedback from this issue:

1. "Strongly agree. Pure leverage." — Confirmed as highest-value work in the open list.

2. Acceptance criteria additions from owner:

   • Stage tests must run in <5s collectively (fast-feedback loop)
   • At least one negative test per stage (corrupted input → stage-local error)
   • Tests cite the stage's typed I/O contract from Split query-engine.ts into retrieval stages #2 explicitly

3. PR test: add stage-local contract tests for compiler and query pipelines #32 review status: 39 comments already addressed in previous session (path resolution, determinism, magic numbers, silent passes, division-by-zero guard). The remaining Codex/CodeRabbit suggestions (whitespace resilience test, malformed fixture test, synthesis isolation assertions) are quality improvements I'll evaluate during the batch landing.

4. Coordinated batch: PR test: add stage-local contract tests for compiler and query pipelines #32 lands with refactor: extract inline heuristic seeds and route constraints to snapshot-attached rules #28 (seeder rules) and refactor: split query-engine.ts into retrieval stages #31 (query split) per owner's recommendation.

Merge order: PRs #28 + #31 + #32 merge as a coordinated batch after #29 and #27.