feat: integration harness against python-sdk v2.0.0 as the 2026-07-28 reference peer (#367)

[shigechika]

Implements #367 per the FINAL DESIGN comment (verdict GO-WITH-CHANGES, all 8 required changes applied).

What this does

Phase 2 of #270 is pinned by 1,635 unit tests, every one against hand-built mocks shaped by our own reading of spec rev 2026-07-28. Design Amendment A9 is the standing proof that this can fail silently: the relay put resourceSubscriptions at the top level of params, its mocks agreed, the suite was green, and a compliant server would have ignored the filter while the relay read the ack echo as "nothing honored" — a double silent no-op.

This PR drives the relay end to end against python-sdk v2.0.0, the first official implementation of that spec revision: a real MCPServer under uvicorn on an ephemeral localhost port, a real mcp-stdio subprocess on --protocol-era auto, and a hand-written 2025-06-18 stdio client on its stdin/stdout. Six scenarios, all green.

Result: every wire shape Phase 2 modelled by hand matches the reference implementation — no shape discrepancy, no relay change needed, the STOP RULE did not fire.

It did, however, immediately earn its keep on timing: the first CI run exposed a platform-dependent behaviour of the reference peer that 64 consecutive local runs could not (see the headline finding below).

Housekeeping (first commit)

Recorded debt from the #270 completion records, cleared first so a harness failure can never be confounded with a known advisory.

	Change	Kind
(a)	CLAUDE.md truth-up: run()'s modern-era stdin reader thread; the signal-handler constraint now covers both transports	wording
(b)	_mrtr_handle_cancel docstring: the close-before-enqueue claim was false in the pre-headers window; the outcome holds via the abort event + D2 checks	wording
(c)	_consume_restart(stop, restart) or reopen, not reopen or … — a stale restart flag could defer #352 R5F1's deliberate re-raise by an iteration	behavior
(d)	Once-latch for the cap-refusal log, as _LISTEN_MAX_SUBSCRIPTIONS' comment already promised. Global, not per-URI	behavior

Revert-checks (undone via string replace against a backup, never git restore):

• (c) → test_a_reopen_still_consumes_the_pending_restart fails with Failed: DID NOT RAISE <class 'RuntimeError'>; restored → green.
• (d) → test_subscribe_cap_refusal_is_logged_exactly_once fails with assert 2 == 1; restored → green. Every other assertion in that test still passed while reverted, so it fails only on the behavior it pins.

AC3 (legacy-era byte-invariance) confirmed: both behavior changes live behind run()'s if era == "modern": gate — _listen_stream_loop is only ever started by _start_listen_stream, and _handle_resource_subscription is only reachable through _handle_modern_special_method. The legacy wire is untouched.

Scenarios

#	Test	Phase 2 record validated	Result
1	test_era_probe_classifies_the_v2_server_as_modern	Phase 1 era detection; PR B interop note "era probe classifies v2 servers as modern" (previously source-verified only)	PASS
2	test_initialize_tools_list_and_call	AC3 header layer (Mcp-Method/Mcp-Name), _meta injection, resultType pass-through	PASS
3	test_mrtr_full_loop_under_the_original_id	PR C #356 — inputRequests/inputResponses maps, retry under the original id (designated watchpoint)	PASS
4	test_subscribe_publish_and_forward	PR B #358 / Design A9 — nested notifications.resourceSubscriptions, honored-subset ack echo, five-gate forwarding (watchpoint #1)	PASS
5	test_listen_graceful_end_does_not_reconnect	PR A #352 graceful-end signals (strong form, see below)	PASS
6	test_cancel_aborts_the_in_flight_post	PR D #362 — disconnect-as-cancellation	PASS

Suite: 6 passed in ~18 s locally (design budget 15-25 s) and 6 passed in 26.9 s on ubuntu-latest in CI, 32/32 clean over a repeated local hunt. The Linux number is larger for a reason that turned out to be the most interesting result in the PR — see below.

Owner questions — proposed defaults ADOPTED

1. Scenario 5 strength → adopted, then exceeded. The default allowed the loosened assertion if no in-server hook existed. Verification found one: ListenHandler.close() is public and documented ("each stream drains its buffered events and sends its SubscriptionsListenResult as the final frame — the spec's graceful closure flow"). Scenario 5 therefore asserts the strong form. Reaching the instance needs one private attribute (MCPServer builds its own handler and does not hand it back), so the harness registers its own through the documented add_request_handler. This is a deviation from the default in the harness's favour, on a verified branch the design itself specified.
2. Pin policy → adopted: range mcp>=2.0,<3, revisit only after ≥2 spurious breakages.
3. Housekeeping packaging → adopted: same PR, first commit.
4. "uvicorn (SSE mode)" → adopted as read. The relay stays on streamable-http; the server's json_response=False default does the SSE framing and subscriptions/listen always takes the SSE path. No flag on either side.

Verified at implementation time

Everything the design marked "never silently assume", checked against the installed wheel.

• mcp==2.0.0 from PyPI matches the design's reading of the tag. Companions: mcp-types==2.0.0, and the SDK's HTTP client is the separate httpx2==2.9.1 distribution, so it coexists with this project's runtime httpx==0.28.1 in one venv exactly as predicted.
• mcp.__version__ DOES NOT EXIST. The design's CI smoke snippet would have failed on every run. The version lives only in distribution metadata; both the workflow step and the in-suite session assert now use importlib.metadata.version("mcp").
• uvicorn.Server.started behaved reliably, but the TCP-connect fallback is kept in the same deadline loop as specced.
• In-server listen-end hook: FOUND (see owner question 1). Lens 1 had found only the backlog-overflow send.close().
• Relay stderr marker: EXISTS. run() logs protocol era: modern (auto-detected) unconditionally on the auto path, so scenario 1 asserts it. This matters: a v2 server answers a legacy initialize too, so the handshake alone does not discriminate modern from legacy.
• elicitation: {} is accepted by the reference peer. A 2025-06-18 client declares the bare object; the 2026-07-28 schema added form/url sub-keys. The peer honours the spec's "elicitation: {} is equivalent to declaring support for form mode only" and runs the resolver. Withholding the capability instead yields -32021 with data.requiredCapabilities = {"elicitation": {"form": {}}} — so the relay's pass-through-what-the-client-declared behaviour (PR C's un-strip) is exactly right, and scenario 3 depends on it.
• MRTR shapes match Phase 2's model exactly: resultType: "input_required", inputRequests keyed module:qualname, entry {method: "elicitation/create", params: {message, mode: "form", requestedSchema}}, opaque requestState string, retry with inputResponses: {key: {action, content}} + echoed requestState.
• The modern envelope is PASSED THROUGH, not translated. A complete tools/call result reaches the 2025-era stdio client verbatim — resultType, structuredContent and the server's _meta included. The relay reads resultType only to spot input_required and route the MRTR branch. This is sound (MCP results are open, so a 2025-era client ignores unknown fields) but it was undocumented and untested; scenario 2 now pins it, because a future "normalize the envelope" change would otherwise alter what every downstream client sees with nothing going red.
• A9 echo shape confirmed on the wire: the ack is notifications/subscriptions/acknowledged with params.notifications.resourceSubscriptions nested, and every frame carries params._meta["io.modelcontextprotocol/subscriptionId"]. This is the side A9 never exercised, and it agrees with the relay.

Headline finding: disconnect-as-cancellation is ~1500× slower on Linux

Found by the first CI run, which is the entire argument for this harness existing.

Scenario 6 passed 64/64 locally on macOS and failed on ubuntu-latest. The diagnostic the harness printed settled it: relay aborted the POST: True — the relay had closed the in-flight upstream POST exactly as #270 PR D specifies, and the reference peer simply had not turned that into a cancellation.

Measured latency from relay abort to the tool's cancellation arm firing:

Platform	Observed
macOS (local)	0.01 s
ubuntu-latest (CI)	14.96 s

14.96 s is _SSE_PING_INTERVAL (15.0 s) to within jitter, which identifies the mechanism: the server has paused reading while it owns an open SSE response, so the client's FIN is not observed until uvicorn next tries to write — and a tool that emits nothing after its last notification has no next write until the SSE keep-alive. Any client-side budget under one ping interval is unpassable on Linux no matter how healthy the relay is.

This is a property of the reference peer, not of the relay: the relay's behaviour is byte-identical on both platforms and provably correct on both. It matters well beyond this test — anything reasoning about "the server MUST treat a client disconnect as cancellation of that request" should assume up to a ping interval of latency on Linux, Phase 3's serve side included.

Scenario 6 therefore waits 40 s (two ping intervals) and measures and prints the latency rather than asserting on it (rule 4 forbids asserting on elapsed time, not recording it), with -s in CI so the healthy number reaches the log and a future drift shows up as a number instead of a sudden red. Its per-test pytest-timeout is raised to 90 s, and rule 3 in the conftest docstring is amended to permit a raised budget when the number is justified by a measured mechanism.

The other surprise

The v2 streamable-HTTP path defers http.response.start by up to 15 seconds. _SSE_PING_INTERVAL = 15.0: the response headers are not committed until the handler emits its first notification or that interval elapses. A tool that parks silently therefore leaves a client with no response object at all for 15 s — for this relay, PR D's documented pre-headers window, for a quarter of a minute.

This is not a scenario-6 implementation detail; it is a property of the reference peer worth recording for Phase 3, which will meet it from the other side. Scenario 6's tool now emits one progress notification before parking (the realistic long-running-tool shape), and the test gates its cancel on that notification coming back out of the relay — the only downstream-observable proof the headers landed. A server-side "handler entered" flag is not sufficient, because report_progress returns before those bytes reach the relay.

Three things hold that gate together, all verified: the relay merges into params._meta rather than replacing it (so a client progressToken survives), the 15 s deferral above, and report_progress reaching the sink.

Structure

tests_integration/          # SIBLING of tests/, never walked by `pytest tests/`
  conftest.py               # determinism rules verbatim + fixtures + mcp-missing guard
  _reference_server.py      # the python-sdk v2 MCPServer app (a factory)
  _stdio_client.py          # minimal 2025-06-18 stdio JSON-RPC client
  test_e2e.py               # the six scenarios
.github/workflows/integration.yml   # ubuntu-only, advisory

• Runtime dependencies unchanged: [project].dependencies is still httpx alone. mcp>=2.0,<3 + uvicorn>=0.30 live only in the new integration extra, and nothing under src/ imports them.
• Unit suite untouched in behaviour and runtime: 1,635 → 1,637 tests (the two revert-checks), same ~103 s. Verified that pytest tests_integration/ in a [dev]-only venv collects zero tests via the find_spec("mcp") guard, and that the timeout injection is filtered to its own directory so a bare root pytest cannot stamp markers onto the unit suite.
• Zero pytest configuration added. (pytest 9 prints configfile: pyproject.toml purely as its rootdir anchor — there is no [tool.pytest.ini_options] section, on this branch or on main.)
• Advisory by construction: main's required checks are dual-defined (branch protection + ruleset ci-gate-main) and list only the six unit-matrix contexts, so the new workflow is advisory simply by not being added to either. Promoting it later means editing both.
• The CI command chain was rehearsed locally, not just written: python3 -m venv → pip install -e ".[dev,integration]" → the metadata version smoke → pytest tests_integration/, in a throwaway venv, resolving mcp 2.0.0 / uvicorn 0.52.0 and leaving the repository tree clean.

For reviewers — where to look hardest

1. _reference_server.build_app()'s one private access, mcp._lowlevel_server.add_request_handler(...). This is what buys scenario 5 the strong form. add_request_handler and ListenHandler registration are both documented; only the attribute is private. An mcp 2.x minor that renames it fails the harness loudly — which the range pin treats as signal, on an advisory job.
2. Scenario 5's bounded quiescence check. It needed an extra drain (last commit): ListenHandler.close() drains buffered events before the final frame, so a publish from the establishment probe is legitimately forwarded by the graceful end. Reviewers should satisfy themselves the remaining negative assertion still means "no reconnect".
3. Scenario 6's dependence on progressToken surviving _meta injection. If the relay ever stops merging into params._meta, this test starts failing for a reason that has nothing to do with cancellation.
4. Fixture teardown order depends on parameter order. relay_factory must be finalized before the server fixture (the relay holds a long-lived subscriptions/listen POST). pytest guarantees this because every test lists dedicated_server/harness_server first, so the relay fixture is set up second and torn down first. The docstring says so; a future test that reverses the parameters would silently reintroduce nondeterministic teardown.
5. Scenario 6's 40 s wait is the one number to scrutinise. It is justified by a measured mechanism (15 s ping interval, 14.96 s observed) and not by tuning until green — the commit message and the in-test comment both carry the derivation, and the test prints the latency so the claim stays checkable on every run. If a reviewer disagrees with the mechanism, that is the place to push back.
6. Flake history, stated honestly. One failure was seen in 22 runs of an earlier revision and was never captured. Rather than assume the first fix had covered it, two real races were then found by re-reading ListenHandler and the modern streamable-HTTP path, and both were fixed: scenario 6's cancel landing in the pre-headers window, and scenario 5's unsound quiescence check (close() drains buffered events, so an establishment-probe publish is legitimately forwarded by the graceful end). The final code is 32/32 green over a clean local hunt and green on ubuntu in CI. Three CI rounds were spent on scenario 6 — red, then instrumented, then diagnosed — rather than papering over the failure; the middle commit is diagnostics only and is worth reading for what the harness now reports on failure.

Anti-scope honoured

No Phase 3 serve work; no relay changes beyond housekeeping (c)/(d); no restructuring of the unit suite or its mocks (their independence is the point); no Windows integration CI; no required-gate status; no mcp/uvicorn anywhere near [project].dependencies. The torture knobs (max_buffered_events=1, short requestState TTL, server SIGKILL) and scenario 6's three-windows probing remain for the follow-up issue, to be filed after a few weeks of green.

__version__ and CHANGELOG.md are untouched (release-please owns both).

🤖 Generated with Claude Code

https://claude.ai/code/session_01YVg3toYKK6J2J9ggdrzYVf

[github-actions]

AI review (gpt-5.6-sol)

Verdict: no blocking findings.

Scope: new commits since ca4608d (full PR diff supplied to the model as context).

No findings clear the reporting bar this round.

Findings ledger (all rounds)

• R1F1 [fixed] — Drain treats relay EOF as successful quiescence
• R1F2 [fixed] — Per-test timeout undercuts cumulative step deadlines

_{Advisory per-push review (round 5) generated by ai-review.yml — verify findings before acting.}

[shigechika]

Review fix applied (ca4608d): R1F1 — StdioClient.drain silently treated relay EOF as successful quiescence, so a relay that crashed mid-drain would make negative assertions (e.g. scenario 6's "no late response") pass vacuously. drain now raises RelayDied on EOF, same as _take. New harness-correctness test tests_integration/test_stdio_client.py::test_drain_raises_on_relay_eof kills the relay and asserts the raise; revert-checked (reverting the raise made it fail with "DID NOT RAISE RelayDied"). Unit suite 1637 passed/3 skipped, integration suite 7 passed, ruff clean at 0.16.0.

🤖 Generated with Claude Code

[shigechika]

Code review

Found 2 issues:

1. The housekeeping commit's CLAUDE.md truth-up introduces a new inaccuracy: "the legacy era keeps the literal synchronous stdin loop with zero threads" — false whenever OAuth is configured, because run() unconditionally starts the proactive-refresh daemon thread before any era branching (proactive refresh is on by default). The claim should be scoped to the era-gated threads (stdin reader / listen streams), not "zero threads". (bug due to CLAUDE.md edit vs _start_proactive_refresh's unconditional call site in run())

mcp-stdio/CLAUDE.md

Lines 47 to 51 in ca4608d

	- **`relay.py`** — Two transport implementations sharing stdin/stdout plumbing (file name kept for import compatibility):
	- `run()` — Streamable HTTP transport (MCP current spec, default). Reads JSON-RPC from stdin line-by-line, streams POST to the remote URL via httpx, parses JSON or SSE responses, writes to stdout. On the modern era (2026-07-28) a dedicated daemon reader thread owns stdin and feeds a FIFO queue, so a `notifications/cancelled` can abort the matching in-flight POST by closing its response stream; the legacy era keeps the literal synchronous stdin loop with zero threads. Handles retry with backoff (3 attempts), session ID tracking (`Mcp-Session-Id` header), 404-based session recovery, and 401-based token refresh.
	- `run_sse()` — SSE transport (MCP 2024-11-05 legacy). Spawns a daemon reader thread that maintains a long-lived `GET /sse` connection, parses `endpoint`/`message` events per the WHATWG SSE spec, and resolves the POST endpoint URL (possibly relative). The main thread reads stdin and POSTs to that endpoint. Auto-reconnects on stream disconnect, synthesizing a `-32000` error for every request whose reply was still in flight on the dropped stream (tracked in `_SseState.pending`, drained by `_drain_pending`; cancelled ids are skipped).
	- Both paths enforce the MCP cancellation spec's receiver-side SHOULD (and shield the client from canceller-side bugs) via a shared `_CancelTracker` + `_emit` gate: ids seen in `notifications/cancelled` on stdin are tracked with a 60 s TTL, and any late JSON-RPC response for a tracked id is dropped before it reaches stdout. Disable with `--no-cancel-filter`.

2. StdioClient.close()'s docstring promises "EOF, then terminate, then kill — all bounded", and the conftest's determinism rules require every teardown wait to be bounded — but the final proc.wait() after SIGKILL passes no timeout. A kernel-stuck relay process would hang teardown (and the whole suite) unboundedly, on the exact path that exists to guarantee the opposite. (bug due to tests_integration/_stdio_client.py close() — third wait() unbounded while the first two are bounded)

mcp-stdio/tests_integration/_stdio_client.py

Lines 281 to 302 in ca4608d

	"""Rule 6's relay half: EOF, then terminate, then kill — all bounded.
	stdin EOF is the relay's own documented shutdown path (`_STDIN_EOF`),
	so the graceful case exercises production code rather than a signal.
	"""
	try:
	if self.proc.stdin and not self.proc.stdin.closed:
	self.proc.stdin.close()
	except OSError: # pragma: no cover — already-dead relay
	pass
	try:
	self.proc.wait(timeout=timeout)
	except subprocess.TimeoutExpired:
	self.proc.terminate()
	try:
	self.proc.wait(timeout=2)
	except subprocess.TimeoutExpired: # pragma: no cover
	self.proc.kill()
	self.proc.wait()
	for stream in (self.proc.stdout, self.proc.stderr):
	try:
	if stream is not None:

Sub-threshold advisories (verified real, below the bar; for the #367 completion record): the integration.yml comment "Private repository: without this, checkout fails" is false — the repo is public and the permissions line is just least-privilege; scenario 5's stderr negative-assert slices a deque(maxlen=500) by a length snapshot, which goes vacuous after ring saturation (same class as R1F1 — content-snapshot diff would be robust); test_e2e.py's module docstring says every scenario pins a Phase 2 subsystem while scenario 1 pins Phase 1 era detection.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[shigechika]

/code-review fixes applied

Finding (score)	Resolution	Commit
CLAUDE.md "zero threads" overclaim (90)	Rescoped the run() bullet to the era-gated threads only; added the era-independent OAuth proactive-refresh/cold-start daemons as their own clause (verified: both start unconditionally, before the era branch). Extended the adjacent signal-handler bullet's thread list to match, so it doesn't repeat the same overclaim by omission. Wording-only, no test.	f61a941
StdioClient.close()'s post-SIGKILL wait unbounded (85)	Bounded the final proc.wait() at 5.0 (matching its sibling). Left TimeoutExpired to propagate on expiry rather than swallowing it — a process still alive after SIGKILL can't be escalated further, so surfacing loudly is the only sound response, matching this file's existing pattern (RelayDied from _take/drain). New unit-style test drives a stub Popen through the full EOF→terminate→kill escalation and asserts every wait() call got a bounded timeout; revert-checked.	09ff6ef

Both gates green: uvx ruff@0.16.0 check/format --check clean at 0.16.0; unit suite 1637 passed/3 skipped; integration suite 8 passed (6 scenarios + 2 harness-correctness tests).

🤖 Generated with Claude Code