fix(release): bound the registry-publish verification retry window - #139
Conversation
The v3.1.1 release published cleanly (PyPI, GitHub Release, and MCP Registry with isLatest=true all agree) but the Release workflow still reported failure. The "Verify the registry now serves the published version" step read the registry one second after publish (16:48:46 -> 16:48:47) and compared with zero tolerance; the registry's own CDN cache layer (modelcontextprotocol/registry docs/design/tech-architecture.md, "CDN Layer") had not yet converged. A gate that fails on a successful release is the kind that gets disarmed, which would remove the only external proof that `mcp-publisher publish` actually worked. Fix: bounded retry (12 attempts x 5s = 60s) before concluding the registry never converged. The bound is a stated engineering assumption (no published cache-TTL SLA exists), sized an order of magnitude above the 1s staleness actually observed on the v3.1.1 release. It still fails, and fails with the same message, when the registry genuinely never serves the expected version -- this is a timeout, not an unbounded retry-until-pass. Also distinguishes the workflow_dispatch recovery path's duplicate case from a real failure: "cannot publish duplicate version" from mcp-publisher means "already published" -- the desired end state when re-running publish-registry to repair a missed publish -- but only for workflow_dispatch, and only as a candidate the verify step above still has to independently confirm. On the normal push-tag path the same error is unexpected (this job runs once per tag, gated on needs: [test, release]) and still fails the job immediately. Any other publish error, on either path, still fails immediately without touching the retry window. Co-Authored-By: Claude <noreply@anthropic.com>
|
ZETETIC-REVIEW: APPROVE This repairs a gate that failed on a success, which is the direction nobody watches and the one that gets gates disarmed. The cause is identified, not guessed: the registry sits behind a CDN, per The bound is stated as an assumption rather than dressed as a measurement, which is the right call and the harder one. Twelve attempts at five seconds is sixty seconds, sized an order of magnitude above the only empirical figure available (staleness still present one second after publish); no cache-TTL SLA is published, and the PR says so instead of inventing a number and attributing it to a source. The two properties that had to survive, did. The loop still exits non-zero when the registry never converges — a timeout, not a retry-until-pass — and the publish step's own failure still fails the job immediately without waiting out the window. A retry that always passes is a deleted check wearing a loop, and this is not one. The duplicate case is distinguished rather than swallowed. Verified in the agent's own thread rather than deferred: the retry loop was run against the live registry, converging on attempt 1 for the real current version and correctly exhausting its window for a version chosen never to match. All five publish-step branches were simulated; only dispatch-plus-duplicate is swallowed. On the two red Merging. |
Same flaw as cortex-viz's Release.yaml (fixed same day, PR cdeust/cortex-viz#139): the "Verify the registry now serves the published version" step in publish_registry compares with zero tolerance immediately after mcp-publisher publish returns, and the registry's own CDN cache layer (modelcontextprotocol/registry docs/design/tech-architecture.md, "CDN Layer" — "Caches all public read endpoints") had not converged by then on cortex-viz's v3.1.1 release: publish completed 16:48:46 UTC, a read one second later (16:48:47) still returned the prior version. This job has the same publish -> immediate-read shape and no reason to be exempt. Fix: bounded retry (12 attempts x 5s = 60s) before concluding non-convergence. The bound is a stated engineering assumption (no published cache-TTL SLA exists), sized an order of magnitude above the 1s staleness measured on the cortex-viz incident. It still fails, with the same message, when the registry genuinely never serves the expected version. Also distinguishes the workflow_dispatch recovery path's duplicate case from a real failure: "cannot publish duplicate version" from mcp-publisher means "already published" -- the desired end state when re-running publish_registry to repair a missed publish (this job's own header documents exactly that gap: v0.8.0 released cleanly while the registry kept serving v0.7.0) -- but only for workflow_dispatch, and only as a candidate the verify step above still independently confirms. On the normal push-tag path the same error stays fatal (this job runs once per tag, gated on needs: [release]), and any other publish error on either path fails immediately without touching the retry window. Co-authored-by: Claude <noreply@anthropic.com>
ci.yml's "Validate official MCP Registry manifest" step downloaded mcp-publisher and POSTed server.json to the live registry.modelcontextprotocol.io/v0/validate endpoint, as a REQUIRED check. Measured on PR #139: both `test` jobs failed with `dial tcp 34.61.200.254:443: i/o timeout` while 1335 tests passed; re-running turned them green, and main shows four consecutive successes since. Not the diff -- a required check that fails whenever a third party is briefly unreachable, indefinitely. Read the registry's own source (modelcontextprotocol/registry, tag v1.8.1) to establish what /v0/validate actually checks before removing the network call, per the standing "verify upstream claims at the source" rule: internal/api/handlers/v0/validate.go:26 calls ValidateServerJSON(server, ValidationAll) -- full JSON Schema (draft-07) structural validation, PLUS Go-only semantic rules (name/version format, repository/website/icon URL rules, package argument and transport template-variable resolution) -- internal/validators/validators.go. The schema half is a pure function of server.json + a static schema document the registry binary embeds via `//go:embed schemas/*.json` (internal/validators/schema.go) rather than fetches per request -- reproducible offline, byte-for-byte, with any spec-conformant draft-07 validator, because both implementations run the identical specification over the identical input. The semantic half has no equivalent expressible in the JSON Schema document and is Go-only. It is not silently dropped from the release pipeline: internal/api/handlers/v0/publish.go:58 runs the SAME semantic checks -- ValidateServerJSON(server, ValidationSchemaVersionAndSemantic) -- authoritatively, over the network, at actual release time (Release.yaml's publish-registry job), and fails the release loudly if violated. Fix: - scripts/validate_server_manifest.py -- offline validator using `jsonschema`'s Draft7Validator against a vendored copy of the exact schema file the registry embeds (scripts/schemas/2025-12-11.json, scripts/schemas/README.md documents provenance + re-vendor procedure). Deliberately does NOT enable jsonschema's format_checker: the Go compiler in schema.go never sets `AssertFormat` (defaults false in santhosh-tekuri/jsonschema/v5), so `format: uri` is annotation-only there -- matching that, rather than being stricter than the real endpoint, avoids rejecting a server.json the registry would accept. Fails loudly (not silently-stale) if server.json's $schema ever moves to a version this repo hasn't re-vendored. - ci.yml: this offline check is now the REQUIRED step. A second, `continue-on-error: true` step still calls live `mcp-publisher validate` for the semantic-only checks the offline step cannot cover -- early warning in the PR checks list when the registry is reachable, never a build failure when it isn't. Uses the shared .github/actions/install-mcp-publisher composite action from the preceding PR (agent/ci-mcp-publisher-dedup) rather than a third pin. Verified: ruff check + format clean; YAML parses; actionlint clean on ci.yml. Adversarial gate, offline (HTTP(S)_PROXY pointed at an unreachable port so any network attempt fails instantly): a deliberately broken server.json (no `/` in name, missing required `version`) is rejected with both violations named; the real, unmodified server.json passes; both outcomes identical whether or not the network is reachable. Co-Authored-By: Claude <noreply@anthropic.com>
) * fix(ci): stop requiring a live registry call to validate server.json ci.yml's "Validate official MCP Registry manifest" step downloaded mcp-publisher and POSTed server.json to the live registry.modelcontextprotocol.io/v0/validate endpoint, as a REQUIRED check. Measured on PR #139: both `test` jobs failed with `dial tcp 34.61.200.254:443: i/o timeout` while 1335 tests passed; re-running turned them green, and main shows four consecutive successes since. Not the diff -- a required check that fails whenever a third party is briefly unreachable, indefinitely. Read the registry's own source (modelcontextprotocol/registry, tag v1.8.1) to establish what /v0/validate actually checks before removing the network call, per the standing "verify upstream claims at the source" rule: internal/api/handlers/v0/validate.go:26 calls ValidateServerJSON(server, ValidationAll) -- full JSON Schema (draft-07) structural validation, PLUS Go-only semantic rules (name/version format, repository/website/icon URL rules, package argument and transport template-variable resolution) -- internal/validators/validators.go. The schema half is a pure function of server.json + a static schema document the registry binary embeds via `//go:embed schemas/*.json` (internal/validators/schema.go) rather than fetches per request -- reproducible offline, byte-for-byte, with any spec-conformant draft-07 validator, because both implementations run the identical specification over the identical input. The semantic half has no equivalent expressible in the JSON Schema document and is Go-only. It is not silently dropped from the release pipeline: internal/api/handlers/v0/publish.go:58 runs the SAME semantic checks -- ValidateServerJSON(server, ValidationSchemaVersionAndSemantic) -- authoritatively, over the network, at actual release time (Release.yaml's publish-registry job), and fails the release loudly if violated. Fix: - scripts/validate_server_manifest.py -- offline validator using `jsonschema`'s Draft7Validator against a vendored copy of the exact schema file the registry embeds (scripts/schemas/2025-12-11.json, scripts/schemas/README.md documents provenance + re-vendor procedure). Deliberately does NOT enable jsonschema's format_checker: the Go compiler in schema.go never sets `AssertFormat` (defaults false in santhosh-tekuri/jsonschema/v5), so `format: uri` is annotation-only there -- matching that, rather than being stricter than the real endpoint, avoids rejecting a server.json the registry would accept. Fails loudly (not silently-stale) if server.json's $schema ever moves to a version this repo hasn't re-vendored. - ci.yml: this offline check is now the REQUIRED step. A second, `continue-on-error: true` step still calls live `mcp-publisher validate` for the semantic-only checks the offline step cannot cover -- early warning in the PR checks list when the registry is reachable, never a build failure when it isn't. Uses the shared .github/actions/install-mcp-publisher composite action from the preceding PR (agent/ci-mcp-publisher-dedup) rather than a third pin. Verified: ruff check + format clean; YAML parses; actionlint clean on ci.yml. Adversarial gate, offline (HTTP(S)_PROXY pointed at an unreachable port so any network attempt fails instantly): a deliberately broken server.json (no `/` in name, missing required `version`) is rejected with both violations named; the real, unmodified server.json passes; both outcomes identical whether or not the network is reachable. Co-Authored-By: Claude <noreply@anthropic.com> * fix(ci): re-vendor the MCP Registry schema from its canonical source, add automated drift check Answering the review question on this branch's schema vendoring: what prevents the vendored copy from drifting silently out of sync with its source? Documented provenance is the minimum, not a guarantee -- so this adds an automatable check rather than leaving it to discipline. While building it, the check found real drift. The vendored file was first pulled from modelcontextprotocol/registry's v1.8.1 git tag (internal/validators/schemas/2025-12-11.json), on the assumption that a dated schema version is immutable once published. It measurably was not: diffed against modelcontextprotocol/static (the schema's actual origin) and the CDN it publishes to (https://static.modelcontextprotocol.io/... -- the exact URL server.json's $schema field points to), the registry-tag copy carried an extra `maxLength: 255` constraint on Package.version that neither of the other two sources has. Root cause: the registry repo's own sync-schema.yml is workflow_dispatch-only ("TODO: Add daily schedule later"), so its embedded copy can lag the source it mirrors. Fix: - Re-vendored scripts/schemas/2025-12-11.json directly from the CDN (ground truth -- the URL server.json's $schema literally references), not the registry repo tag. Net diff: one stray constraint removed. - scripts/check_vendored_schema_drift.py -- fetches the vendored file's own $id (which IS the canonical URL) and compares structurally. Wired into ci.yml as a third non-blocking (`continue-on-error`) step, same trade as the live semantic pre-flight already in this job: signal when reachable, never a required-check failure when it isn't. The offline structural check earlier in the job remains the only gate. - scripts/schemas/README.md rewritten: CDN as the documented source of truth (not the registry repo tag), the drift that was found, and how to run the check standalone. Verified: ruff check + format clean on scripts/; YAML parses; actionlint clean. Drift check exercised three ways -- passes against the corrected vendored file, fails with a named diagnosis when the removed maxLength constraint is reintroduced, and fails loudly (nonzero exit, not a silent pass) when the network is unreachable. Offline structural-validation gate re-run against the corrected schema: unaffected (server.json's version string is far under either bound). Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
Summary
isLatest=trueall agree at 16:48:46), but thepublish-registryjob's last step, "Verify the registry now serves the published version", still reported failure: it read the registry one second later (16:48:47) with zero retry tolerance and hit the registry's own CDN cache layer before it converged (source:modelcontextprotocol/registrydocs/design/tech-architecture.md, "CDN Layer" — "Caches all public read endpoints").workflow_dispatchrecovery path:mcp-publisher publishreturningcannot publish duplicate versionmeans the registry already has the version — the desired end state for a recovery re-run — but that swallow is scoped toworkflow_dispatchonly and is still independently confirmed by the verify step. On the normal push-tag path the same error stays fatal (this job runs once per tag, gated onneeds: [test, release]), and any other publish error on either path fails immediately without touching the retry window.Test plan
python3 -c "import yaml; yaml.safe_load(...)"— YAML parses.actionlint .github/workflows/Release.yaml— clean, no findings.registry.modelcontextprotocol.io/v0/servers?search=hypermnesia-mcp-viz): converges on attempt 1 for the real current version (3.1.1), and correctly exhausts all attempts and fails for a version that will never match (99.99.99).workflow_dispatch(cannot run in this session — no container/full-suite execution per machine-load constraint; requester will watch CI).🤖 Generated with Claude Code