From 5fa991a254437a14865bc9f31a385d9b6beea6f2 Mon Sep 17 00:00:00 2001 From: cdeust Date: Mon, 10 Aug 2026 19:40:36 +0200 Subject: [PATCH 1/2] fix(ci): stop requiring a live registry call to validate server.json ci.yml's "Validate official MCP Registry manifest" step downloaded mcp-publisher and POSTed server.json to the live registry.modelcontextprotocol.io/v0/validate endpoint, as a REQUIRED check. Measured on PR #139: both `test` jobs failed with `dial tcp 34.61.200.254:443: i/o timeout` while 1335 tests passed; re-running turned them green, and main shows four consecutive successes since. Not the diff -- a required check that fails whenever a third party is briefly unreachable, indefinitely. Read the registry's own source (modelcontextprotocol/registry, tag v1.8.1) to establish what /v0/validate actually checks before removing the network call, per the standing "verify upstream claims at the source" rule: internal/api/handlers/v0/validate.go:26 calls ValidateServerJSON(server, ValidationAll) -- full JSON Schema (draft-07) structural validation, PLUS Go-only semantic rules (name/version format, repository/website/icon URL rules, package argument and transport template-variable resolution) -- internal/validators/validators.go. The schema half is a pure function of server.json + a static schema document the registry binary embeds via `//go:embed schemas/*.json` (internal/validators/schema.go) rather than fetches per request -- reproducible offline, byte-for-byte, with any spec-conformant draft-07 validator, because both implementations run the identical specification over the identical input. The semantic half has no equivalent expressible in the JSON Schema document and is Go-only. It is not silently dropped from the release pipeline: internal/api/handlers/v0/publish.go:58 runs the SAME semantic checks -- ValidateServerJSON(server, ValidationSchemaVersionAndSemantic) -- authoritatively, over the network, at actual release time (Release.yaml's publish-registry job), and fails the release loudly if violated. Fix: - scripts/validate_server_manifest.py -- offline validator using `jsonschema`'s Draft7Validator against a vendored copy of the exact schema file the registry embeds (scripts/schemas/2025-12-11.json, scripts/schemas/README.md documents provenance + re-vendor procedure). Deliberately does NOT enable jsonschema's format_checker: the Go compiler in schema.go never sets `AssertFormat` (defaults false in santhosh-tekuri/jsonschema/v5), so `format: uri` is annotation-only there -- matching that, rather than being stricter than the real endpoint, avoids rejecting a server.json the registry would accept. Fails loudly (not silently-stale) if server.json's $schema ever moves to a version this repo hasn't re-vendored. - ci.yml: this offline check is now the REQUIRED step. A second, `continue-on-error: true` step still calls live `mcp-publisher validate` for the semantic-only checks the offline step cannot cover -- early warning in the PR checks list when the registry is reachable, never a build failure when it isn't. Uses the shared .github/actions/install-mcp-publisher composite action from the preceding PR (agent/ci-mcp-publisher-dedup) rather than a third pin. Verified: ruff check + format clean; YAML parses; actionlint clean on ci.yml. Adversarial gate, offline (HTTP(S)_PROXY pointed at an unreachable port so any network attempt fails instantly): a deliberately broken server.json (no `/` in name, missing required `version`) is rejected with both violations named; the real, unmodified server.json passes; both outcomes identical whether or not the network is reachable. Co-Authored-By: Claude --- .github/workflows/ci.yml | 47 ++- pyproject.toml | 1 + scripts/schemas/2025-12-11.json | 575 ++++++++++++++++++++++++++++ scripts/schemas/README.md | 34 ++ scripts/validate_server_manifest.py | 130 +++++++ uv.lock | 2 + 6 files changed, 781 insertions(+), 8 deletions(-) create mode 100644 scripts/schemas/2025-12-11.json create mode 100644 scripts/schemas/README.md create mode 100644 scripts/validate_server_manifest.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9c085f4..6595513 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -115,18 +115,49 @@ jobs: run: uv build - name: Verify distribution metadata and compatibility entry points run: uv run --no-sync python -m scripts.check_distribution_artifact - # Was pinned independently of Release.yaml's copy at v1.7.9 (Release.yaml - # had already moved to v1.8.1 with a different checksum — two pins of - # the same binary, already drifted apart). Now installed by the shared - # composite action, whose defaults are the only place the version+ - # checksum pin lives — a future bump is one edit to that file, not two. - # v1.8.1 confirmed as the latest release via the registry's GitHub - # Releases API on 2026-08-10; both workflows are aligned to it. + # Was "download mcp-publisher and POST server.json to the live + # registry.modelcontextprotocol.io/v0/validate" (`mcp-publisher + # validate`), as a REQUIRED check. That made a required check depend + # on a third party being reachable from this runner with no retry + # budget: on PR #139 both `test` jobs failed with `dial tcp ...: i/o + # timeout` while 1335 tests passed, and a bare re-run went green. + # + # Reading the registry's own source (tag v1.8.1) shows /v0/validate + # runs `ValidateServerJSON(server, ValidationAll)`: full JSON Schema + # (draft-07) structural validation, against a schema the registry + # binary embeds at build time (`//go:embed schemas/*.json`) rather + # than fetches per-request -- so that half of the check is a pure + # function of server.json + a static schema document, reproducible + # offline with any spec-conformant draft-07 validator. This script + # does exactly that against a vendored copy of the same schema file + # (scripts/schemas/2025-12-11.json) and fails loudly if server.json + # ever moves to a schema version that hasn't been re-vendored. + # + # What is NOT replicated: ValidateServerJSON's Go-only semantic + # checks (version-is-not-a-range/"latest", per-source repository URL + # rules, https-only website/icon URLs, package argument and + # transport-templating rules) have no equivalent expressible in the + # JSON Schema document itself. Those are not silently dropped from + # the pipeline -- the registry's own /v0/publish handler runs the + # same semantic checks authoritatively, over the network, at release + # time (Release.yaml's publish-registry job), and fails the release + # loudly if violated. See scripts/validate_server_manifest.py's + # module docstring for the full source-backed breakdown. + - name: Validate MCP Registry manifest (offline schema check) + run: uv run --no-sync python -m scripts.validate_server_manifest + # Informational only (`continue-on-error`) — the Go-only semantic + # checks the offline step above cannot cover (see its comment). This + # is what used to be the required, flaky step; kept here non-blocking + # so a third-party outage never fails this job again, while a + # reachable registry still surfaces a real semantic problem in the PR + # checks list instead of only at release time. The offline step above + # is the gate; this is early warning, not a second gate. - name: Install mcp-publisher (checksum-verified) uses: ./.github/actions/install-mcp-publisher with: destination: /tmp - - name: Validate official MCP Registry manifest + - name: Validate MCP Registry manifest (live semantic pre-flight, non-blocking) + continue-on-error: true run: /tmp/mcp-publisher validate js-test: diff --git a/pyproject.toml b/pyproject.toml index 4c3584f..d817b48 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -105,6 +105,7 @@ dev = [ # nothing. Bumping it is a deliberate edit with the new findings fixed # in the same PR — the same contract dependabot already applies to npm. "ruff>=0.15.0,<0.17.0", + "jsonschema>=4.23,<5", ] [tool.hatch.build.targets.wheel] diff --git a/scripts/schemas/2025-12-11.json b/scripts/schemas/2025-12-11.json new file mode 100644 index 0000000..70902ed --- /dev/null +++ b/scripts/schemas/2025-12-11.json @@ -0,0 +1,575 @@ +{ + "$comment": "This file is auto-generated from docs/reference/api/openapi.yaml. Do not edit manually. Run 'make generate-schema' to update.", + "$id": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", + "$ref": "#/definitions/ServerDetail", + "$schema": "http://json-schema.org/draft-07/schema#", + "definitions": { + "Argument": { + "anyOf": [ + { + "$ref": "#/definitions/PositionalArgument" + }, + { + "$ref": "#/definitions/NamedArgument" + } + ], + "description": "Warning: Arguments construct command-line parameters that may contain user-provided input. This creates potential command injection risks if clients execute commands in a shell environment. For example, a malicious argument value like ';rm -rf ~/Development' could execute dangerous commands. Clients should prefer non-shell execution methods (e.g., posix_spawn) when possible to eliminate injection risks entirely. Where not possible, clients should obtain consent from users or agents to run the resolved command before execution." + }, + "Icon": { + "description": "An optionally-sized icon that can be displayed in a user interface.", + "properties": { + "mimeType": { + "description": "Optional MIME type override if the source MIME type is missing or generic. Must be one of: image/png, image/jpeg, image/jpg, image/svg+xml, image/webp.", + "enum": [ + "image/png", + "image/jpeg", + "image/jpg", + "image/svg+xml", + "image/webp" + ], + "example": "image/png", + "type": "string" + }, + "sizes": { + "description": "Optional array of strings that specify sizes at which the icon can be used. Each string should be in WxH format (e.g., '48x48', '96x96') or 'any' for scalable formats like SVG. If not provided, the client should assume that the icon can be used at any size.", + "examples": [ + [ + "48x48", + "96x96" + ], + [ + "any" + ] + ], + "items": { + "pattern": "^(\\d+x\\d+|any)$", + "type": "string" + }, + "type": "array" + }, + "src": { + "description": "A standard URI pointing to an icon resource. Must be an HTTPS URL. Consumers SHOULD take steps to ensure URLs serving icons are from the same domain as the server or a trusted domain. Consumers SHOULD take appropriate precautions when consuming SVGs as they can contain executable JavaScript.", + "example": "https://example.com/icon.png", + "format": "uri", + "maxLength": 255, + "type": "string" + }, + "theme": { + "description": "Optional specifier for the theme this icon is designed for. 'light' indicates the icon is designed to be used with a light background, and 'dark' indicates the icon is designed to be used with a dark background. If not provided, the client should assume the icon can be used with any theme.", + "enum": [ + "light", + "dark" + ], + "type": "string" + } + }, + "required": [ + "src" + ], + "type": "object" + }, + "Input": { + "properties": { + "choices": { + "description": "A list of possible values for the input. If provided, the user must select one of these values.", + "example": [], + "items": { + "type": "string" + }, + "type": "array" + }, + "default": { + "description": "The default value for the input. This should be a valid value for the input. If you want to provide input examples or guidance, use the `placeholder` field instead.", + "type": "string" + }, + "description": { + "description": "A description of the input, which clients can use to provide context to the user.", + "type": "string" + }, + "format": { + "default": "string", + "description": "Specifies the input format. Supported values include `filepath`, which should be interpreted as a file on the user's filesystem.\n\nWhen the input is converted to a string, booleans should be represented by the strings \"true\" and \"false\", and numbers should be represented as decimal values.", + "enum": [ + "string", + "number", + "boolean", + "filepath" + ], + "type": "string" + }, + "isRequired": { + "default": false, + "type": "boolean" + }, + "isSecret": { + "default": false, + "description": "Indicates whether the input is a secret value (e.g., password, token). If true, clients should handle the value securely.", + "type": "boolean" + }, + "placeholder": { + "description": "A placeholder for the input to be displaying during configuration. This is used to provide examples or guidance about the expected form or content of the input.", + "type": "string" + }, + "value": { + "description": "The value for the input. If this is not set, the user may be prompted to provide a value. If a value is set, it should not be configurable by end users.\n\nIdentifiers wrapped in `{curly_braces}` will be replaced with the corresponding properties from the input `variables` map. If an identifier in braces is not found in `variables`, or if `variables` is not provided, the `{curly_braces}` substring should remain unchanged.\n", + "type": "string" + } + }, + "type": "object" + }, + "InputWithVariables": { + "allOf": [ + { + "$ref": "#/definitions/Input" + }, + { + "properties": { + "variables": { + "additionalProperties": { + "$ref": "#/definitions/Input" + }, + "description": "A map of variable names to their values. Keys in the input `value` that are wrapped in `{curly_braces}` will be replaced with the corresponding variable values.", + "type": "object" + } + }, + "type": "object" + } + ] + }, + "KeyValueInput": { + "allOf": [ + { + "$ref": "#/definitions/InputWithVariables" + }, + { + "properties": { + "name": { + "description": "Name of the header or environment variable.", + "example": "SOME_VARIABLE", + "type": "string" + } + }, + "required": [ + "name" + ], + "type": "object" + } + ] + }, + "LocalTransport": { + "anyOf": [ + { + "$ref": "#/definitions/StdioTransport" + }, + { + "$ref": "#/definitions/StreamableHttpTransport" + }, + { + "$ref": "#/definitions/SseTransport" + } + ], + "description": "Transport protocol configuration for local/package context" + }, + "NamedArgument": { + "allOf": [ + { + "$ref": "#/definitions/InputWithVariables" + }, + { + "properties": { + "isRepeated": { + "default": false, + "description": "Whether the argument can be repeated multiple times.", + "type": "boolean" + }, + "name": { + "description": "The flag name, including any leading dashes.", + "example": "--port", + "type": "string" + }, + "type": { + "enum": [ + "named" + ], + "example": "named", + "type": "string" + } + }, + "required": [ + "type", + "name" + ], + "type": "object" + } + ], + "description": "A command-line `--flag={value}`." + }, + "Package": { + "properties": { + "environmentVariables": { + "description": "A mapping of environment variables to be set when running the package.", + "items": { + "$ref": "#/definitions/KeyValueInput" + }, + "type": "array" + }, + "fileSha256": { + "description": "SHA-256 hash of the package file for integrity verification. Required for MCPB packages and optional for other package types. Authors are responsible for generating correct SHA-256 hashes when creating server.json. If present, MCP clients must validate the downloaded file matches the hash before running packages to ensure file integrity.", + "example": "fe333e598595000ae021bd27117db32ec69af6987f507ba7a63c90638ff633ce", + "pattern": "^[a-f0-9]{64}$", + "type": "string" + }, + "identifier": { + "description": "Package identifier - either a package name (for registries) or URL (for direct downloads)", + "examples": [ + "@modelcontextprotocol/server-brave-search", + "https://github.com/example/releases/download/v1.0.0/package.mcpb" + ], + "type": "string" + }, + "packageArguments": { + "description": "A list of arguments to be passed to the package's binary.", + "items": { + "$ref": "#/definitions/Argument" + }, + "type": "array" + }, + "registryBaseUrl": { + "description": "Base URL of the package registry", + "examples": [ + "https://registry.npmjs.org", + "https://pypi.org", + "https://docker.io", + "https://api.nuget.org/v3/index.json", + "https://github.com", + "https://gitlab.com" + ], + "format": "uri", + "type": "string" + }, + "registryType": { + "description": "Registry type indicating how to download packages (e.g., 'npm', 'pypi', 'oci', 'nuget', 'mcpb')", + "examples": [ + "npm", + "pypi", + "oci", + "nuget", + "mcpb" + ], + "type": "string" + }, + "runtimeArguments": { + "description": "A list of arguments to be passed to the package's runtime command (such as docker or npx). The `runtimeHint` field should be provided when `runtimeArguments` are present.", + "items": { + "$ref": "#/definitions/Argument" + }, + "type": "array" + }, + "runtimeHint": { + "description": "A hint to help clients determine the appropriate runtime for the package. This field should be provided when `runtimeArguments` are present.", + "examples": [ + "npx", + "uvx", + "docker", + "dnx" + ], + "type": "string" + }, + "transport": { + "$ref": "#/definitions/LocalTransport", + "description": "Transport protocol configuration for the package" + }, + "version": { + "description": "Package version. Must be a specific version. Version ranges are rejected (e.g., '^1.2.3', '~1.2.3', '\u003e=1.2.3', '1.x', '1.*').", + "example": "1.0.2", + "minLength": 1, + "maxLength": 255, + "not": { + "const": "latest" + }, + "type": "string" + } + }, + "required": [ + "registryType", + "identifier", + "transport" + ], + "type": "object" + }, + "PositionalArgument": { + "allOf": [ + { + "$ref": "#/definitions/InputWithVariables" + }, + { + "anyOf": [ + { + "required": [ + "valueHint" + ] + }, + { + "required": [ + "value" + ] + } + ], + "properties": { + "isRepeated": { + "default": false, + "description": "Whether the argument can be repeated multiple times in the command line.", + "type": "boolean" + }, + "type": { + "enum": [ + "positional" + ], + "example": "positional", + "type": "string" + }, + "valueHint": { + "description": "An identifier for the positional argument. It is not part of the command line. It may be used by client configuration as a label identifying the argument. It is also used to identify the value in transport URL variable substitution.", + "example": "file_path", + "type": "string" + } + }, + "required": [ + "type" + ], + "type": "object" + } + ], + "description": "A positional input is a value inserted verbatim into the command line." + }, + "RemoteTransport": { + "allOf": [ + { + "anyOf": [ + { + "$ref": "#/definitions/StreamableHttpTransport" + }, + { + "$ref": "#/definitions/SseTransport" + } + ] + }, + { + "properties": { + "variables": { + "additionalProperties": { + "$ref": "#/definitions/Input" + }, + "description": "Configuration variables that can be referenced in URL template {curly_braces}. The key is the variable name, and the value defines the variable properties.", + "type": "object" + } + }, + "type": "object" + } + ], + "description": "Transport protocol configuration for remote context - extends StreamableHttpTransport or SseTransport with variables" + }, + "Repository": { + "description": "Repository metadata for the MCP server source code. Enables users and security experts to inspect the code, improving transparency.", + "properties": { + "id": { + "description": "Repository identifier from the hosting service (e.g., GitHub repo ID). Owned and determined by the source forge. Should remain stable across repository renames and may be used to detect repository resurrection attacks - if a repository is deleted and recreated, the ID should change. For GitHub, use: gh api repos/\u003cowner\u003e/\u003crepo\u003e --jq '.id'", + "example": "b94b5f7e-c7c6-d760-2c78-a5e9b8a5b8c9", + "type": "string" + }, + "source": { + "description": "Repository hosting service identifier. Used by registries to determine validation and API access methods.", + "example": "github", + "type": "string" + }, + "subfolder": { + "description": "Optional relative path from repository root to the server location within a monorepo or nested package structure. Must be a clean relative path.", + "example": "src/everything", + "type": "string" + }, + "url": { + "description": "Repository URL for browsing source code. Should support both web browsing and git clone operations.", + "example": "https://github.com/modelcontextprotocol/servers", + "format": "uri", + "type": "string" + } + }, + "required": [ + "url", + "source" + ], + "type": "object" + }, + "ServerDetail": { + "description": "Schema for a static representation of an MCP server. Used in various contexts related to discovery, installation, and configuration.", + "properties": { + "$schema": { + "description": "JSON Schema URI for this server.json format", + "example": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json", + "format": "uri", + "type": "string" + }, + "_meta": { + "description": "Extension metadata using reverse DNS namespacing for vendor-specific data", + "properties": { + "io.modelcontextprotocol.registry/publisher-provided": { + "additionalProperties": true, + "description": "Publisher-provided metadata for downstream registries", + "example": { + "buildInfo": { + "commit": "abc123def456", + "pipelineId": "build-789", + "timestamp": "2023-12-01T10:30:00Z" + }, + "tool": "publisher-cli", + "version": "1.2.3" + }, + "type": "object" + } + }, + "type": "object" + }, + "description": { + "description": "Clear human-readable explanation of server functionality. Should focus on capabilities, not implementation details.", + "example": "MCP server providing weather data and forecasts via OpenWeatherMap API", + "maxLength": 100, + "minLength": 1, + "type": "string" + }, + "icons": { + "description": "Optional set of sized icons that the client can display in a user interface. Clients that support rendering icons MUST support at least the following MIME types: image/png and image/jpeg (safe, universal compatibility). Clients SHOULD also support: image/svg+xml (scalable but requires security precautions) and image/webp (modern, efficient format).", + "items": { + "$ref": "#/definitions/Icon" + }, + "type": "array" + }, + "name": { + "description": "Server name in reverse-DNS format. Must contain exactly one forward slash separating namespace from server name.", + "example": "io.github.user/weather", + "maxLength": 200, + "minLength": 3, + "pattern": "^[a-zA-Z0-9.-]+/[a-zA-Z0-9._-]+$", + "type": "string" + }, + "packages": { + "items": { + "$ref": "#/definitions/Package" + }, + "type": "array" + }, + "remotes": { + "items": { + "$ref": "#/definitions/RemoteTransport" + }, + "type": "array" + }, + "repository": { + "$ref": "#/definitions/Repository", + "description": "Optional repository metadata for the MCP server source code. Recommended for transparency and security inspection." + }, + "title": { + "description": "Optional human-readable title or display name for the MCP server. MCP subregistries or clients MAY choose to use this for display purposes.", + "example": "Weather API", + "maxLength": 100, + "minLength": 1, + "type": "string" + }, + "version": { + "description": "Version string for this server. SHOULD follow semantic versioning (e.g., '1.0.2', '2.1.0-alpha'). Equivalent of Implementation.version in MCP specification. Non-semantic versions are allowed but may not sort predictably. Version ranges are rejected (e.g., '^1.2.3', '~1.2.3', '\u003e=1.2.3', '1.x', '1.*').", + "example": "1.0.2", + "maxLength": 255, + "type": "string" + }, + "websiteUrl": { + "description": "Optional URL to the server's homepage, documentation, or project website. This provides a central link for users to learn more about the server. Particularly useful when the server has custom installation instructions or setup requirements.", + "example": "https://modelcontextprotocol.io/examples", + "format": "uri", + "type": "string" + } + }, + "required": [ + "name", + "description", + "version" + ], + "type": "object" + }, + "SseTransport": { + "properties": { + "headers": { + "description": "HTTP headers to include", + "items": { + "$ref": "#/definitions/KeyValueInput" + }, + "type": "array" + }, + "type": { + "description": "Transport type", + "enum": [ + "sse" + ], + "example": "sse", + "type": "string" + }, + "url": { + "description": "Server-Sent Events endpoint URL template. Variables in {curly_braces} are resolved based on context: In Package context, they reference argument valueHints, argument names, or environment variable names from the parent Package. In Remote context, they reference variables from the transport's 'variables' object. After variable substitution, this should produce a valid URI.", + "example": "https://mcp-fs.example.com/sse", + "pattern": "^https?://[^\\s]+$", + "type": "string" + } + }, + "required": [ + "type", + "url" + ], + "type": "object" + }, + "StdioTransport": { + "properties": { + "type": { + "description": "Transport type", + "enum": [ + "stdio" + ], + "example": "stdio", + "type": "string" + } + }, + "required": [ + "type" + ], + "type": "object" + }, + "StreamableHttpTransport": { + "properties": { + "headers": { + "description": "HTTP headers to include", + "items": { + "$ref": "#/definitions/KeyValueInput" + }, + "type": "array" + }, + "type": { + "description": "Transport type", + "enum": [ + "streamable-http" + ], + "example": "streamable-http", + "type": "string" + }, + "url": { + "description": "URL template for the streamable-http transport. Variables in {curly_braces} are resolved based on context: In Package context, they reference argument valueHints, argument names, or environment variable names from the parent Package. In Remote context, they reference variables from the transport's 'variables' object. After variable substitution, this should produce a valid URI.", + "example": "https://api.example.com/mcp", + "pattern": "^https?://[^\\s]+$", + "type": "string" + } + }, + "required": [ + "type", + "url" + ], + "type": "object" + } + }, + "title": "server.json defining a Model Context Protocol (MCP) server" +} diff --git a/scripts/schemas/README.md b/scripts/schemas/README.md new file mode 100644 index 0000000..d730e3b --- /dev/null +++ b/scripts/schemas/README.md @@ -0,0 +1,34 @@ +# Vendored MCP Registry server schema + +`2025-12-11.json` is a byte-identical copy of +`internal/validators/schemas/2025-12-11.json` from +[`modelcontextprotocol/registry`](https://github.com/modelcontextprotocol/registry) +at tag `v1.8.1` (the same file the registry's Go binary embeds via +`//go:embed schemas/*.json` and serves, unmodified, as +`https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json`). +Fetched: + +``` +curl -fsSL https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/internal/validators/schemas/2025-12-11.json +``` + +It exists so `scripts/validate_server_manifest.py` can run JSON Schema +structural validation on `server.json` **offline** — see that script's +module docstring for what this replicates (and does not replicate) from +the registry's live `/v0/validate` endpoint. + +## When `server.json`'s `$schema` field changes + +`validate_server_manifest.py` fails loudly, by design, if the vendored +file's `$id` no longer matches `server.json`'s `$schema` value — it will +not silently validate against a stale schema version. To fix that failure, +re-vendor the new version: + +``` +NEW_VERSION= +curl -fsSL "https://raw.githubusercontent.com/modelcontextprotocol/registry//internal/validators/schemas/${NEW_VERSION}.json" \ + -o "scripts/schemas/${NEW_VERSION}.json" +rm scripts/schemas/2025-12-11.json # or keep both if migrating gradually +``` + +Then update `SCHEMA_FILENAME` in `scripts/validate_server_manifest.py`. diff --git a/scripts/validate_server_manifest.py b/scripts/validate_server_manifest.py new file mode 100644 index 0000000..13dae81 --- /dev/null +++ b/scripts/validate_server_manifest.py @@ -0,0 +1,130 @@ +#!/usr/bin/env python3 +"""Validate server.json offline, against the vendored MCP Registry schema. + +Replaces the CI step that downloaded `mcp-publisher` and ran +`mcp-publisher validate`, which POSTs server.json to the live +`https://registry.modelcontextprotocol.io/v0/validate` endpoint. That +endpoint is a required check with no retry budget of its own; a brief +outage or network blip on the runner fails CI on a diff that never touched +server.json (measured on cortex-viz PR #139: both `test` jobs failed with +`dial tcp ...: i/o timeout` while 1335 tests passed; re-running turned them +green). This script removes that dependency for the part of the check that +can be replicated exactly offline. + +Scope, established by reading the registry's own source +(github.com/modelcontextprotocol/registry, tag v1.8.1): + + * `/v0/validate` (and `mcp-publisher validate`) runs + `validators.ValidateServerJSON(server, ValidationAll)` -- + `internal/api/handlers/v0/validate.go`. `ValidationAll` is full JSON + Schema (draft-07) structural validation PLUS a set of Go-only semantic + checks -- `internal/validators/validation_types.go`. + * The schema itself is compiled into the registry binary via + `//go:embed schemas/*.json` (`internal/validators/schema.go`) and is + NOT fetched from the network at request time -- it is a static + document, published unmodified at + `https://static.modelcontextprotocol.io/schemas//server.schema.json`. + That makes the schema half of the check reproducible offline: the same + schema document, run through any spec-conformant draft-07 validator, + produces identical results to the Go implementation + (`santhosh-tekuri/jsonschema/v5`), because both are generic + implementations of the same specification over the same input. + * The Go compiler in schema.go is constructed with + `jsonschema.NewCompiler()` and never sets `AssertFormat`, which + defaults to `false` in that library -- so `format: "uri"` etc. are + annotations only, not asserted. This script matches that: it does NOT + pass a `format_checker` to `jsonschema`, to avoid being *stricter* + than the real endpoint and rejecting a server.json the registry would + accept. + +What this script does NOT replicate -- the Go-only semantic checks, which +have no equivalent expressible in the JSON Schema document itself (source: +`internal/validators/validators.go`): + * version is not "latest" and does not look like a semver range/wildcard + * repository URL validity is checked per declared `source` (github, etc.) + * `websiteUrl` / icon `src` must be absolute https URLs with no raw + control/quote characters + * `title` must not be whitespace-only + * package `identifier` must contain no spaces; argument name/value rules + * package and remote transport URLs are resolved against declared + template variables (env vars, runtime/package arguments) and validated + +These are not silently dropped from the project's release pipeline: the +registry's `/v0/publish` handler runs the SAME semantic checks -- +`validators.ValidateServerJSON(server, ValidationSchemaVersionAndSemantic)` +-- authoritatively, over the network, at release time +(`internal/api/handlers/v0/publish.go:58`), and Release.yaml's +`publish-registry` job fails the release loudly if they are violated. This +script narrows the CI gate to what CI should assert before a release +exists (RFC-free structural conformance); the semantic gate stays where it +already was authoritative. +""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path + +from jsonschema import Draft7Validator +from jsonschema.validators import validator_for + +ROOT = Path(__file__).resolve().parents[1] +SERVER_JSON = ROOT / "server.json" +# Bump this alongside re-vendoring — see scripts/schemas/README.md. +SCHEMA_FILE = ROOT / "scripts" / "schemas" / "2025-12-11.json" + + +def require(condition: bool, message: str) -> None: + """Raise even when Python assertions are disabled.""" + if not condition: + raise ValueError(message) + + +def load_schema() -> dict: + schema = json.loads(SCHEMA_FILE.read_text()) + require("$id" in schema, f"{SCHEMA_FILE} is missing required '$id' field") + require( + schema.get("$schema") == "http://json-schema.org/draft-07/schema#", + f"{SCHEMA_FILE} is not a draft-07 schema (upstream changed dialect; " + "re-check whether jsonschema.Draft7Validator still applies)", + ) + return schema + + +def main() -> None: + server = json.loads(SERVER_JSON.read_text()) + schema = load_schema() + + require( + server.get("$schema") == schema["$id"], + f"server.json's $schema ({server.get('$schema')!r}) does not match the " + f"vendored schema's $id ({schema['$id']!r}). server.json has moved to a " + "schema version this repo has not vendored yet — re-vendor per " + "scripts/schemas/README.md before trusting this check.", + ) + + require( + validator_for(schema) is Draft7Validator, + "vendored schema no longer resolves to Draft7Validator; the Go side " + "may have changed dialect — re-verify before updating this script", + ) + + validator = Draft7Validator(schema) + errors = sorted(validator.iter_errors(server), key=lambda e: list(e.path)) + if errors: + print(f"server.json FAILED schema validation with {len(errors)} issue(s):") + for i, error in enumerate(errors, 1): + path = "$" + "".join(f"[{p!r}]" for p in error.path) if error.path else "$" + print(f"{i}. {path}: {error.message}") + raise SystemExit(1) + + print(f"server.json is schema-valid against {schema['$id']}") + + +if __name__ == "__main__": + try: + main() + except (AssertionError, KeyError, OSError, ValueError) as exc: + print(f"server.json manifest validation FAILED: {exc}", file=sys.stderr) + raise SystemExit(1) from exc diff --git a/uv.lock b/uv.lock index ccb2285..3ab2885 100644 --- a/uv.lock +++ b/uv.lock @@ -794,6 +794,7 @@ data = [ { name = "psycopg-pool" }, ] dev = [ + { name = "jsonschema" }, { name = "mutmut" }, { name = "pytest" }, { name = "pytest-asyncio" }, @@ -818,6 +819,7 @@ requires-dist = [ { name = "fastmcp", specifier = ">=2.0.0" }, { name = "igraph", marker = "extra == 'community'", specifier = ">=0.11" }, { name = "igraph", marker = "extra == 'viz-tile'", specifier = ">=0.11" }, + { name = "jsonschema", marker = "extra == 'dev'", specifier = ">=4.23,<5" }, { name = "leidenalg", marker = "extra == 'community'", specifier = ">=0.10" }, { name = "mutmut", marker = "extra == 'dev'", specifier = ">=3.6.0" }, { name = "numpy", specifier = ">=1.24.0" }, From 85894d3374a1a8e88aa8d06a5b777d044f870e68 Mon Sep 17 00:00:00 2001 From: cdeust Date: Mon, 10 Aug 2026 19:54:47 +0200 Subject: [PATCH 2/2] fix(ci): re-vendor the MCP Registry schema from its canonical source, add automated drift check Answering the review question on this branch's schema vendoring: what prevents the vendored copy from drifting silently out of sync with its source? Documented provenance is the minimum, not a guarantee -- so this adds an automatable check rather than leaving it to discipline. While building it, the check found real drift. The vendored file was first pulled from modelcontextprotocol/registry's v1.8.1 git tag (internal/validators/schemas/2025-12-11.json), on the assumption that a dated schema version is immutable once published. It measurably was not: diffed against modelcontextprotocol/static (the schema's actual origin) and the CDN it publishes to (https://static.modelcontextprotocol.io/... -- the exact URL server.json's $schema field points to), the registry-tag copy carried an extra `maxLength: 255` constraint on Package.version that neither of the other two sources has. Root cause: the registry repo's own sync-schema.yml is workflow_dispatch-only ("TODO: Add daily schedule later"), so its embedded copy can lag the source it mirrors. Fix: - Re-vendored scripts/schemas/2025-12-11.json directly from the CDN (ground truth -- the URL server.json's $schema literally references), not the registry repo tag. Net diff: one stray constraint removed. - scripts/check_vendored_schema_drift.py -- fetches the vendored file's own $id (which IS the canonical URL) and compares structurally. Wired into ci.yml as a third non-blocking (`continue-on-error`) step, same trade as the live semantic pre-flight already in this job: signal when reachable, never a required-check failure when it isn't. The offline structural check earlier in the job remains the only gate. - scripts/schemas/README.md rewritten: CDN as the documented source of truth (not the registry repo tag), the drift that was found, and how to run the check standalone. Verified: ruff check + format clean on scripts/; YAML parses; actionlint clean. Drift check exercised three ways -- passes against the corrected vendored file, fails with a named diagnosis when the removed maxLength constraint is reintroduced, and fails loudly (nonzero exit, not a silent pass) when the network is unreachable. Offline structural-validation gate re-run against the corrected schema: unaffected (server.json's version string is far under either bound). Co-Authored-By: Claude --- .github/workflows/ci.yml | 10 ++++ scripts/check_vendored_schema_drift.py | 73 ++++++++++++++++++++++++++ scripts/schemas/2025-12-11.json | 1 - scripts/schemas/README.md | 59 +++++++++++++++------ 4 files changed, 127 insertions(+), 16 deletions(-) create mode 100644 scripts/check_vendored_schema_drift.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6595513..70de5cd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -159,6 +159,16 @@ jobs: - name: Validate MCP Registry manifest (live semantic pre-flight, non-blocking) continue-on-error: true run: /tmp/mcp-publisher validate + # A copied file drifts silently -- documenting its provenance + # (scripts/schemas/README.md) is the minimum, not a guarantee. + # scripts/schemas/2025-12-11.json's own $id IS the canonical URL, so + # this fetches it and compares structurally. Non-blocking for the + # same reason as the step above: useful signal when the network is + # reachable, never a required-check failure when it isn't -- the + # offline schema check earlier in this job is the only gate. + - name: Check vendored MCP Registry schema for drift (non-blocking) + continue-on-error: true + run: uv run --no-sync python -m scripts.check_vendored_schema_drift js-test: # Required job: the browser UI (ui/, ~25.5k lines) is the product's primary diff --git a/scripts/check_vendored_schema_drift.py b/scripts/check_vendored_schema_drift.py new file mode 100644 index 0000000..db91a5f --- /dev/null +++ b/scripts/check_vendored_schema_drift.py @@ -0,0 +1,73 @@ +#!/usr/bin/env python3 +"""Detect drift between the vendored MCP Registry schema and its live source. + +A copied file drifts silently -- provenance in scripts/schemas/README.md is +the minimum, not a guarantee. This is the automatable check for it: +scripts/schemas/2025-12-11.json's own `$id` field IS the canonical URL +(https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json +-- the same one server.json's `$schema` field points to, the same one a +spec-conformant client would fetch). Fetch it and compare, structurally +(parsed JSON equality, not byte-for-byte, so whitespace/key-order changes +upstream don't produce false alarms) against the vendored copy. + +Non-blocking by design (`continue-on-error` at the call site in ci.yml): +this makes the same trade the live semantic pre-flight step makes -- +useful signal when the network is reachable, never a required-check +failure when it briefly isn't. Structural validation +(scripts/validate_server_manifest.py) is the actual gate and never depends +on this succeeding. + +Real drift was found while building this check (2026-08-10, manual curl + +diff against the three candidate sources -- registry repo tag v1.8.1, +modelcontextprotocol/static main, and the CDN): the vendored copy, first +sourced from the registry repo's v1.8.1 git tag, carried a `maxLength: +255` constraint on `Package.version` that the CDN's currently-served +document does not have. The registry repo's own `sync-schema.yml` is +`workflow_dispatch`-only ("TODO: Add daily schedule later"), so its +embedded copy can silently lag the canonical modelcontextprotocol/static +source it is meant to mirror. The vendored file here was re-pulled from +the CDN itself (the ground truth per the paragraph above, and the URL +this script fetches), not from the registry repo tag, to fix it. +""" + +from __future__ import annotations + +import json +import sys +import urllib.error +import urllib.request +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +SCHEMA_FILE = ROOT / "scripts" / "schemas" / "2025-12-11.json" +TIMEOUT_SECONDS = 15 + + +def main() -> None: + vendored = json.loads(SCHEMA_FILE.read_text()) + source_url = vendored["$id"] + + try: + with urllib.request.urlopen(source_url, timeout=TIMEOUT_SECONDS) as resp: # noqa: S310 + live = json.loads(resp.read()) + except (urllib.error.URLError, TimeoutError) as exc: + # Non-blocking by design (see module docstring) -- the caller sets + # continue-on-error, but exit nonzero anyway so the step is visibly + # red rather than silently green on an unreachable network. + print(f"could not reach {source_url}: {exc}", file=sys.stderr) + raise SystemExit(1) from exc + + if vendored == live: + print(f"{SCHEMA_FILE} matches {source_url}") + return + + print( + f"{SCHEMA_FILE} has DRIFTED from {source_url} — re-vendor per " + "scripts/schemas/README.md.", + file=sys.stderr, + ) + raise SystemExit(1) + + +if __name__ == "__main__": + main() diff --git a/scripts/schemas/2025-12-11.json b/scripts/schemas/2025-12-11.json index 70902ed..9f6469b 100644 --- a/scripts/schemas/2025-12-11.json +++ b/scripts/schemas/2025-12-11.json @@ -283,7 +283,6 @@ "description": "Package version. Must be a specific version. Version ranges are rejected (e.g., '^1.2.3', '~1.2.3', '\u003e=1.2.3', '1.x', '1.*').", "example": "1.0.2", "minLength": 1, - "maxLength": 255, "not": { "const": "latest" }, diff --git a/scripts/schemas/README.md b/scripts/schemas/README.md index d730e3b..08fc5d8 100644 --- a/scripts/schemas/README.md +++ b/scripts/schemas/README.md @@ -1,34 +1,63 @@ # Vendored MCP Registry server schema -`2025-12-11.json` is a byte-identical copy of -`internal/validators/schemas/2025-12-11.json` from -[`modelcontextprotocol/registry`](https://github.com/modelcontextprotocol/registry) -at tag `v1.8.1` (the same file the registry's Go binary embeds via -`//go:embed schemas/*.json` and serves, unmodified, as -`https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json`). +`2025-12-11.json` is a copy of +`https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json` +— the exact URL in this repo's `server.json`'s `$schema` field, and the +canonical document a spec-conformant client fetches. It exists so +`scripts/validate_server_manifest.py` can run JSON Schema structural +validation on `server.json` **offline** — see that script's module +docstring for what this replicates (and does not replicate) from the +registry's live `/v0/validate` endpoint. + Fetched: ``` -curl -fsSL https://raw.githubusercontent.com/modelcontextprotocol/registry/v1.8.1/internal/validators/schemas/2025-12-11.json +curl -fsSL https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json ``` -It exists so `scripts/validate_server_manifest.py` can run JSON Schema -structural validation on `server.json` **offline** — see that script's -module docstring for what this replicates (and does not replicate) from -the registry's live `/v0/validate` endpoint. +## A copied file drifts silently — this one is checked automatically + +Provenance is the minimum, not a guarantee. `scripts/check_vendored_schema_drift.py` +reads the vendored file's own `$id` (which *is* the canonical URL above), +fetches it, and compares structurally. It runs as a non-blocking step in +`ci.yml` on every `test` job (never gates a PR — see that step's comment +for why), and can be run standalone: + +``` +uv run --no-sync python -m scripts.check_vendored_schema_drift +``` + +**This already caught real drift once.** The file was first vendored from +`modelcontextprotocol/registry`'s `v1.8.1` git tag +(`internal/validators/schemas/2025-12-11.json`), on the assumption that a +dated schema version is immutable once published. It measurably was not: +that copy carried a `maxLength: 255` constraint on `Package.version` that +the CDN's currently-served document does not have. The registry repo's own +schema-sync workflow (`sync-schema.yml`, upstream) is +`workflow_dispatch`-only — no schedule — so its embedded copy can lag the +canonical `modelcontextprotocol/static` source it mirrors. Re-vendoring +directly from the CDN (rather than the registry repo tag) removed the +stray constraint; see git blame on this line for the commit. + +**Trust the CDN as the source, not a registry-repo git tag.** The registry +repo is a secondary distribution of the schema (embedded into its Go +binary at build time), not the schema's origin — `modelcontextprotocol/static` +is, and the CDN is `static`'s public face. ## When `server.json`'s `$schema` field changes `validate_server_manifest.py` fails loudly, by design, if the vendored file's `$id` no longer matches `server.json`'s `$schema` value — it will not silently validate against a stale schema version. To fix that failure, -re-vendor the new version: +re-vendor the new version straight from the CDN URL server.json now +points to: ``` -NEW_VERSION= -curl -fsSL "https://raw.githubusercontent.com/modelcontextprotocol/registry//internal/validators/schemas/${NEW_VERSION}.json" \ +NEW_VERSION= # read off server.json's new $schema field +curl -fsSL "https://static.modelcontextprotocol.io/schemas/${NEW_VERSION}/server.schema.json" \ -o "scripts/schemas/${NEW_VERSION}.json" rm scripts/schemas/2025-12-11.json # or keep both if migrating gradually ``` -Then update `SCHEMA_FILENAME` in `scripts/validate_server_manifest.py`. +Then update `SCHEMA_FILE` in `scripts/validate_server_manifest.py` and +`SCHEMA_FILE` in `scripts/check_vendored_schema_drift.py`.