CHECK24 MCP Proxy Challenge

A single MCP endpoint for ChatGPT that aggregates multiple verticals behind one proxy. Built for the CHECK24 GenDev Scholarship Challenge.

The project focuses on MCP aggregation, routing, conformance validation, versioning, and developer tooling, while keeping the architecture reusable across different MCP hosts.

Live Demo

Feature	URL
MCP Endpoint	`https://check24-chatgpt-app-challenge-web.vercel.app/api/mcp`
Conformance Dashboard	`/conformance`
Versioning Dashboard	`/versioning`
Tool-call Logs	`/logs`
Sandbox UI	`/sandbox`

Problem Overview

CHECK24 consists of multiple independent product verticals such as:

Flights
Hotels
Package holidays
Vacation rentals
etc.

Each vertical owns its own MCP server and defines its own tools independently.

For a more unified integration surface, all vertical MCP servers are aggregated behind a single ChatGPT-facing proxy.

This project introduces a proxy layer responsible for aggregation, routing, conformance validation, telemetry, and feedback to vertical teams.

System Overview

The whole system in one picture: the runtime that serves ChatGPT, the GitHub side that drives builds and releases, and the local stdio path for development.

The system spans three zones:

Vercel Cloud runs the Next.js 15 App Router. The MCP endpoint (POST /api/mcp) and the internal dashboards both call MCP-Proxy-Core, which fans out in-process to the four vertical MCP servers. The shared package holds the Zod schemas every vertical depends on. The mock-data generators live alongside the verticals themselves, in packages/mcp-verticals/src/data/.
GitHub Monorepo drives the build. Pnpm workspaces and Turbo orchestrate the package builds. Vertical release tags (flight@v1.0.0) trigger a GitHub Actions workflow that regenerates version-manifest.json and commits it back to main, and every push to main triggers a Vercel deploy of the new app.
Local Development wraps the same proxy core in a stdio transport, so MCP Inspector or Claude Desktop can connect to the local server without any deployment.

The runtime stack itself is short:

Layer	Choice
Monorepo	pnpm workspaces + Turbo
Frontend and API	Next.js 15 App Router
MCP runtime	`@modelcontextprotocol/sdk`
Schemas	Zod
Hosting	Vercel (serverless Node.js)
Release automation	GitHub Actions

Architecture

The system is built around a framework-agnostic proxy core that sits between ChatGPT and the vertical MCP servers.

The architecture is split into several layers:

Module	Responsibility
`packages/mcp-proxy-core`	Aggregation, routing, validation, telemetry
`packages/mcp-verticals`	Mock MCP servers
`packages/shared`	Shared schemas and types
`apps/web`	Next.js frontend and HTTP API
`apps/cli`	CLI and stdio MCP server

One important design decision was separating the proxy core from the hosting layer.

@project/mcp-proxy-core knows nothing about HTTP, Next.js, or stdio. It only speaks JSON-RPC. Any host can wire it up by calling proxy.handle(request) and passing back the response. This means the same aggregation, routing, conformance, and telemetry logic runs identically in three different contexts:

Host	How it wires up the proxy	Used by
`apps/web`	Next.js route calls `proxy.handle()` on every POST to `/api/mcp`	ChatGPT, MCP Inspector (HTTP)
`apps/cli` commands	Calls `proxy.handle()` directly and prints the result	Developers
`apps/cli serve`	Wraps the proxy in a `StdioServerTransport`	Claude Desktop, Cursor, MCP Inspector (stdio)

Adding a new host (WebSocket, Lambda, a different framework) therefore requires zero changes to the proxy itself.

Request Flow

The proxy handles two distinct MCP operations differently.

On tools/list (when ChatGPT connects):

The proxy queries all vertical MCP servers in parallel
Each tool is prefixed with its vertical name (flight__search_flights, hotel__search_hotels)
Conformance validation runs, tools that violate the rules are blocked and reported
The merged, validated list is returned to ChatGPT

On tools/call (when ChatGPT invokes a tool):

The proxy strips the prefix to identify the vertical (flight__search_flights → flight)
Conformance re-runs as a defence-in-depth check
The call is forwarded to the correct vertical MCP server in-process
The result and metadata are recorded by the telemetry hook

ChatGPT never communicates directly with the vertical MCP servers.

Mock Vertical MCP Servers

The repository includes four mocked vertical MCP servers:

Vertical	Example tools
Flight	`search_flights`
Hotel	`search_hotels`
Holiday Package	`search_holiday_packages`
Vacation Renting	`search_vacation_rentals`

Each vertical defines its own schemas and tools independently.

Some demo variants intentionally violate Apps SDK expectations so the conformance system can demonstrate blocking and feedback behavior.

Conformance Validation

Before any tool reaches ChatGPT, the proxy validates it against the OpenAI Apps SDK rules. Tools that fail are blocked from tools/list entirely. They are not silently dropped, but reported with the violated rule, the exact match that triggered it, and an actionable fix suggestion.

The checks cover five categories:

Category	What is checked
Tool name	Must start with an action verb (`search_`, `get_`), no promotional terms (`best`, `official`, `nr1`)
Description	No disparagement, model-steering, or over-broad activation language; length between 20–1024 chars
Annotations	All three MCP hints required: `readOnlyHint`, `destructiveHint`, `openWorldHint`
Input fields	Forbidden fields blocked: GPS coordinates, full conversation history
Schema structure	Name format, inputSchema shape validated via Zod

Feedback surfaces in three places so vertical teams can act on it regardless of their workflow:

/conformance dashboard: full report per vertical with rule ID, matched terms, and fix hint
_meta.conformance in every tools/list response: visible to any MCP client, not just the dashboards
pnpm cli conformance: same report on the command line, usable in CI

Versioning Concept

Verticals evolve independently and at different speeds. A flight team might ship a new tool every week, a vacation rentals team might be stable for months. The challenge is letting each vertical be auditable and reviewable on its own clock inside one shared deployment, without standing up a separate package registry or version database for the proxy to consult.

The idea: Git as the versioning platform

This project treats Git itself as the version registry. Each vertical owns a namespace of tags (flight@v1.0.0, hotel@v2.1.0, proxy@v1.0.0), and the manifest the dashboard reads is a JSON file generated from those tags and committed back to the repo. There is no separate database, no external registry, no version service to call. The repo is the source of truth.

The three states the dashboard surfaces are derived deterministically from Git:

State	Meaning
`reviewed`	The latest tag matches the current commit (signed off and ready for release)
`pending`	New commits have landed since the last tag (needs re-review before the next release)
`untagged`	No tag exists yet (the vertical has never been reviewed)

How it fits together

Each vertical declares the paths it owns (e.g. packages/mcp-verticals/src/flight.ts, packages/shared/src/schemas/flight.ts, apps/web/src/app/widgets/flights). The release flow is:

A developer changes code under one of a vertical's declared paths and merges to main.
When the vertical is ready for release, the maintainer pushes a namespaced tag: flight@v1.1.0.
The tag push triggers a GitHub Actions workflow that checks out the full Git history (Vercel cannot do this, its clones are shallow), iterates over each vertical, finds the latest matching tag, and runs git rev-list --count <tag>..HEAD -- <paths> to count how many commits have touched that vertical since.
The workflow writes the result into version-manifest.json and commits it back to main.
That commit triggers a normal Vercel deploy. The bundled manifest ships with the app.
The /versioning dashboard reads the manifest at runtime and renders the state per vertical, plus a workflow guide that walks contributors through the tag-and-release flow.

The proxy core is treated as just another versioned component, with its own proxy@v1.0.0 namespace, so infrastructure changes go through the same review gate as a vertical.

Why this approach

Zero infrastructure. The version registry is the Git repo. No Artifactory, no Redis, no external API. A git tag is the entire release ceremony, and git describe + git rev-list answer every question the dashboard needs.
Auditable by construction. git log flight@v1.0.0..HEAD -- <flight paths> lists exactly what landed between releases, and there is no parallel system that could disagree with Git. Tags are immutable once pushed, which means the audit trail cannot be quietly rewritten.
Per-vertical independence. Flight can be on v3.2.0 while Hotel is still on v1.0.0. Each team controls its own release cadence without coordinating versions with the others.
Drift is visible immediately. The moment a commit lands on a tracked path without a matching tag, the vertical flips to pending on the dashboard. Reviewers see at a glance what has not yet been signed off.
The proxy itself is reviewable too. proxy@v1.0.0 covers packages/mcp-proxy-core/src, so changes to aggregation, routing, conformance, or telemetry follow the same gate as a vertical, not a separate process.

Trade-offs

The approach is lightweight, which means some things are out of scope:

All verticals share one deployment. Releasing one vertical implies redeploying the proxy and every other vertical in the same Next.js bundle. Per-vertical canary releases or independent rollback would need separate hosts (one Vercel project per vertical, or a different runtime entirely).
Tagging is a manual decision. Nothing forces a maintainer to tag. If nobody pushes flight@v1.1.0, the vertical sits in pending forever. Automating tag creation from conventional commits (e.g. changesets or release-please) is a natural next step.
Only declared paths count. If a change lands outside any vertical's path list (for example a generic refactor of apps/web or a bump in pnpm-lock.yaml), no vertical reflects it. This keeps the per-vertical signal clean but means cross-cutting infrastructure changes need their own conventions.
commits ahead counts commits, not semantic changes. A doc-only edit inside a vertical's tracked path still bumps the counter. The signal is conservative (better to ask for review than to skip it), but housekeeping commits create noise.
The proxy does not gate on version state. reviewed vs pending is informational, not blocking. Pending code still serves traffic on Vercel, but ChatGPT users only see changes after OpenAI re-approves the connector. The real release gate is connector review; the dashboard is the team's pre-submission discipline.
Vercel cannot regenerate the manifest at build time. Vercel uses shallow clones with no tag history, so the manifest must be precomputed by GitHub Actions and committed. The build-time path only patches generatedAt and gitCommit, never the statuses.

Telemetry and Monitoring

Every tool call is captured by the telemetry hook and stored with its full context: timestamp, vertical, tool name, arguments, latency, status (ok / error / blocked), and the conversation and session IDs extracted from the request headers.

The /logs dashboard shows a live feed of calls with 2-second polling, and a grouped-by-conversation view so you can trace exactly what happened in a given ChatGPT session.

For this challenge the telemetry uses an in-memory ring buffer (500 entries, FIFO). This is intentional for a demo deployment: no database required, zero configuration, resets cleanly on redeploy. Every entry is also written to console.log, which Vercel Observability captures automatically and makes searchable across deployments, so even in the demo, calls are not truly lost when the instance recycles.

In a production setup the same onCall hook in McpProxyOptions would be wired to a proper observability sink: Vercel Observability with log drains, OpenTelemetry, Datadog, or similar. Because the hook is part of the proxy options and not baked into the core, switching requires no changes to the proxy itself:

const proxy = new McpProxy({
  verticals: [...],
  onCall: (entry) => {
    otelSpan.record(entry);   // OpenTelemetry
    // or: axiom.ingest(entry), datadog.log(entry), redis.xadd(entry), …
  },
});

Vercel Observability with log drains would be the natural first step here. It requires no code changes, just a Vercel Pro plan. That was not set up for this challenge submission, but the console.log entries are already structured and ready to be picked up the moment a drain is configured.

Per-Vertical QA Mode

Vertical teams need to validate their own tools through the same proxy ChatGPT uses, but without other verticals cluttering the tool list. A single request header scopes the proxy to one vertical:

curl https://.../api/mcp \
  -H 'x-mcp-vertical: hotel' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

With the scope active, tools/list returns only that vertical's tools, and any tools/call targeting a different vertical returns a -32602 error. This lets a vertical team run a full end-to-end QA cycle (connect, list, call, inspect the conformance report) without touching a separate deployment or bypassing the proxy.

The scope can also be set via query string (?vertical=hotel), the sandbox UI, or pnpm cli serve --vertical hotel for stdio clients.

Interfaces

Web (primary)

The main interface is the Next.js app deployed on Vercel. It exposes the MCP endpoint and four internal dashboards:

Path	Purpose
`POST /api/mcp`	The ChatGPT-facing MCP endpoint
`/conformance`	Live conformance report per vertical
`/versioning`	Release state per vertical with workflow guide
`/logs`	Live tool-call telemetry
`/sandbox`	Chat UI for testing against the proxy directly

CLI (optional)

The CLI is not required for the core use case. It exists to demonstrate that @project/mcp-proxy-core is genuinely framework-agnostic. The same proxy logic that runs behind the Next.js HTTP route can be driven from the command line or exposed as a stdio MCP server without any changes to the core.

pnpm cli list                                        # list all aggregated tools
pnpm cli call flight__search_flights '{"origin":"FRA","destination":"MAD","date":"2025-08-01","passengers":1}'
pnpm cli conformance                                 # full conformance report
pnpm cli serve                                       # start a stdio MCP server (Claude Desktop, Cursor)

Repository Structure

apps/
  cli/
  web/

packages/
  mcp-proxy-core/
  mcp-verticals/
  shared/

scripts/

Local Development

Install dependencies

pnpm install

Start development server

pnpm dev

Runs on:

http://localhost:3014

Tests

pnpm test

The Vitest suite covers packages/mcp-proxy-core with 33 tests across three files:

File	What it tests
`proxy.test.ts`	End-to-end routing with stub verticals, correct vertical receives the call, unknown prefixes return `-32602`, vertical scoping header filters the tool list, `onCall` hook fires with the right payload
`conformance.test.ts`	`validateTool`, clean tools pass, promotional names are caught, missing annotations are flagged, forbidden input fields (GPS, conversation history) are blocked, weighted keyword score becomes a warning at 2–5 and an error at ≥6
`logger.test.ts`	`RingBuffer` ordering, wrap-around at capacity, clear, copy semantics

Optional Features Implemented

Cross-vertical ambiguity mitigation

When multiple verticals cover similar intents (flights vs. package holidays, hotel vs. vacation rental), the model needs clear signals to pick the right one. Three layers address this:

Namespace prefix: Every tool name is <vertical>__<tool>, so flight__search_flights and holiday_package__search_holiday_packages are unambiguous by construction. The proxy rejects any tools/call without a valid prefix.
Routing hints: Each vertical's tools carry an explicit hint in the description: "ONLY for standalone flight searches. Do NOT use for Pauschalreisen." The negative cases matter most, the most common confusion is between Flug and Pauschalreise, which bundles flight + hotel.
System instructions: The initialize response includes a top-level instruction listing every vertical prefix and its scope, with the rule: match the user intent to exactly one vertical.

These signals influence the model but cannot guarantee correct routing. If a user says "Ich brauch was für Mallorca", whether ChatGPT picks flight__search_flights or holiday_package__search_holiday_packages is the model's decision. By the time tools/call reaches the proxy, the routing has already happened. The proxy can shape the decision through tool definitions, but cannot intercept or override it.

Sandbox UI

/sandbox is a chat interface that connects directly to the deployed proxy. It supports switching between LLM providers (OpenAI, Anthropic, Google) and scoping to a single vertical via the x-mcp-vertical header. Widgets render in real iframes so vertical teams can validate the full search → result → widget flow without a separate deployment.

Conversation-level telemetry

The proxy extracts openai-conversation-id and mcp-session-id from request headers and attaches them to every telemetry entry. The /logs dashboard has a Conversations view that groups calls by conversation, so you can replay exactly what happened in a given ChatGPT session, which tools fired, in what order, with what latency.

Security Notes

This is a challenge submission, not a production deployment. The trade-offs below are deliberate for the demo context but would not be acceptable in a real rollout:

Trade-off	Production approach
Verticals run in-process, sharing the proxy's memory	Deploy verticals as separate services with mutual auth
No authentication on the MCP endpoint	OAuth via the ChatGPT connector flow
Telemetry stored in-memory	Persistent store with sensitive fields scrubbed before writing
`?bypass-conformance=true` query param exists for QA	Scope to authenticated developer accounts only
Optional LLM deep-check accepts API key via query string	Server-side secret, never user input

Future Improvements

In rough priority order:

Persistent telemetry: Swap the ring buffer for Vercel KV or Upstash Redis, add Vercel log drain for structured export to Datadog or Axiom
Health dashboard: Aggregate the last hour of telemetry into per-vertical error rates and p95 latency with a green/yellow/red signal
Conformance rule versioning: Tag rule sets (conformance@v1.0.0) so a vertical can pin against a stable snapshot of the rules
Widget integration tests: Playwright against /sandbox to assert widgets render correctly for each vertical's structured content format
More vertical mocks: Stress the routing under realistic ambiguity (e.g. mietwagen, bahn: both transport, neither is flight)

Deployment

The proxy is deployed on Vercel at:

https://check24-chatgpt-app-challenge-web.vercel.app/api/mcp

The endpoint runs as a serverless Node.js function. Each request spins up an isolated execution context, handles the JSON-RPC call, and returns. This fits the MCP Streamable HTTP transport (the current MCP spec). Clients POST a request and receive a JSON response. SSE streaming responses, which the spec also allows, are not used here because serverless functions cannot hold persistent connections. For a deployment that needs server-initiated messages or long-running streams, a persistent host (e.g. a containerized Node.js server) would be needed.

Vertical MCP servers are never exposed publicly. They run in-process inside the same Vercel deployment, only reachable through proxy.handle(). There is no network address for a vertical. ChatGPT cannot reach one directly even if it tried.

Name		Name	Last commit message	Last commit date
Latest commit History 119 Commits
.github/workflows		.github/workflows
apps		apps
docs		docs
packages		packages
scripts		scripts
.gitignore		.gitignore
.nvmrc		.nvmrc
README.md		README.md
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.base.json		tsconfig.base.json
turbo.json		turbo.json

Navigation Menu

Repository files navigation

CHECK24 MCP Proxy Challenge

Live Demo

Problem Overview

System Overview

Architecture

Request Flow

Mock Vertical MCP Servers

Conformance Validation

Versioning Concept

The idea: Git as the versioning platform

How it fits together

Why this approach

Trade-offs

Telemetry and Monitoring

Per-Vertical QA Mode

Interfaces

Web (primary)

CLI (optional)

Repository Structure

Local Development

Install dependencies

Start development server

Tests

Optional Features Implemented

Cross-vertical ambiguity mitigation

Sandbox UI

Conversation-level telemetry

Security Notes

Future Improvements

Deployment

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

CHECK24 MCP Proxy Challenge

Live Demo

Problem Overview

System Overview

Architecture

Request Flow

Mock Vertical MCP Servers

Conformance Validation

Versioning Concept

The idea: Git as the versioning platform

How it fits together

Why this approach

Trade-offs

Telemetry and Monitoring

Per-Vertical QA Mode

Interfaces

Web (primary)

CLI (optional)

Repository Structure

Local Development

Install dependencies

Start development server

Tests

Optional Features Implemented

Cross-vertical ambiguity mitigation

Sandbox UI

Conversation-level telemetry

Security Notes

Future Improvements

Deployment

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages