Skip to content

chore(deps): bump deepeval from 4.0.3 to 4.1.5 in /tests/eval_test #394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dtehan-td merged 2 commits into from
Aug 6, 2026
Merged

chore(deps): bump deepeval from 4.0.3 to 4.1.5 in /tests/eval_test #394

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7c51ef6
chore(deps): bump deepeval from 4.0.3 to 4.1.5 in /tests/eval_test
dependabot[bot] Aug 6, 2026
977784f
Resolve merge conflicts in PR #394
dtehan-td Aug 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
319 changes: 319 additions & 0 deletions docs/fastmcp-v4-migration-plan.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,319 @@
# FastMCP v4 Migration Plan

> **Status:** Draft — 2026-08-04
> **Target:** FastMCP 4.0.0 (currently 4.0.0b1 "Fourgone Conclusion")
> **Current pin:** `fastmcp>=3.2.0,<3.4.5` / `mcp[cli]>=1.28.1,<2.0.0`

---

## Why This Matters

FastMCP 4 is a major version bump driven by the **2026-07-28 MCP specification**, which eliminates transport-level sessions. Every request is now self-describing; `initialize` handshakes are gone; capability negotiation moves to `server/discover`. This architectural shift is not cosmetic — it unlocks horizontal scaling without sticky sessions, enables a new class of interactive multi-step tools, and brings the project into alignment with where the MCP ecosystem is heading.

The good news: the core patterns in this codebase (lifespan context manager, yield-based pool lifecycle, synchronous handler functions wrapped with `asyncio.to_thread`) are preserved in v4. The migration is incremental rather than a rewrite.

---

## Strategic Value

| Capability | What it unlocks for teradata-mcp-server |
|---|---|
| **Sessionless protocol** | Stateless HTTP deployment; any replica handles any request — a prerequisite for running multiple server instances behind a load balancer |
| **`TasksExtension`** | `tdml_*` analytic functions (KMeans, XGBoost, etc.) can take minutes; background tasks let clients poll instead of blocking or timing out |
| **Guard-mode tools (`InputRequiredResult`)** | Multi-step confirmation flows for destructive `bar_*` and `sec_*` operations — server asks "are you sure?" before executing |
| **Argument completion** | Table and column name autocomplete for tool parameters — richer UX without extra round-trip tools |
| **Response caching hints** | Read-only `base_*` / `dba_*` tools can advertise cache TTLs; clients and gateways avoid redundant Teradata queries |
| **`ClientCredentialsOAuthProvider`** | Machine-to-machine auth for enterprise CI/CD pipelines and agent-to-agent calls without browser redirects |
| **Resource path security** | Built-in traversal protection on URI template resources — removes a class of injection risk |

---

## Dependency Map

Before any code changes, the full dependency floor must be met:

| Package | Current | Required for v4 |
|---|---|---|
| `fastmcp` | `>=3.2.0,<3.4.5` | `>=4.0.0` (pin to `==4.0.0b1` during evaluation) |
| `mcp[cli]` | `>=1.28.1,<2.0.0` | `>=2.0.0` |
| `pydantic` | implied | `>=2.12` |
| `fastapi` | implied | `>=0.133.0` |
| `starlette` | implied | `>=1.0.1` |
| `httpx` | no direct usage | replaced by `httpx2` in fastmcp internals — no source changes needed here |
| `truststore` | not present | pulled in transitively — affects TLS cert handling (OS trust store, not bundled certifi) |

> **Note on corporate CAs:** `truststore` reads from the OS certificate store rather than the bundled `certifi` bundle. Teradata environments using internal certificate authorities must ensure certs are installed at the OS level, not just added to a Python certifi bundle. This may require Docker image or VM config changes.

---

## Phase 0 — Spike and Validation Gate (1–2 weeks)

**Goal:** Confirm the migration is viable and surface any undiscovered issues before committing dev capacity.

### Work items

1. **Create an isolated evaluation branch** — pin `fastmcp==4.0.0b1` and `mcp>=2.0.0` in a throwaway `pyproject.toml` override. Run `uv sync` and confirm the dependency graph resolves without conflicts.

2. **Cold-start test** — run the server without a Teradata connection (`DATABASE_URI` unset) in stdio mode. Confirm the process starts, registers tools, and exits cleanly. This validates that the lifespan machinery, middleware registration, and tool decorator calls are all compatible.

3. **Verify internal FastMCP module paths** — the following import paths are used internally and are not part of FastMCP's public API contract. Check each still exists in v4:
- `fastmcp.server.middleware.error_handling.ErrorHandlingMiddleware`
- `fastmcp.server.middleware.ping.PingMiddleware`
- `fastmcp.server.dependencies.get_http_headers`
- `fastmcp.server.dependencies.get_context`
- `fastmcp.server.middleware.{CallNext, Middleware, MiddlewareContext}`

4. **Check `mcp_camelcase_compat` warning output** — run with the server briefly and observe deprecation warnings. The field names in `ToolAnnotations` (`readOnlyHint`, `destructiveHint`, `idempotentHint`) are camelCase from MCP SDK v1. SDK v2 moved to snake_case; the compat shim emits warnings. Confirm warnings are visible in log output so they are not silently masked.

5. **Evaluate beta stability** — FastMCP 4.0.0b1 was released 2026-07-28. Assess whether GA release is imminent or if the beta will be the basis for work for several weeks. Gate on this decision: proceed with beta, or wait for GA.

**Exit criterion:** Server cold-starts successfully with v4 pinned and all internal import paths verified.

---

## Phase 1 — Core Dependency and Protocol Upgrade (1–2 weeks)

**Goal:** Upgrade to FastMCP v4 and make the minimum changes required for the server to function correctly.

### 1.1 Dependency pins (`pyproject.toml`)

Update the version constraints:
```toml
# Before
"fastmcp>=3.2.0,<3.4.5",
"mcp[cli]>=1.28.1,<2.0.0",

# After
"fastmcp>=4.0.0",
"mcp[cli]>=2.0.0",
```

### 1.2 MCP SDK v2 snake_case field migration

MCP SDK v2 renames all protocol model fields from camelCase to snake_case. The `mcp_camelcase_compat` shim (default on) keeps the old names working with deprecation warnings, but they will be removed in a subsequent release. Migrate proactively.

**`app.py`** — `ToolAnnotations` constructor calls (lines 53–69):
```python
# Before
ToolAnnotations(readOnlyHint=True, idempotentHint=True)
ToolAnnotations(readOnlyHint=False, destructiveHint=True)

# After
ToolAnnotations(read_only_hint=True, idempotent_hint=True)
ToolAnnotations(read_only_hint=False, destructive_hint=True)
```

**`utils.py`** — `types.TextContent` is constructed with `type="text"` which is already correct. Verify no camelCase fields are used on other `mcp.types` objects.

### 1.3 Import path updates

**`app.py`** — `fastmcp.prompts.prompt` path is simplified:
```python
# Before
from fastmcp.prompts.prompt import Message, TextContent

# After
from fastmcp.prompts import Message, TextContent
```

Verify and update internal middleware import paths identified in Phase 0.

### 1.4 Disable `mcp_camelcase_compat` in CI

Add to the test environment to catch any remaining camelCase field usage early:
```python
import fastmcp
fastmcp.settings.mcp_camelcase_compat = False
```

**Testing:** Run the full integration test suite against a live Teradata database. All existing test cases should pass before moving to Phase 2.

---

## Phase 2 — Middleware Modernization (1–2 weeks)

**Goal:** Adapt the middleware layer to the sessionless protocol model.

### 2.1 `on_initialize` replacement for registry refresh

`RequestContextMiddleware.on_initialize` (middleware.py:207) is used to trigger a registry tool refresh when a new client session connects. In v4 on modern `2026-07-28` protocol, **`on_initialize` never fires** — there is no handshake. The server must handle this differently.

**Options (choose one):**

| Option | Trade-off |
|---|---|
| Move refresh to `on_request` with a time-based debounce (e.g. refresh at most once per N seconds) | Simple; refresh happens on first request after the interval elapses rather than at session start |
| Trigger refresh on a background schedule (e.g. using `asyncio.create_task` inside the lifespan) | Decoupled from requests; more predictable; slightly more code |
| Remove session-triggered refresh entirely; require server restart to pick up registry changes | Simplest; acceptable if registry changes are infrequent and deployments are cheap |

The recommended option is the **background schedule** approach, as it mirrors what the registry refresh is semantically trying to achieve (periodic sync) without depending on session lifecycle events that no longer exist.

### 2.2 Type guards in `on_request`

In v4, `on_request` (and `on_message`) fires for **every inbound message** — including `notifications/cancelled`, `notifications/initialized`, `notifications/progress`, and requests that fail validation. In v3 it only fired for routable requests.

`RequestContextMiddleware.on_request` (middleware.py:62) does not currently check the message type before processing. It will need a guard:

```python
async def on_request(self, context: MiddlewareContext, call_next: CallNext):
# Guard: skip non-routable messages
if not hasattr(context, 'request') or context.request is None:
return await call_next(context)
# ... existing logic
```

The exact check depends on the v4 `MiddlewareContext` shape confirmed in Phase 0. Review the v4 `MiddlewareContext` API before writing the guard.

### 2.3 Confirm `set_state` / `get_state` within-request semantics

The pattern `context.fastmcp_context.set_state("request_context", rc)` (middleware.py:76) followed by `ctx.get_state("request_context")` (factory.py:11) stores a `RequestContext` object inside the middleware and retrieves it inside the tool handler. This is a within-request (within-call) flow, not a cross-request session flow.

Verify with a live test that this pattern continues to work under v4's sessionless model. The expectation is that it does — the state object is scoped to a single request's processing pipeline, which is unchanged — but confirm empirically.

### 2.4 QueryBand per-request vs per-session semantics

Currently, QueryBand is set once during `on_initialize` for a session and reused for all subsequent requests in that session. In a sessionless model every request is independent. Confirm that QueryBand is set on each request's connection (it is: `execute_db_tool` in `app.py` calls `_set_queryband` on each connection before running the tool). No change required here, but document the behaviour explicitly.

**Testing:** Manually test session-establishing clients (Claude Desktop, a direct MCP client) against the v4 server. Confirm QueryBand is correctly applied on every tool call. Verify log output shows the expected per-request context.

---

## Phase 3 — SSE Transport Deprecation (1 week)

**Goal:** Remove SSE as a supported transport before v4 completes its removal.

The MCP specification deprecated HTTP+SSE in the `2025-03-26` release. FastMCP v4 retains SSE support for now but it will not be a long-term option. The codebase currently supports three transports: `stdio`, `streamable-http`, and `sse`.

### Work items

1. **Identify active SSE users** — check configuration docs, deployment guides, and known client integrations for anyone relying on `MCP_TRANSPORT=sse`.

2. **Remove SSE from the transport enum** in `config/__init__.py` and the CLI argument validation in `server.py`.

3. **Update documentation and Docker Compose** — remove SSE examples; replace with `streamable-http` equivalents.

4. **Migration notice** — add a deprecation warning in v3.x release notes and/or a startup warning if SSE is configured, pointing users to streamable-http.

---

## Phase 4 — New Capability Adoption (2–4 weeks, can be split across releases)

These are independent features that can be shipped individually. Order them by value-to-effort.

### 4.1 `TasksExtension` for long-running analytic tools (High value)

`tdml_*` analytic functions — KMeans, XGBoost, ARIMA, and ~86 others — can run for minutes. Currently the MCP client must keep the connection open for the entire duration. With v4's `TasksExtension`, the tool returns immediately with a task ID; clients poll for completion.

**Implementation sketch:**
```python
from fastmcp_tasks import TasksExtension

mcp.add_extension(TasksExtension())

# Then @mcp.tool(task=True) on tdml_* wrappers
```

The tdml tool wrappers are generated inside `teradata_lifespan` — the `task=True` flag can be applied there. An in-memory backend is sufficient for single-instance deployments; Redis/Valkey is needed for multi-replica.

### 4.2 Argument completion for table and column names (Medium value)

Add a `@mcp.completion` handler for tool parameters that accept table or column names. The handler queries Teradata's information schema and returns matching values as the user types.

```python
@mcp.completion
async def complete_table_name(ref, argument, context):
if argument.name == "table_name":
# query DBC.TablesV for matches to argument.value
...
```

This surfaces across all tools that take a `table_name` parameter without any changes to the tool handlers themselves.

### 4.3 Response caching hints (Low effort, medium value)

`base_*`, `dba_*`, and `rag_*` tools are declared read-only and idempotent via `ToolAnnotations`. Pair this with v4's `cache_ttl` and `cache_scope` constructor parameters to give MCP clients and gateways explicit freshness signals:

```python
mcp = FastMCP(
"teradata-mcp-server",
lifespan=teradata_lifespan,
mask_error_details=True,
cache_ttl=300, # 5 minutes for read-only tools
cache_scope="public",
)
```

### 4.4 Guard-mode tools for destructive operations (Medium value, higher effort)

`bar_*` (backup/restore) and specific `sec_*` operations are marked `destructiveHint=True`. Use `InputRequiredResult` to gate them behind a confirmation step:

```python
from fastmcp import InputRequiredResult

async def handle_bar_drop_table(table_name: str, ctx):
if not ctx.input_responses:
return InputRequiredResult(
message=f"Drop table '{table_name}'? This is irreversible.",
response_type=bool,
)
if not ctx.input_responses[0]:
return "Operation cancelled."
# proceed
```

This is most impactful for operations invoked from agentic workflows where the LLM might otherwise proceed without human confirmation.

---

## Phase 5 — Horizontal Scaling Readiness (Future / stretch goal)

With the sessionless protocol in place after Phases 1–3, the server is ready in principle for multi-replica deployment. Formalising this requires:

1. **Shared session state backend** — replace the in-memory `_ConnState` holder with a shared store if per-user state is needed across replicas.
2. **`UserSession` primitive** — explicitly scope per-user auth context using v4's `UserSession` rather than relying on per-process state.
3. **`RequestStateSecurity`** — if guard-mode tools are in use across replicas, configure a shared key so opaque `requestState` tokens are verifiable by any replica.
4. **`ClientCredentialsOAuthProvider`** — add machine-to-machine authentication for automated pipeline scenarios.

This phase depends on the deployment architecture decision (single instance vs fleet) and is not required for v4 compatibility.

---

## Risks and Mitigations

| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| Internal FastMCP module paths (middleware, dependencies) change without notice in v4 | Medium | Medium | Validate every import path in Phase 0 spike; switch to public API alternatives where available |
| `on_initialize` removal breaks registry refresh for clients that rely on per-session tool list refresh | High | Medium | Phase 2.1 — design the background-schedule alternative before merging v4 upgrade |
| `mcp_camelcase_compat` shim masks latent field-name bugs until it is removed in a future version | Low | Low | Disable the shim in CI (Phase 1.4) |
| TLS failures in environments using internal CAs after `truststore` replaces `certifi` | Medium | High for enterprise | Test against corporate CA environment in Phase 1; update Docker base image cert setup |
| FastMCP 4.0.0b1 introduces regressions before GA | Low-Medium | Medium | Pin to the exact beta version during evaluation; upgrade to GA before merging to main |
| `set_state` / `get_state` within-request semantics silently break | Low | High | Explicit integration test in Phase 2.3 before shipping |

---

## Suggested Sequencing

```
Phase 0 ──► Phase 1 ──► Phase 2 ──► Phase 3
(spike) (upgrade) (middleware) (SSE removal)
│
▼
Phase 4 (features, can ship independently)
│
▼
Phase 5 (scaling, future)
```

Phases 0–3 are the migration proper and should be treated as a single contiguous workstream. Phase 4 items are enhancements that can be scoped into normal feature releases after the migration is stable.

---

## Open Questions

1. **Beta or wait for GA?** FastMCP 4.0.0b1 was released 2026-07-28. The spike in Phase 0 will reveal whether the API is stable enough to commit to. If the GA release is within a few weeks, waiting is low risk.

2. **Which Phase 4 feature ships first?** `TasksExtension` (long-running analytics) has the clearest operational value. Argument completion is lower effort and directly improves UX. These can proceed in parallel if bandwidth allows.

3. **SSE client inventory** — who is currently using `MCP_TRANSPORT=sse`? This determines how much lead time Phase 3 needs before removal.

4. **Multi-replica intent** — if horizontal scaling (Phase 5) is on the near-term roadmap, the middleware design in Phase 2 should account for it now rather than requiring a second refactor.
Loading