Skip to content

chat(): client-tool and approval waits skip onFinish and leave OpenTelemetry spans open #1028

Description

@JsonKim

TanStack AI version

0.42.0

Framework/Library version

Framework-agnostic - chat() directly in Node.js, without a UI framework

Describe the bug and the steps to reproduce it

Summary

When chat() pauses for client-side tool execution or tool approval, the
current server invocation ends by emitting an interrupt boundary. However, the
middleware terminal lifecycle is skipped when the tool phase is "wait".

As a result:

  • onFinish is not called for the completed server invocation.
  • otelMiddleware leaves root and iteration spans open.
  • Operation duration and token metrics are not finalized.
  • The same lifecycle is skipped when an unresolved client tool or approval is
    restored from the initial message history.

The overall generation is still waiting for a client result, but the current
server invocation has ended and should have exactly one terminal middleware
callback.

Steps to reproduce

  1. Define a client-side tool or a tool that requires approval.
  2. Add middleware with an onFinish callback, or configure otelMiddleware.
  3. Call chat() with an adapter that returns a tool call and finishes the
    iteration with finishReason: "tool_calls".
  4. Consume the stream until it emits a RUN_FINISHED interrupt for client-tool
    execution or approval.
  5. Inspect the middleware callbacks and telemetry state.

Actual behavior

  • The interrupt boundary is emitted.
  • onFinish is not called.
  • OpenTelemetry root and iteration spans remain unfinished.
  • gen_ai.client.operation.duration is not recorded.

For a pending interrupt restored from message history, a middleware error while
emitting the boundary can also produce RUN_ERROR and then incorrectly continue
into another provider call.

Expected behavior

A client-tool or approval wait should:

  • End the current middleware lifecycle exactly once with onFinish.
  • Finalize OpenTelemetry spans and metrics for that server invocation.
  • Continue using onAbort and onError for cancellation and failures.
  • Stop immediately if emitting a pending interrupt boundary fails.
  • Keep structured-output finalization suspended while waiting.
  • Resume and finalize structured output only in the later invocation that
    includes the client tool result or approval decision.

Root cause

The terminal middleware hook is skipped when toolPhase === "wait". Pending
client-tool and approval waits can also return before reaching the normal
terminal lifecycle block.

Middleware finalization and structured-output finalization need separate
semantics at this boundary: middleware should finish for the current invocation,
while structured output should remain pending for the overall generation.

A fix and regression coverage are available in #996.

Your Minimal, Reproducible Example - (Sandbox Highly Recommended)

https://stackblitz.com/~/github.com/JsonKim/tanstack-ai-client-tool-wait-repro

Screenshots or Videos (Optional)

No response

Do you intend to try to help solve this bug with your own PR?

Yes, I am also opening a PR that solves the problem along side this issue

Terms & Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions