Skip to content

fix(ai): delegate lazy stream watchdogs to transports - #3708

Closed
innocarpe wants to merge 1 commit into
Yeachan-Heo:devfrom
innocarpe:fix/provider-stream-stall-progress-timeout
Closed

fix(ai): delegate lazy stream watchdogs to transports#3708
innocarpe wants to merge 1 commit into
Yeachan-Heo:devfrom
innocarpe:fix/provider-stream-stall-progress-timeout

Conversation

@innocarpe

@innocarpe innocarpe commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Problem

The lazy provider wrapper measured idle time from normalized assistant events while OpenAI-family providers already watched richer raw transport events. Real transport progress such as response.created could refresh only the provider watchdog, causing the outer wrapper to abort an active stream first.

Changes

  • Let Anthropic, OpenAI, and Azure paths with raw transport watchdogs own timeout decisions.
  • Retain the lazy wrapper watchdog for providers without a transport watchdog.
  • Keep toolChoiceIncapability inside the first-event window.
  • Honor caller-supplied OpenAI Completions idle timeout overrides.
  • Apply semantic Responses progress filtering and caller abort propagation to Azure Responses.
  • Preserve Alibaba's centralized 600-second first-event fallback and current OpenAI Responses endpoint-query behavior.

Rebase and delivery metadata

  • Target: dev
  • Current base: f9dffed426a433bf4948bdeb8fb2fe76f25ef5ca
  • Exact head: 4702d265e5d23744721fa5cd53498757604787a1
  • Rebased from the previously reviewed head onto current upstream/dev.
  • Worktree was clean before the rebase; the branch has one PR commit over current dev.

Verification on exact head

  • bun --cwd=packages/ai run check — passed
  • Focused watchdog/abort suite — 42 passed, 0 failed, 124 assertions
  • bun test --timeout 20000 packages/ai — 2,153 passed, 337 skipped, 0 failed, 10,230 assertions
  • bun --cwd=packages/natives run build — passed (required by native-backed test loading)
  • git diff --check upstream/dev...HEAD — passed

The focused suite includes first-event timeout, provider registration, abort-source, idle iterator, and caller-abort regression coverage. Live provider outage recovery was not tested; this change does not add retries or recovery for genuine outages.

Rollback

Revert the single PR commit 4702d265e5d23744721fa5cd53498757604787a1. No schema or data migration is involved.

@Yeachan-Heo Yeachan-Heo left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES

Reviewed exact head 7ce66811479d6fc860065a0085b5ea508bc492f9 against current origin/dev c1bf3be50e2cf0dfdf51b57eb43f7a99521c94b8.

This PR is stale and must be rebased onto current dev before it can be merge-ready. The PR base is 2c99a7f226635e812bc8fe8669787242029ce662, not the requested current baseline. I inspected the watchdog handoff in packages/ai/src/providers/register-builtins.ts:235-261,436-232 and the transport-level signal/progress handling. The reviewed change preserves provider-owned raw watchdogs for OpenAI-family/Azure/Anthropic transports, keeps caller abort signals wired through those transports, and retains control-plane progress filtering; I found no additional security, credential-routing, fallback, or cancellation blocker in this exact diff.

Focused verification at the exact head passed: bun test packages/ai/test/openai-first-event-timeout.test.ts packages/ai/test/register-builtins.test.ts (27 pass). Rebase onto c1bf3be50e2cf0dfdf51b57eb43f7a99521c94b8, rerun the timeout/concurrency coverage, and do not merge this stale head.


[repo owner's gaebal-gajae (clawdbot) 🦞]

@innocarpe
innocarpe force-pushed the fix/provider-stream-stall-progress-timeout branch from 7ce6681 to 905aa90 Compare August 3, 2026 00:43
@innocarpe

innocarpe commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Requested changes are addressed on head 741beb34781a2ee79457748d13d5b1bd356f0a3d.

  • Rebased onto current dev at 36ca903537b87150551836de23f2514fca3fb127.
  • Resolved both reported conflicts while preserving the endpoint-query changes from dev, the Alibaba 600-second first-event fallback from fix(ai): widen Alibaba first-event timeout #3740, and the watchdog regression coverage in this PR.
  • bun --cwd=packages/ai run check passed.
  • The timeout/concurrency matrix passed: 115 tests, 450 assertions.
  • The complete AI package suite passed with --timeout 20000: 2153 tests passed, 337 skipped, 0 failed, 10224 assertions.
  • An independent post-rebase review found no HIGH or CRITICAL issue.

Please re-review the updated head.

@innocarpe
innocarpe force-pushed the fix/provider-stream-stall-progress-timeout branch from 905aa90 to 741beb3 Compare August 3, 2026 01:06
The lazy provider wrapper watched normalized assistant events while several providers already watched richer raw transport events. That second clock could expire after transport-only progress, replace a live response with a blank generic stall error, and race provider-specific failure handling.

Providers with raw watchdogs now own timeout decisions; the shared wrapper remains for providers that need it. OpenAI Completions now honors caller idle overrides internally, Azure shares the semantic Responses progress filter, and provider-owned paths preserve caller cancellation.

Lore-id: e4a32f9c
Constraint: providers without raw transport watchdogs retain the shared lazy watchdog
Rejected: raise the global timeout | masks watchdog ownership and delays genuine stalls
Confidence: high
Scope-risk: medium
Reversibility: code-only
Tested: packages/ai check; 2153 package tests and 10224 assertions; 115 timeout and concurrency tests
Not-tested: live provider outage recovery
@innocarpe
innocarpe force-pushed the fix/provider-stream-stall-progress-timeout branch from 741beb3 to 4702d26 Compare August 4, 2026 12:52
@innocarpe

Copy link
Copy Markdown
Contributor Author

Rebased and force-pushed the addressed head onto current upstream/dev.

  • Exact base: f9dffed426a433bf4948bdeb8fb2fe76f25ef5ca
  • Exact head: 4702d265e5d23744721fa5cd53498757604787a1
  • bun --cwd=packages/ai run check: passed
  • Focused watchdog/abort suite: 42 passed, 0 failed
  • Full AI suite: 2,153 passed, 337 skipped, 0 failed
  • Native-backed test dependency rebuilt successfully

Please re-review this exact head.

@Yeachan-Heo

Copy link
Copy Markdown
Owner

Signed: GJC / bounded AI transport owner review

VERDICT: REQUEST_CHANGES

Reviewed exact head 4702d265e5d23744721fa5cd53498757604787a1 against exact PR base f9dffed426a433bf4948bdeb8fb2fe76f25ef5ca. GitHub reports open, non-draft, clean, and mergeable. Exact-head Dev CI run 30911046824 and Public site sync run 30911046993 are terminal green; the affected @gajae-code/ai check plus register-builtins.test.ts and openai-first-event-timeout.test.ts jobs passed. There are no bot/automated reviews or inline review comments on this PR/head.

Blocking timeout-parity regression: register-builtins.ts completely removes the outer first-event watchdog for every PROVIDER_OWNED_STREAM_WATCHDOG path, but OpenAI Responses and Azure Responses only arm iterateWithIdleTimeout after client.responses.create(...) has resolved and response headers/stream setup have completed. Unlike OpenAI Completions, those transports do not map the caller's streamFirstEventTimeoutMs to the SDK request/setup timeout. On the lazy exported path, an explicit 5-second first-event limit can therefore wait up to the SDK setup timeout (the suite demonstrates a 600-second Responses setup timeout) before the provider watchdog even exists. The direct-provider tests asserting that setup may outlive streamFirstEventTimeoutMs do not cover the contract that the removed lazy watchdog previously enforced. This is also a cancellation/error-attribution gap for setup hangs: provider-specific raw-progress ownership is correct after transport iteration starts, but ownership is incomplete before then.

Keep the raw-event idle delegation, but preserve a setup/first-event bound until each marked provider has actually armed its transport watchdog, or pass and normalize the explicit first-event timeout through the Responses/Azure SDK request setup as Completions already does. Add lazy-path regression coverage for a fetch that never resolves before headers, including caller cancellation and provider-specific stream_first_event_timeout attribution.

@innocarpe

Copy link
Copy Markdown
Contributor Author

Could not reopen this PR after the force-push that addressed REQUEST_CHANGES (GitHub: state cannot be changed; branch was force-pushed or recreated).

Replacement PR with the same branch/intent and the setup/first-event parity fix: #3829

  • Exact head: a961919e4d93d1ea4d0167f5d2cd831022008750
  • Base: current dev 732856b3ccb3fade6e9fbc17908a4fbca5a7682f

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants