fix(generator): add a per-call timeout to the LLM generation request#3
Open
JoshuaBearup wants to merge 1 commit into
Open
fix(generator): add a per-call timeout to the LLM generation request#3JoshuaBearup wants to merge 1 commit into
JoshuaBearup wants to merge 1 commit into
Conversation
callLLM() fetched api.anthropic.com with no timeout, so a stalled request hung the entire deploy indefinitely (observed: a generation call that never returned, leaving the deploy wedged with no error). Wrap the fetch in an AbortController with a configurable timeout (POLYRANGE_CALL_TIMEOUT_MS, default 150000ms). On expiry the request is aborted and callLLM rejects with "LLM call timed out after Nms", so the caller's retry logic can recover instead of hanging. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a hard per-call timeout to
callLLM()ingenerator/call-llm.mjs.Why
The generation request fetched
api.anthropic.comwith no timeout:If that request stalls (network blip, provider hang), there's nothing to interrupt it — the whole deploy wedges indefinitely with no error and no recovery. I hit exactly this: a generation call that never returned, leaving the deploy stuck.
Fix
Wrap the fetch in an
AbortControllerwith a configurable timeout (POLYRANGE_CALL_TIMEOUT_MS, default150000ms). On expiry the request aborts andcallLLMrejects withLLM call timed out after Nms, so the existing retry logic can recover instead of hanging.Testing
node -cclean. Verified the timeout path withPOLYRANGE_CALL_TIMEOUT_MS=1: the call rejects fast withLLM call timed out after 1msinstead of hanging. Default behaviour is unchanged (150s ceiling, only trips on a genuine stall).