publish provider concurrency on-chain so gateways skip / wait - #17
Merged
Conversation
When every provider for a model is mid-job, gateways had no way to tell: the selector saw only `active` + heartbeat + price, picked one anyway, and the request died on the 30s ack timeout. - `ProviderRegistry.Provider` gains `maxConcurrentJobs` (uint32, 0 = unset) and `setMaxConcurrentJobs()` mutator + event. `openJobs` was already on-chain; combined the two tell selectors when a provider is full. - Provider node syncs `T4T_MAX_CONCURRENT_JOBS` to chain on startup if it differs from the published value. - Selector reads `openJobs` + `maxConcurrentJobs` per candidate, filters out the full ones, and returns the rest of the pool so callers can fall over or wait. - Gateway wraps `selectProvider → postJob` in a wait-and-retry loop bounded by `T4T_PROVIDER_WAIT_SECONDS` (default 120s, poll every 5s). Surfaces a `waiting_for_capacity` SSE event so clients see the wait. - Tests: registry mutator + initial-zero invariant; selector capacity filter, fallback-to-cheapest-not-busy, zero-as-unlimited, no-match fail-fast.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When every provider for a model is mid-job, gateways had no way to tell:
the selector saw only
active+ heartbeat + price, picked one anyway,and the request died on the 30s ack timeout.
ProviderRegistry.ProvidergainsmaxConcurrentJobs(uint32, 0 = unset)and
setMaxConcurrentJobs()mutator + event.openJobswas alreadyon-chain; combined the two tell selectors when a provider is full.
T4T_MAX_CONCURRENT_JOBSto chain on startup ifit differs from the published value.
openJobs+maxConcurrentJobsper candidate, filtersout the full ones, and returns the rest of the pool so callers can
fall over or wait.
selectProvider → postJobin a wait-and-retry loopbounded by
T4T_PROVIDER_WAIT_SECONDS(default 120s, poll every 5s).Surfaces a
waiting_for_capacitySSE event so clients see the wait.filter, fallback-to-cheapest-not-busy, zero-as-unlimited, no-match
fail-fast.