feat(player-info): wire dispatch buffer execution gate - #141
Merged
Conversation
- Add RateLimitGate (infrastructure/browser/gate.py): cooldown+probe recovery, idempotent signal_rate_limit(), cancel_recovery() for dead-engine teardown paths - Add release_to_pending() to PlayerInfoQueueRepository: IN_PROGRESS → PENDING without retry penalty (gate-after-claim race handler) - Wire gate into _run_buffered_loop: wait_if_closed() before navigation, gate-after-claim race release, signal on RateLimitError with probe_fn, cancel_recovery() in both engine teardown paths - Add 3 gate config fields to ScrapingSettings (cooldown, probe timeout, max probe attempts) - Add direct/buffered mode logging in main() - Add 16 unit tests for RateLimitGate, 10 for mode selection, 4 integration tests for release_to_pending() - Add manual soak procedure (docs/operations/player_info_buffered_mode_soak.md)
- cancel_recovery() no longer blindly reopens gate; gate stays CLOSED until new engine proves readiness via mark_engine_ready() - Add mark_engine_ready(): called after on_browser_ready()/warmup() succeeds; also cancels any residual stale recovery task - Add shutdown(): cancels recovery task on clean process exit without changing gate state; called in main() finally block - Add _MAX_PROBE_BACKOFF_SECS = 300.0 module-level constant - Add cancel_recovery() to inner except block in run_buffered() so stale probes are cancelled when on_browser_ready/loop throws unexpectedly - Fix probe_fn for non-WarmableEngine: pass probe_fn=None instead of returning True blindly (gate reopens after cooldown, not immediately) - Pass wait_fn=_gate.wait_if_closed to CandidateProducer so producer explicitly pauses while gate is closed (not just via backpressure) - Add execution_options(synchronize_session=False) to release_to_pending UPDATE to prevent identity-map stale-read trap - Add test: wait_if_closed blocks while gate is closed and unblocks on mark_engine_ready - Add tests: producer respects wait_fn; cancel_recovery leaves gate closed; mark_engine_ready opens gate; shutdown is idempotent - Add integration tests: release_to_pending is noop for DONE and FAILED rows
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
PR 6 of the player_info buffered-mode series — partial PR6. Integrates the `RateLimitGate` execution gate and `CandidateProducer` / `BoundedCandidateBuffer` dispatch pipeline into the buffered player_info worker. Adds controlled cooldown + readiness-probe recovery, gate-after-claim race handling, dead-engine teardown safety, and explicit producer pause when gate is closed.
`WarmBrowserPool` is NOT integrated in this PR. See Warm pool status section below.
Branch
`feat/player-info-buffered-mode-integration` → `main`
Head: `055c285`
Relationship to merged PRs
Changed files
Feature flag behavior
`SCRAPING__PLAYER_INFO_DISPATCH_BUFFER_ENABLED=false` (default) → direct mode, no gate, no producer, no buffer created.
`SCRAPING__PLAYER_INFO_DISPATCH_BUFFER_ENABLED=true` → buffered mode, `RateLimitGate` created from settings, shared across all workers, `CandidateProducer` started with `wait_fn=gate.wait_if_closed`.
`SCRAPING__PLAYER_INFO_WARM_POOL_ENABLED` flag exists in settings (default `false`) but is not wired into the buffered execution path in this PR.
Mutual exclusion
Strict `if/else` in `main()` — only one mode runs per process.
Dispatch buffer integration
Buffered mode uses `CandidateProducer` + `BoundedCandidateBuffer`. Producer has an explicit `wait_fn=gate.wait_if_closed` that pauses it when the gate is closed. Workers consume from the buffer via `buffer.get()` and claim via `claim_by_id()` at handoff — no direct `claim_next()` DB poll.
Gate safety invariants
Gate-after-claim race handling
If gate closes between `claim_by_id()` and navigation, `release_to_pending()` transitions IN_PROGRESS → PENDING without incrementing `retry_count`. Only player_info IN_PROGRESS rows are affected; DONE, FAILED, and PENDING rows are untouched (verified by integration tests).
Warm pool status — NOT INTEGRATED
`WarmBrowserPool` (PR #138 scaffold) is not used in the buffered execution path. Workers in buffered mode still call `self._build_engine()` independently — each worker owns its own `PydollEngine` instance.
Why not integrated in this PR: Integration requires replacing `run_buffered()`'s entire outer lifecycle loop with `WorkerSlot.run()`, adapting `CooldownRequired` semantics (no equivalent in `WorkerSlot`), threading `on_browser_ready`/`mark_engine_ready` into the slot post-warmup step, rewiring gate `cancel_recovery()` calls into the slot's exception path, and resolving a protocol mismatch (`BrowserSlotEngine` exposes only `start/warmup/close`; `_run_buffered_loop` requires `navigate/wait_for_challenge`). Estimated 100–200 lines of structural change. This is architectural work beyond the gate-integration scope of this PR.
This PR is a partial PR6. It delivers: dispatch buffer wiring + rate-limit gate + cooldown + readiness reopening + producer pause + all gate safety invariants + soak docs. It does NOT deliver: warm browser pool wiring.
Security notes
PostgreSQL impact
`release_to_pending()` issues one `UPDATE ... RETURNING id` per gate-after-claim race event. No new indexes or migrations required.
Test evidence
Quality gates
Soak procedure
`docs/operations/player_info_buffered_mode_soak.md` — 5-stage manual soak with stop criteria, SQL snapshots, and rollback instructions.
Unresolved risks
PR 7 status
PR 7 has not been started.
Merge status
This PR has not been merged and has not been configured for auto-merge.