feat(player-info): integrate warm pool with buffered execution

[ChechiDev]

Scope

PR 6b of the player_info buffered-mode series. Integrates the WarmBrowserPool scaffold (PR #138) into the buffered player_info execution path. When both player_info_dispatch_buffer_enabled=True and player_info_warm_pool_enabled=True, the pool owns the engine lifecycle; CandidateProducer and BoundedCandidateBuffer (PR #141) remain in place. This closes the warm-pool gap deferred from PR #141.

Branch

feat/player-info-buffered-warm-pool → main
Head: 18be778

Relationship to merged PRs

• PR feat(browser): add disabled warm browser pool scaffold #138 (merged): WarmBrowserPool scaffold + BrowserSlotEngine Protocol + BackoffPolicy, feature flag off
• PR feat(player-info): add bounded dispatch buffer scaffold #139 (merged): bounded CandidateBuffer + CandidateProducer, claim-at-handoff semantics
• PR fix(player-info): back off duplicate-only dispatch batches #140 (merged): duplicate-peek backoff hardening
• PR feat(player-info): wire dispatch buffer execution gate #141 (merged): RateLimitGate + dispatch buffer integration — buffered mode wired, warm pool deferred
• This PR: wires WarmBrowserPool into the buffered execution path when player_info_warm_pool_enabled=True

Feature flag matrix

dispatch_buffer_enabled	warm_pool_enabled	Mode
false	(any)	direct — workers call run_direct() independently
true	false	buffered — workers call run_buffered(buffer) independently
true	true	buffered+warm_pool — WarmBrowserPool owns engine lifecycle per slot

Both flags default to false. This PR is additive and completely backward-compatible.

Default-disabled behavior

With player_info_warm_pool_enabled=False (default), execution is unchanged from PR #141: buffered workers each own their own PydollEngine via run_buffered(). The warm pool code path is never entered.

Direct mode preservation

The if not _sc.player_info_dispatch_buffer_enabled: branch is untouched. Direct mode workers call run_direct() with no gate, no buffer, and no pool — same as before PR #139.

Buffered mode without warm pool

When dispatch_buffer_enabled=True and warm_pool_enabled=False:

• _warm_pool is None
• Workers proceed to asyncio.gather(*[w.run_buffered(_dispatch_buffer) ...]) as in PR feat(player-info): wire dispatch buffer execution gate #141
• RateLimitGate and CandidateProducer still function normally
• _producer.request_stop() + _gate.shutdown() in finally

Buffered mode with warm pool

When both flags are True:

• _warm_pool = WarmBrowserPool(size=workers, slot_factory=_make_slot) is created
• results = await _warm_pool.run() replaces the asyncio.gather call
• _warm_pool.shutdown() added to finally block
• Each WorkerSlot receives a fresh PydollEngine via engine_factory, using the matching worker's profile_dir and engine_name
• claim_loop_fn wraps worker._run_buffered_loop(engine, buffer), catching CooldownRequired → returns -1 (triggers slot task_backoff + engine restart)

WarmBrowserPool lifecycle ownership

• WarmBrowserPool.run() supervises N concurrent WorkerSlot tasks
• Each slot: factory → engine.start() → engine.warmup() → on_warmup_success() → claim_loop_fn(engine) loop → on_engine_teardown() → engine.close() in finally
• On browser failure: on_engine_teardown() fires, slot waits browser_backoff, then restarts
• On task failure (claim_loop_fn returns -1): on_engine_teardown() and engine.close() fire in the finally block; slot then waits task_backoff and the next iteration creates a fresh engine via engine_factory() — intentional lifecycle ownership
• Pool exit: shutdown() cancels all slot tasks

Worker/slot lifecycle

_make_slot(slot_id) is a closure that captures:

• _w = _worker_instances[slot_id - 1] — worker instance for this slot
• _disp = shared_display — singleton display bound at slot-creation time (not at call time)
• engine_factory creates a new PydollEngine per slot using the worker's profile/name

_worker_instances are still constructed but their internal engine lifecycle is NOT used in warm_pool mode — the pool provides the engine via claim_loop_fn.

Readiness/warmup behavior

• on_warmup_success=_gate.mark_engine_ready — called by WorkerSlot after a successful engine.warmup(). This opens the RateLimitGate for production work.
• The first engine to warm up opens the gate; subsequent engines also call mark_engine_ready (idempotent for an already-open gate).

RateLimitGate interaction

• on_warmup_success=_gate.mark_engine_ready — reopens gate after any engine finishes warmup
• on_engine_teardown=_gate.cancel_recovery — cancels stale probe task when engine is torn down, WITHOUT reopening the gate (gate stays CLOSED until a new engine warms up)
• CooldownRequired in claim_fn → returns -1 → WorkerSlot triggers task_backoff + engine restart. A successful restart warmup calls mark_engine_ready again.
• gate.shutdown() in finally cancels any in-flight recovery task on clean process exit.

CandidateProducer/BoundedCandidateBuffer interaction

• _dispatch_buffer is shared across all slots via claim_fn closure capture
• CandidateProducer still runs with wait_fn=gate.wait_if_closed — pauses during rate-limit cooldown
• The buffer contract is unchanged: buffer.get() yields PlayerInfoRow; claim_by_id() acquires at handoff

DB impact

No new queries. release_to_pending() (introduced in PR #141) still handles the gate-after-claim race. No schema changes.

Migration impact

None. No Alembic migrations added or modified.

Changed files

File	Change
infrastructure/browser/pool.py	+on_warmup_success and on_engine_teardown optional callbacks to WorkerSlot
scripts/scrape_player_info.py	+warm pool branch in main() (three-mode matrix), mode log updated
tests/unit/infrastructure/browser/test_warm_browser_pool.py	+make_slot helper extended, TestWorkerSlotCallbacks class (8 tests), test_run_cancelled_error_propagates hardened
tests/unit/infrastructure/player_info/test_warm_pool_integration.py	NEW — 24 unit tests across 4 classes

Tests added/changed

TestWorkerSlotCallbacks (8 new tests in test_warm_browser_pool.py):

• test_on_warmup_success_called_after_warmup — ordering: warmup → callback → claim
• test_on_warmup_success_not_called_on_warmup_failure — suppressed on exception
• test_on_engine_teardown_called_in_finally — fires even when claim raises
• test_on_engine_teardown_called_before_engine_close — ordering: callback → close
• test_on_engine_teardown_called_on_browser_failure — fires on engine crash
• test_default_none_callbacks_no_error — None is safe (no AttributeError)
• test_both_callbacks_called_per_engine — per-engine semantics, not per-pool
• Fix: test_run_cancelled_error_propagates now asserts engine.close.assert_awaited_once()

test_warm_pool_integration.py (24 tests, new file):

• TestModeSelection (6): direct mode, buffered without pool, buffered with pool, flag matrix, default behavior, mode log string
• TestWarmPoolGateInteraction (8): mark_engine_ready on warmup, cancel_recovery on teardown, gate stays closed after teardown, CooldownRequired → -1, multiple engines gate semantics, shutdown wires gate, gate open via warmup, gate stays closed without warmup
• TestDispatchWarmPoolInteraction (6): buffer shared across slots, claim_fn passes buffer, producer runs with pool, producer stop on exit, pool shutdown in finally, buffer get semantics
• TestSecurityInvariants (4): no sensitive data in claim_fn log, engine name not logged verbosely, profile_dir not logged, gate state not exposed in logs

Unit test evidence

tests/unit/infrastructure/browser/test_warm_browser_pool.py     ~45 tests pass (new callbacks class adds 8)
tests/unit/infrastructure/player_info/test_warm_pool_integration.py  24 passed
Full suite: 833 passed, coverage 83.02%

Quality gates

ruff check .                CLEAN
flake8 .                    CLEAN
mypy --strict               Success: 188 source files, 0 errors
lint-imports                4 contracts KEPT, 0 broken
pytest (80% threshold)      833 passed, 83.02%
pip-audit                   No known vulnerabilities

Security notes

• No cookies, CDP tokens, WebSocket URLs, session credentials, or profile paths are logged
• _disp = shared_display binding makes singleton capture explicit — no accidental scope leak
• gate.py and pool.py have no config or persistence imports (enforced by lint-imports)
• CooldownRequired caught in claim_fn — exception message never logged; only converted to return code -1

Unresolved risks

• _worker_instances are fully constructed even in warm_pool mode but their internal engine lifecycle is not used — minor overhead, not a correctness issue
• recover_failed() (pre-existing) missing locked_at = NULL in recovery SQL — not in scope
• ScrapeStatus.value vs Postgres enum label mismatch (pre-existing) — not in scope

PR 7 status

PR 7 has not been started.

Merge status

This PR has not been merged and has not been configured for auto-merge.