fix(firmware): cancel hard-timeout token before throwing TimeoutException (fixes flaky test + ordering race) - #358
Conversation
…tion (fixes flaky test + ordering race) RunWithHardTimeoutAsync decided to time out via Task.Delay(hardTimeoutMs) winning the WhenAny race, but the worker's cancellation came from a SEPARATE hardTimeoutCts timer of the same duration. The two timers can fire in either order, so the delay could win a hair before hardTimeoutCts, letting a late-returning worker observe an as-yet-uncancelled token and run one further state-changing step after the caller already received a TimeoutException — the exact guarantee #326 finding #2 added. Now the abandon path calls hardTimeoutCts.Cancel() up front (idempotent), so the worker's linked token is deterministically cancelled before TimeoutException propagates. This also fixes the intermittent CI failures of RunWithHardTimeoutAsync_HardTimeoutElapses_StopsWorkerBeforeLateStep, which caught this race under load (observed failing on unrelated PRs #356 and #357). - Existing test is the regression guard (passed 30/30 locally after the fix); full suite 1635 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
/agentic_review |
PR Summary by QodoCancel hard-timeout CTS before throwing TimeoutException in WifiBridgeActivator
AI Description
Diagram
High-Level Assessment
Files changed (1)
|
|
Qodo review clean (Bugs 0), CI green, full suite 1635 pass, flaky test 30/30 after the fix. Ready for review. |
The PR looks ready for review. The fix explicitly cancels |
Summary
WifiBridgeActivator.RunWithHardTimeoutAsynchad a subtle ordering gap that was intermittently failing CI (theRunWithHardTimeoutAsync_HardTimeoutElapses_StopsWorkerBeforeLateSteptest) and, in production, could let an abandoned worker run one extra state-changing step after a timeout.Root cause
Two independent timers of the same duration:
Task.Delay(hardTimeoutMs)in theWhenAnyrace — decides when to throwTimeoutException.hardTimeoutCts's own timer — decides when the worker's linked token is cancelled.Because they're separate timers, the
Task.Delaycan win a hair beforehardTimeoutCtsfires. In that windowTimeoutExceptionis thrown while the worker'slinkedCts.Tokenis still un-cancelled — so a worker whose blockingOpen()returns late sees an un-cancelled token and can perform a further step, defeating the guarantee #326 finding #2 added ("stop before performing a further state-changing step"). Under CI load the ordering varies, so the test flaked (observed failing on unrelated PRs #356 and #357).Fix
The abandon path now calls
hardTimeoutCts.Cancel()up front (idempotent if the timer already fired), so the worker's linked token is deterministically cancelled beforeTimeoutExceptionpropagates. No reliance on the two timers racing in a lucky order. Placement is before the existing abandon/observe-fault/dispose logic; disposal ownership and the caller-cancellation preference are unchanged.Testing
dotnet test— 1635 passed / 0 failed / 2 skipped (net9.0 + net10.0); WifiBridgeActivator suite 17/17.RunWithHardTimeoutAsync_HardTimeoutElapses_StopsWorkerBeforeLateStep— which is the correct regression guard for this race — passed 30/30 consecutive runs after the fix.Not merging — for review.
🤖 Generated with Claude Code