Skip to content

[DRK-175] Atomic idempotency key reservation (PostgreSQL store) - #341

Merged
baoduy merged 3 commits into
devfrom
feature/drk-175-idempotency-postgres-atomic-check
Aug 5, 2026
Merged

[DRK-175] Atomic idempotency key reservation (PostgreSQL store)#341
baoduy merged 3 commits into
devfrom
feature/drk-175-idempotency-postgres-atomic-check

Conversation

@baoduy

@baoduy baoduy commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Problem

IdempotencyPostgresStore had the same non-atomic check-then-act race DRK-75 fixed in IdempotencySqlServerStore: IsKeyProcessedAsync was a pure read, and MarkKeyAsProcessedAsync only wrote afterward, so two concurrent requests with the same idempotency key could both execute the protected handler.

Fix

  • IdempotencyPostgresStore.IsKeyProcessedAsync now atomically inserts a StatusCode=102 reservation placeholder (leveraging the existing UX_CompositeKey unique index) before returning "not processed" — only one concurrent caller per key proceeds to run the handler.
  • IdempotencyKeyEntity gained an internal Complete(CachedResponse) mutator so MarkKeyAsProcessedAsync completes the reservation row in place instead of blind-inserting.
  • IdempotencyOptions.InFlightReservationTimeout (default 30s) bounds how long a reservation is honoured before being treated as expired/abandoned.
  • A reservation collision against an expired row is itself reclaimed atomically via a conditional ExecuteUpdateAsync (only one racer's update affects a row) — an intermediate build had a residual race here (an unconditional "proceed as new" let every racer through), closed during this cycle's QC pass.

Tests

  • Tightened CreateItem_ConcurrentRequestsWithSameKey_OnlyOneProcessed to assert every concurrent Created response carries the identical item id (proves the handler ran once, not just that the DB has one row).
  • New CreateItem_WithExpiredInFlightReservation_ProcessesAsNewRequest — an expired reservation doesn't permanently block retries.
  • New CreateItem_ConcurrentRequestsAgainstExpiredReservation_OnlyOneProcessed — concurrent requests racing an expired reservation still execute the handler exactly once.
  • AspCore.Idempotency.NpgsqlStore.Tests: 19/19, repeated across 5 full runs for determinism. AspCore.Idempotency.Tests: 57/57. Coverage on touched files ≥ 97.7%. Clean dotnet build and dotnet pack.

Scope

src/AspNet/DKNet.AspCore.Idempotency.NpgsqlStore/Store/IdempotencyPostgresStore.cs, .../Data/IdempotencyKeyEntity.cs, src/AspNet/DKNet.AspCore.Idempotency/IdempotencyOptions.cs, src/AspNet/AspCore.Idempotency.NpgsqlStore.Tests/Integration/IdempotencyIntegrationTests.cs — no other files touched.

Addresses finding IDEM-CONCURRENCY-001 (DRK-175).

baoduy and others added 3 commits August 5, 2026 12:03
- IsKeyProcessedAsync now atomically reserves the composite key by
  inserting a StatusCode=102 placeholder row, relying on the existing
  UX_CompositeKey unique index to serialize concurrent callers (only
  the winner proceeds to run the protected handler).
- On insert collision, the blocking row is re-queried: an unexpired
  completed row replays its cached response, an unexpired reservation
  returns the existing 409/conflict path, and an expired row (stale
  reservation or completed entry nothing purges) lets the request
  proceed as new instead of permanently blocking that key.
- MarkKeyAsProcessedAsync now completes the tracked reservation row
  in place via the new IdempotencyKeyEntity.Complete(...) instead of
  blindly inserting a second row.
- Add IdempotencyOptions.InFlightReservationTimeout (default 30s)
  controlling how long a reservation is honoured before being treated
  as abandoned (R1).
- Tighten the concurrency integration test to assert the handler ran
  exactly once (identical Id across every 201), and add a new test
  proving an expired in-flight reservation does not permanently block
  retries.

DRK-175 / IDEM-CONCURRENCY-001

Co-authored-by: multica-agent <github@multica.ai>
… an expired row

Adds CreateItem_ConcurrentRequestsAgainstExpiredReservation_OnlyOneProcessed.
Seeds an already-expired StatusCode=102 reservation, then fires 5 concurrent
requests for the same key. All 5 requests miss the unexpired-row filter and
collide on the same stale row's unique-index INSERT; ReserveKeyAsync's
collision branch returns (false, null) for every one of them when the
blocking row is expired, so all 5 proceed to run the handler.

FAILS on 4802a1b: 5 distinct handler executions observed instead of 1.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: multica-agent <github@multica.ai>
…e residual race

ReserveKeyAsync's expired-collision branch returned (false, null) to every
concurrent caller that collided with the same expired row, so none of them
actually won the reservation - reopening the exact double-execution race
DRK-175 closed for the fresh-key path.

Replace the unconditional return with a conditional UPDATE (ExecuteUpdateAsync)
that only matches while the row is still expired, giving the same
single-winner guarantee the unique index gives the fresh-insert path. A
caller whose UPDATE affects zero rows lost the race and re-reads the row to
branch like the unexpired collision path.

Fixes DRK-182.

Co-authored-by: multica-agent <github@multica.ai>
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

📊 Code Coverage Report

| Metric | Coverage |
|--------|----------|
| **Line Coverage** | 82.3% |
| **Branch Coverage** | 80.6% |
| **Method Coverage** | 84.4% |

**Lines:** 3455/undefined covered
**Branches:** 1292/undefined covered

📈 [View Full Coverage Report](https://github.com/baoduy/DKNet/actions/runs/30974921330)

@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.88235% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.14%. Comparing base (4e9c6de) to head (f7438f3).

Files with missing lines Patch % Lines
...ency.NpgsqlStore/Store/IdempotencyPostgresStore.cs 79.03% 9 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #341      +/-   ##
==========================================
+ Coverage   78.99%   79.14%   +0.15%     
==========================================
  Files         169      169              
  Lines        4142     4196      +54     
  Branches      608      613       +5     
==========================================
+ Hits         3272     3321      +49     
- Misses        689      694       +5     
  Partials      181      181              
Flag Coverage Δ
unittests 79.14% <80.88%> (+0.15%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@baoduy baoduy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: #341 — Atomic idempotency key reservation (PostgreSQL store)

Verdict: APPROVED (score 10.0/10) — merging into dev. Review vote skipped (self-authored PR; GitHub rejects self-approvals).

Score

Category Score Weight
Correctness & logic 10 25%
Security 10 20%
Testing & coverage 10 20%
Maintainability & design 10 15%
Spec conformance 10 10%
Style & conventions 9.5 5%
AI-slop gate 10 5%

Weighted: 9.975 → 10.0. No hard caps triggered.

Top findings

  1. [praise] IdempotencyPostgresStore.cs:163–171 — The atomic ExecuteUpdateAsync reclaim elegantly solves the expired-row concurrency race without a schema change, using affected-row count for the same single-winner guarantee the unique index gives the fresh-insert path.
  2. [praise] IdempotencyIntegrationTests.cs:75–93 — The tightened concurrency test now asserts all 201 responses carry the identical handler-generated Id, proving single-execution rather than the old lenient check.
  3. [praise] IdempotencyKeyEntity.cs:111Complete() reuses the existing private-setter infrastructure rather than adding new public setters.
  4. [nit] IdempotencyPostgresStore.cs:83DateTime.UtcNow vs DateTimeOffset? ExpiresAt type mismatch in the LINQ expression. Works correctly in EF Core; pre-existing pattern.
  5. [suggestion] IdempotencyPostgresStore.cs:238 — The defensive catch block in MarkKeyAsProcessedAsync logs at Information level; consider Warning so unexpected triggering is observable.

Preconditions

  • CI: 6/6 passed (build-test-coverage green)
  • Coverage: ≥97.7% (QC-verified via [D175-2])
  • Base: dev, not draft, no protected paths
  • No secrets, no new dependencies, no scope creep

Out-of-scope

IdempotencySqlServerStore.cs:60–86 still uses the non-atomic check-then-act pattern — that's DRK-75's scope (parallel cycle), not introduced here.

@baoduy
baoduy merged commit a3448e6 into dev Aug 5, 2026
6 checks passed
@baoduy
baoduy deleted the feature/drk-175-idempotency-postgres-atomic-check branch August 5, 2026 04:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant