fix(identity): close the check-then-act race on max_domains - #901
Merged
jiashuoz merged 1 commit intoAug 16, 2026
Merged
Conversation
…opy#822) CheckDomainCreate counted existing domains, then ClaimDomain inserted the new row, as two independent transactions with nothing serializing them: concurrent creates could all read the same pre-insert count and all pass, blowing the cap by an arbitrary factor (reported with a 12-concurrent-request repro against a cap of 1, all 12 succeeding). ClaimOrCreateDomainWithLimit now takes a per-user advisory lock and re-checks the count inside the same transaction as the INSERT, so the check and the insert can no longer interleave across requests. EnforceDomainCreate stays as a fast pre-check (plan code, upgrade URL); the transactional check inside ClaimDomain is now the race-proof source of truth for the cap itself. Fixes tokencanopy#822 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
jiashuoz
approved these changes
Aug 16, 2026
jiashuoz
left a comment
Member
There was a problem hiding this comment.
Solid fix — the race is real, the advisory-lock approach is sound, and the test coverage (including a fails-on-main e2e repro) is exactly what this needed. Approving.
Verified while reviewing:
- Lock design:
hashtextextended(userID, 1)is a distinct keyspace from the seed-0 domain locks and thesender-identity:session lock; ordering (domain locks → user lock) is consistent and the user lock is taken in exactly one place, so no deadlock cycle is possible. Under READ COMMITTED the post-lockcount(*)sees every insert committed before lock acquisition — properly serialized across connections, pools, and processes on the same DB. - The in-tx count is byte-identical to
usage.CountDomainsByUser, so the backstop andCheckDomainCreatecan't disagree about what consumes a slot. - Cap is only charged on genuine inserts; same-owner re-claims return before the count check, preserving idempotent re-register at cap.
GetLimitsfailure → 500 beforeClaimDomainis called (fail-safe, and pinned by the new unit test), andDomainLimitExceededErrormaps to the same402 limit_exceededshape clients already get fromlimitEnvelope. No API-shape change, so no contract-scenario/spec obligations.
Two non-blocking suggestions:
- Consider fetching
GetLimitsunconditionally inhandleRegisterDomain(while keeping theEnforceDomainCreatepre-check skipped on thealreadyOwnedpath). TodayalreadyOwnedpassesmaxDomains=0, so aDELETEracing a re-claim of the same domain would fall through to the insert branch with the cap check disabled. Contrived, but the store only charges on genuine inserts anyway, so passing the real limit costs nothing. - The
maxDomains <= 0= "unlimited" sentinel inverts the pre-check's semantics (whereMaxDomains=0blocks everything). Unreachable today since a zero-cap plan 402s in the pre-check first, but worth a one-line comment at the lock site so a future zero-cap plan doesn't silently disable the backstop.
Nit: the 500 message for a GetLimits failure is identical to the EnforceDomainCreate failure message ("limits check failed") — distinguishing them would ease log triage.
Contributor
Author
|
Thanks for the thorough review, Josh, and for catching the lock-ordering and idempotent-reclaim details. Good calls on both suggestions, I will follow up with a small PR for the unconditional |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CheckDomainCreate(the limits pre-check) andClaimOrCreateDomain(the insert) ran astwo independent transactions with nothing serializing them: concurrent
POST /v1/domainsrequests could all read the same pre-insert count and all pass, so
max_domainscould beexceeded by an arbitrary factor. Reported with a live reproduction against real Postgres
(12 concurrent requests all succeeded against a cap of 1).
ClaimOrCreateDomainWithLimitnow takes a per-user advisory lock and re-checks the countinside the same transaction as the
INSERT, so the check and the write can no longerinterleave across requests.
EnforceDomainCreatestays as a fast, richly-detailedpre-check (plan code, upgrade URL); the transactional check inside
ClaimDomainis therace-proof source of truth for the cap itself.
Operational risk
No schema change, no behavior change on the non-concurrent path. A request that
previously slipped past the cap under a race now gets
402 limit_exceeded, the sameresponse the endpoint already returns for a non-racy over-cap request.
Test plan
internal/e2e,-tags integration): 8 concurrentPOST /v1/domainsagainst
max_domains=1: fails onmain(8/8 succeed, cap blown 8x), passes on thisbranch (1 created, 7
402 limit_exceeded), both against a real Postgres 16 containergo test ./internal/identity/...and./internal/httpapi/...green, including twonew unit tests for the
ClaimDomain/GetLimitserror-mapping pathsgo test -p 4 -covermode=atomic ./internal/...(theGo coverage gatejob's owncommand) green
connection pool / process
Fixes #822