Skip to content

Add unique constraints to prevent duplicate rows across dedup tables - #61

Open
damilolaedwards wants to merge 1 commit into
ethpandaops:masterfrom
damilolaedwards:fix/unique-constraints-on-dedup-tables
Open

Add unique constraints to prevent duplicate rows across dedup tables#61
damilolaedwards wants to merge 1 commit into
ethpandaops:masterfrom
damilolaedwards:fix/unique-constraints-on-dedup-tables

Conversation

@damilolaedwards

Copy link
Copy Markdown

Summary

  • Concurrent identical requests could each pass the check-then-insert dedup logic before either write landed, producing duplicate rows for beacon states, beacon blocks, bad blocks, bad blobs, execution block traces, execution bad blocks, and permanent blocks. Some handlers had no dedup check at all.
  • Adds real unique constraints on each affected table's natural key, and treats a constraint violation as the definitive dedup signal instead of a racy prior read. Callers now get a clean AlreadyExists instead of a duplicate row or a leaked database error.
  • Existing databases may already contain duplicates from the race this closes, so migration now removes duplicate rows (keeping the earliest) immediately before each affected table's constraint is added.

Test plan

  • go build ./... and go vet ./...
  • go test ./pkg/server/persistence/... (full suite, no regressions)
  • New migration tests: fresh database with no prior data, and upgrade from a database seeded with duplicate rows via the pre-fix schema
  • New concurrency tests: 40 concurrent identical create requests against previously racy handlers, and 20 concurrent permanent-block record calls, each landing exactly one row
  • Full repo test suite not run locally: some existing test files depend on Docker/testcontainers, unavailable in this environment

Concurrent identical requests could each pass the check-then-insert
dedup logic before either write landed, producing duplicate rows for
beacon states, beacon blocks, bad blocks, bad blobs, execution block
traces, execution bad blocks, and permanent blocks. Some of these
handlers had no dedup check at all.

Adds real unique constraints on each table's natural key and treats a
constraint violation as the definitive dedup signal instead of a
racy prior read. Callers now get a clean AlreadyExists rather than a
duplicate row or a leaked database error.

Existing databases may already contain duplicates from the race this
closes, so migration now removes duplicate rows (keeping the earliest)
immediately before each affected table's constraint is added.
@damilolaedwards
damilolaedwards requested a review from Savid as a code owner July 27, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant