Skip to content

Drop (conversation_id, seq) unique on llm_messages — a conversation is a tree#62

Merged
Esity merged 5 commits into
mainfrom
fix/llm-messages-seq-nonunique
Jul 6, 2026
Merged

Drop (conversation_id, seq) unique on llm_messages — a conversation is a tree#62
Esity merged 5 commits into
mainfrom
fix/llm-messages-seq-nonunique

Conversation

@Esity

@Esity Esity commented Jul 3, 2026

Copy link
Copy Markdown

Problem

Production ledger crash under load:

Sequel::UniqueConstraintViolation: PG::UniqueViolation:
  duplicate key value violates unique constraint "llm_messages_conversation_id_seq_key"
  DETAIL: Key (conversation_id, seq)=(158938, 2) already exists.

The message nacks and retries forever ("asked 14 times").

Root cause

(conversation_id, seq) UNIQUE (migration 078) modeled a conversation as a flat line. But a conversation is a tree — a single parent can legitimately have two children at the same depth:

a(1) -> b(2) -> c(3, failed branch)
              -> d(3) -> e(4)

c and d are both depth 3, both children of b. Two distinct messages (different uuids) that legitimately share (conversation_id, seq). That is truth, not a duplicate — but the unique constraint rejects one of them.

Because seq had to be unique, the centralized ledger generated it via a racy MAX(seq)+1 / latest.seq+1 read-modify-write. Under concurrent consumers on a shared audit queue, two writers compute the same seq → collision → nack/retry loop. The seq integer is not a true data point — it's fabricated at write time in a place (a centralized sink receiving events from ~100k distributed producers) that cannot know the real ordering.

Fix

Drop the (conversation_id, seq) unique constraint; keep a non-unique composite index for conversation-ordered reads. Message identity remains guaranteed by the existing unique uuid (producer-derived, stable across redelivery, and already the key for the ledger's coalescing upsert). Ordering/lineage lives in parent_message_id + created_at, which reflect the real tree.

Cross-DB

Verified against the real migration chain (078 → 135 → 136) on SQLite and via generated PostgreSQL DDL:

  • PostgreSQL: ALTER TABLE llm_messages DROP CONSTRAINT llm_messages_conversation_id_seq_key — surgical, touches nothing else.
  • SQLite: emulates the drop by rebuilding the table, which drops the inline uuid UNIQUE — so the migration re-asserts the uuid unique index (IF NOT EXISTS, a no-op on PG).

Verified: after migration, two different-uuid messages can share (conversation_id, seq), uuid uniqueness stays intact, and down restores the original constraint.

Blast radius

  • lex-llm-ledger already handles genuine redelivery via ON CONFLICT (uuid) DO UPDATE coalescing upsert (separate PR). This migration removes the constraint that was crashing; the ledger no longer needs to invent a collision-free integer.
  • No data migration required — dropping a constraint, adding an index.

A conversation is a tree, not a line: a single parent can legitimately
have two children at the same depth (e.g. a failed branch and the branch
that continued), so two distinct messages can share (conversation_id, seq)
and that is truth, not a duplicate.

The unique constraint rejected those legitimate siblings and forced the
ledger into a racy MAX(seq)+1 generation that collided under concurrent
writers (PG::UniqueViolation on llm_messages_conversation_id_seq_key at
scale). Message identity is already guaranteed by the unique uuid; seq
becomes descriptive tree-depth with a non-unique composite index for
conversation-ordered reads.

SQLite emulates the constraint drop by rebuilding the table (which drops
the inline uuid UNIQUE), so we re-assert the uuid unique index; on
PostgreSQL DROP CONSTRAINT is surgical and it is a no-op.
@Esity Esity requested a review from a team as a code owner July 3, 2026 21:59
Esity added 2 commits July 3, 2026 17:34
The debug toggle (log: true, query_log: true) had leaked into committed
defaults. With query_log true it wins over log in Connection#setup, installing
QueryFileLogger instead of SlowQueryLogger and skipping log_warn_duration —
breaking connection_spec. Restore both to false (opt-in only).
- RescueLogLevel: :debug -> :warn in rescue handle_exception calls
  (data.rb, extract/handlers/base.rb, models/function.rb, models/node.rb)
- NoLoopDo: archiver batch loop uses bounded `while` instead of `loop do`
- NoUnderscorePrefixedKwargs / UnusedMethodArgument: unused **_opts -> **
  (connection_replicas_spec, tls_spec)
- TaxonomyEnum: exclude migrations and specs — the cop validates LLM lane
  taxonomy (:tier/:type/:circuit_state) but `type:` in those paths is a
  Sequel column type (foreign_key type: :uuid) or the extract API, not LLM
  taxonomy. Removes the now-redundant inline disable in migration 136.
@Esity Esity self-assigned this Jul 6, 2026
@Esity Esity merged commit 1698079 into main Jul 6, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant