Drop (conversation_id, seq) unique on llm_messages — a conversation is a tree#62
Merged
Conversation
A conversation is a tree, not a line: a single parent can legitimately have two children at the same depth (e.g. a failed branch and the branch that continued), so two distinct messages can share (conversation_id, seq) and that is truth, not a duplicate. The unique constraint rejected those legitimate siblings and forced the ledger into a racy MAX(seq)+1 generation that collided under concurrent writers (PG::UniqueViolation on llm_messages_conversation_id_seq_key at scale). Message identity is already guaranteed by the unique uuid; seq becomes descriptive tree-depth with a non-unique composite index for conversation-ordered reads. SQLite emulates the constraint drop by rebuilding the table (which drops the inline uuid UNIQUE), so we re-assert the uuid unique index; on PostgreSQL DROP CONSTRAINT is surgical and it is a no-op.
6 tasks
The debug toggle (log: true, query_log: true) had leaked into committed defaults. With query_log true it wins over log in Connection#setup, installing QueryFileLogger instead of SlowQueryLogger and skipping log_warn_duration — breaking connection_spec. Restore both to false (opt-in only).
- RescueLogLevel: :debug -> :warn in rescue handle_exception calls (data.rb, extract/handlers/base.rb, models/function.rb, models/node.rb) - NoLoopDo: archiver batch loop uses bounded `while` instead of `loop do` - NoUnderscorePrefixedKwargs / UnusedMethodArgument: unused **_opts -> ** (connection_replicas_spec, tls_spec) - TaxonomyEnum: exclude migrations and specs — the cop validates LLM lane taxonomy (:tier/:type/:circuit_state) but `type:` in those paths is a Sequel column type (foreign_key type: :uuid) or the extract API, not LLM taxonomy. Removes the now-redundant inline disable in migration 136.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Production ledger crash under load:
The message nacks and retries forever ("asked 14 times").
Root cause
(conversation_id, seq)UNIQUE (migration 078) modeled a conversation as a flat line. But a conversation is a tree — a single parent can legitimately have two children at the same depth:canddare both depth 3, both children ofb. Two distinct messages (different uuids) that legitimately share(conversation_id, seq). That is truth, not a duplicate — but the unique constraint rejects one of them.Because
seqhad to be unique, the centralized ledger generated it via a racyMAX(seq)+1/latest.seq+1read-modify-write. Under concurrent consumers on a shared audit queue, two writers compute the sameseq→ collision → nack/retry loop. Theseqinteger is not a true data point — it's fabricated at write time in a place (a centralized sink receiving events from ~100k distributed producers) that cannot know the real ordering.Fix
Drop the
(conversation_id, seq)unique constraint; keep a non-unique composite index for conversation-ordered reads. Message identity remains guaranteed by the existing uniqueuuid(producer-derived, stable across redelivery, and already the key for the ledger's coalescing upsert). Ordering/lineage lives inparent_message_id+created_at, which reflect the real tree.Cross-DB
Verified against the real migration chain (078 → 135 → 136) on SQLite and via generated PostgreSQL DDL:
ALTER TABLE llm_messages DROP CONSTRAINT llm_messages_conversation_id_seq_key— surgical, touches nothing else.uuidUNIQUE — so the migration re-asserts the uuid unique index (IF NOT EXISTS, a no-op on PG).Verified: after migration, two different-uuid messages can share
(conversation_id, seq), uuid uniqueness stays intact, anddownrestores the original constraint.Blast radius
ON CONFLICT (uuid) DO UPDATEcoalescing upsert (separate PR). This migration removes the constraint that was crashing; the ledger no longer needs to invent a collision-free integer.