Skip to content

Fix QuestDB playback hangs by using thread-local PG connections#2948

Open
jmthomas wants to merge 5 commits intomainfrom
quest_connection
Open

Fix QuestDB playback hangs by using thread-local PG connections#2948
jmthomas wants to merge 5 commits intomainfrom
quest_connection

Conversation

@jmthomas
Copy link
Member

@jmthomas jmthomas commented Mar 6, 2026

The shared singleton PG::Connection with a mutex only guarding creation (not usage) allowed multiple puma threads to call exec_params concurrently on the same connection. PG::Connection is not thread-safe, causing queries to hang for minutes and starving the thread pool, which resulted in "no response received" errors on the frontend during playback.

Switch to thread-local connections via Thread.current[:questdb_conn] so each puma thread gets its own PG connection.

The shared singleton PG::Connection with a mutex only guarding creation
(not usage) allowed multiple puma threads to call exec_params concurrently
on the same connection. PG::Connection is not thread-safe, causing queries
to hang for minutes and starving the thread pool, which resulted in
"no response received" errors on the frontend during playback.

Switch to thread-local connections via Thread.current[:questdb_conn] so
each puma thread gets its own PG connection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jmthomas jmthomas requested review from clayandgen and ryanmelt March 6, 2026 20:12
@codecov
Copy link

codecov bot commented Mar 6, 2026

Codecov Report

❌ Patch coverage is 16.66667% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.47%. Comparing base (71c95ef) to head (be81411).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
openc3/lib/openc3/utilities/questdb_client.rb 16.66% 10 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2948      +/-   ##
==========================================
- Coverage   78.48%   78.47%   -0.01%     
==========================================
  Files         673      673              
  Lines       55333    55334       +1     
  Branches      728      728              
==========================================
- Hits        43428    43425       -3     
- Misses      11827    11831       +4     
  Partials       78       78              
Flag Coverage Δ
python 79.74% <ø> (-0.02%) ⬇️
ruby-api 80.62% <ø> (ø)
ruby-backend 82.14% <16.66%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 9, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant