Skip to content

Mble/reduce query memory pressure#28

Draft
mble wants to merge 4 commits intomainfrom
mble/reduce-query-memory-pressure
Draft

Mble/reduce query memory pressure#28
mble wants to merge 4 commits intomainfrom
mble/reduce-query-memory-pressure

Conversation

@mble
Copy link
Copy Markdown

@mble mble commented Mar 27, 2026

No description provided.

mble added 4 commits March 27, 2026 10:42
…le queries

These queries used CROSS JOIN with pg_database or UNNEST to produce
zero-filled result sets, causing memory pressure proportional to
(lock_modes × databases) or (idle_connections × bucket_count).

- pg_locks: replace VALUES CROSS JOIN pg_database with direct JOIN,
  only returning lock modes actually held
- pg_stat_activity: remove VALUES CROSS JOIN pg_database, query
  pg_stat_activity directly with GROUP BY
- pg_process_idle: replace UNNEST implicit cross-join with raw query
  and Go-side histogram bucketing; also fixes existing bug where
  QueryRowContext only processed a single (state, application_name) group
Replace per-database metrics (19 metrics × N databases) with a single
aggregated row containing only metrics not available from other
collectors: xact_commit, xact_rollback, temp_files, temp_bytes,
deadlocks, blk_read_time, blk_write_time, active_time.

Metrics that duplicate pg_stat_activity, pg_stat_user_tables, or
pg_statio_user_tables (numbackends, blks_read, blks_hit, tup_*) are
removed. The datid/datname labels are dropped.

This is a breaking change for anyone querying these metrics by database
name.
Benchmarks pg_locks, pg_stat_database, pg_stat_user_tables,
pg_statio_user_tables, and pg_database at various scale points
(100/500/1000/5000) to measure allocation growth as catalog objects
increase.
Before running expensive queries, collectors now check the row count
against a configurable limit. When exceeded, the collector skips metric
emission and instead emits a pg_exporter_collector_rows_exceeded gauge
with the actual count.

Flags added:
  --collector.stat_user_tables.max_rows  (default: 375)
  --collector.statio_user_tables.max_rows (default: 750)
  --collector.database.max_rows          (default: 750)

Set to 0 to disable the limit. Defaults derived from benchmark data
targeting ~4MB per collector within a 32MiB memory budget.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant