Skip to content

[+] allow sink schema upgrades without --metrics/--sources, fixes #1114#1175

Merged
pashagolub merged 3 commits intomasterfrom
1114-fix-upgrade
Feb 4, 2026
Merged

[+] allow sink schema upgrades without --metrics/--sources, fixes #1114#1175
pashagolub merged 3 commits intomasterfrom
1114-fix-upgrade

Conversation

@pashagolub
Copy link
Collaborator

Removed the requirement that both sources and metrics must be PostgreSQL connections. Added support for upgrading only sink when sources/metrics are YAML files. Changed error messages to warnings ([WARN]) for non-upgradeable configurations. Only fails if no components support upgrade (previously failed if any didn't support upgrade).

@pashagolub pashagolub self-assigned this Feb 3, 2026
@pashagolub pashagolub requested a review from 0xgouda February 3, 2026 17:15
@pashagolub pashagolub added refactoring Something done as it should've been done from the start sinks Where and how to store monitored data labels Feb 3, 2026
@coveralls
Copy link

coveralls commented Feb 3, 2026

Pull Request Test Coverage Report for Build 21673980335

Details

  • 47 of 72 (65.28%) changed or added relevant lines in 5 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.008%) to 76.14%

Changes Missing Coverage Covered Lines Changed/Added Lines %
internal/db/conn.go 0 4 0.0%
internal/sinks/postgres.go 0 21 0.0%
Totals Coverage Status
Change from base Build 21667726737: 0.008%
Covered Lines: 4126
Relevant Lines: 5419

💛 - Coveralls

@0xgouda
Copy link
Collaborator

0xgouda commented Feb 4, 2026

Also, when upgrading from v4.1 to v5, now the CLI states that we should run config upgrade, which succeeds, but without changing the admin schema. Maybe we should add a new sink migration for changing the functions that got updates in the admin schema?

$ go run ./cmd/pgwatch --sources="contrib/sample.sources.yaml" --sink="postgres://postgres:postgres@localhost:5437/postgres" --metrics="internal/metrics/metrics.yaml" 
2026-02-04 02:43:30.882 [INFO] [sink:postgres] [db:postgres] initialising measurements database...
2026-02-04 02:43:30.891 [INFO] [sink:postgres] [db:postgres] measurements sink is activated
2026-02-04 02:43:30.891 [ERROR] sink database schema is outdated, please run migrations using `pgwatch config upgrade` command
exit status 1

$ go run ./cmd/pgwatch --sources="contrib/sample.sources.yaml" --sink="postgres://postgres:postgres@localhost:5437/postgres" --metrics="internal/metrics/metrics.yaml" config upgrade
[WARN] configuration storage does not support upgrade, skipping
2026-02-04 02:43:35.098 [INFO] [sink:postgres] [db:postgres] initialising measurements database...
2026-02-04 02:43:35.107 [INFO] [sink:postgres] [db:postgres] measurements sink is activated
2026-02-04 02:43:35.117 [INFO] [sink:postgres] [db:postgres] Applying migration named '01110 Apply postgres sink schema migrations'...
2026-02-04 02:43:35.118 [INFO] [sink:postgres] [db:postgres] Applied migration named '01110 Apply postgres sink schema migrations'

$ go run ./cmd/pgwatch --sources="contrib/sample.sources.yaml" --sink="postgres://postgres:postgres@localhost:5437/postgres" --metrics="internal/metrics/metrics.yaml"               
2026-02-04 02:43:39.332 [INFO] [sink:postgres] [db:postgres] initialising measurements database...
2026-02-04 02:43:39.341 [INFO] [sink:postgres] [db:postgres] measurements sink is activated
2026-02-04 02:43:39.342 [INFO] [sources:1] sources refreshed
2026-02-04 02:43:39.346 [INFO] [metrics:74] [presets:14] metrics and presets refreshed
2026-02-04 02:43:39.352 [INFO] [source:omdb.test] [recovery:false] Connect OK. Version: PostgreSQL 18.1 (Ubuntu 18.1-1.pgdg24.04+2) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, 64-bit
2026-02-04 02:43:39.352 [INFO] [source:omdb.test] [metric:wal] [interval:60] starting gatherer
2026-02-04 02:43:39.353 [INFO] [source:omdb.test] [metric:wal] [rows:1] [cache:false] measurements fetched
2026-02-04 02:43:40.296 [ERROR] [sink:postgres] [db:postgres] [sql:select * from admin.ensure_partition_metric_dbname_time($1, $2, $3, $4)] [args:[wal omdb.test 2026-02-04 02:43:39.353655 +0200 EET 1 week]] [err:ERROR: cross-database references are not implemented: "subpartitions.wal_omdb.test" (SQLSTATE 0A000)] [pid:46954] [time:4.732447ms] Query

@pashagolub
Copy link
Collaborator Author

Also, when upgrading from v4.1 to v5, now the CLI states that we should run config upgrade, which succeeds, but without changing the admin schema. Maybe we should add a new sink migration for changing the functions that got updates in the admin schema?

I thought they were added already... Would you please add those in a separate PR to mention the fact of migration. thanks

Removed the requirement that both sources and metrics must be PostgreSQL
connections. Added support for upgrading only sink when sources/metrics
are YAML files. Changed error messages to warnings ([WARN]) for
non-upgradeable configurations. Only fails if no components support
upgrade (previously failed if any didn't support upgrade).
@pashagolub pashagolub merged commit 2b9e932 into master Feb 4, 2026
8 checks passed
@pashagolub pashagolub deleted the 1114-fix-upgrade branch February 4, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

refactoring Something done as it should've been done from the start sinks Where and how to store monitored data

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants