Conversation
* Fix pg_description joins with classoid * Add regression tests for pg_class classoid joins * Fix remaining pg_class-backed description joins * chore: remove trailing spaces from classoid filter query * Fix style of sequence/index pg_class query builders * chore(changelog): add v1.0.19 bug fix entry for pr 177 --------- Co-authored-by: iBarBuba <350579+iBarBuba@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR applies v1.0.19, focusing on fixing PostgreSQL catalog comment join collisions, improving dump-write performance via streaming ZIP serialization, and reducing comparer overhead on grants/serial detection/routine handling.
Changes:
- Scoped
pg_descriptionjoins topg_class(classoid+objsubid) and refactored several SQL builders into helper methods with normalized formatting. - Streamed dump serialization directly into
ZipWriterthrough a buffered writer to reduce peak memory and improve throughput. - Reduced comparer overhead (column grants, serial column keys, routines/sequences handling) and added regression tests for the above.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| app/src/dump/table.rs | Adds pg_class scoping for column/index comment joins; extracts index query builder + regression test. |
| app/src/dump/core.rs | Streams dump JSON into ZIP via BufWriter; extracts multiple query builders + adds regression tests. |
| app/src/comparer/core.rs | Optimizes grants dispatch, serial column keying, and reduces routine/sequences overhead. |
| app/src/comparer/core_tests.rs | Adds regression tests for column-grants dispatch and dotted identifiers in serial detection. |
| app/Cargo.toml | Bumps crate version to 1.0.19. |
| CHANGELOG | Documents v1.0.19 release notes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug fixes:
helper methods on
Dump:build_tables_standalone_query,build_regular_views_query,build_materialized_views_query,build_view_column_comments_query,build_sequences_standalone_query,and on
Table:build_indexes_bulk_query.pg_descriptionjoins scoped to relation comments byfiltering on
pg_description.classoid = 'pg_class'::regclassand
pg_description.objsubid = 0to avoid catalogcollisions.
Comparer::compare_grantsno longer runs a duplicateiter().find(...)overfrom_colsfor every TO columnwhose own ACL is empty. The first scan was discarded
and the second was redone unconditionally; both are
now replaced by a single per-table
HashMap<&str, &[String]>lookup, which also reducesthe column-grants emission from O(C^2) to O(C) on
wide tables.
Comparer::mark_serial_columnsnow keysserial_columnsby(schema, table, column)tupleinstead of a
format!("{}.{}.{}", ...)string thatwas parsed back via
splitn(3, '.'). The stringform silently misparsed any identifier containing a
literal
.(legal in PostgreSQL when quoted),leaving the corresponding column unmarked.
Comparer::compare_sequenceshad a deaddropped_sequences: HashSet<String>that waschecked and updated on every iteration of a loop
over
self.from.sequences. Sequences are unique by(schema, name)so the dedupe could never fire;the set and its per-iteration
format!/contains/insertcalls are removed.Comparer::compare_routinesandComparer::compare_routines_and_viewsno longerclone every
Routine(each carries the fullsource_codestring). The drop path now holdsroutines_to_drop: Vec<&Routine>borrowing intoself.from.routines. The create/update path —previously forced to clone because
process_target_routinetook&mut selfand soconflicted with any borrow into
self.from/self.to— is unblocked by refactoring thatmethod into a free associated function
Self::emit_routine_diff(&mut script, use_drop, routine, from_routine). Disjoint-field splitborrows now allow
&mut self.scriptto coexistwith
&Routineborrows out of the dump fields,removing the per-emit clones. Same pattern as the
pre-existing
Self::emit_drop.Performance:
Dump::processno longer materializes the entireserialized dump as a
Stringbefore handing it tothe zip writer. The JSON payload is now streamed
into
ZipWriterviaserde_json::to_writerthrough a 256 KiB
BufWriter, bounding peakmemory at the buffer plus zlib's internal state
rather than ~2x the uncompressed dump size. The
BufWriteris required for speed:to_writeremits one write per JSON token, and feeding those
straight into the deflate stream paid a per-call
cost on every one (an early unbuffered version of
this change made dumps roughly 10x slower). The
write path is exposed as
Dump::write_to_file,mirroring the existing
Dump::read_from_fileandmaking the round-trip directly testable.
Tests:
builder includes the
pg_classclassoid filter:build_tables_standalone_query_filters_by_pg_class,build_regular_views_query_filters_by_pg_class,build_materialized_views_query_filters_by_pg_class,build_view_column_comments_query_filters_by_pg_class,build_sequences_standalone_query_filters_by_pg_class,and
build_indexes_bulk_query_filters_by_pg_class.write_to_file_round_trips_via_read_from_filebuilds a Dump with schemas, extensions, tables,
views, sequences, and routines, writes it via the
new streaming path, reads it back, and asserts
every collection size and a few content fields
match.
compare_column_grants_dispatches_per_column_acl_correctlybuilds a table with three columns whose effective
from_acldiffers (kept / revoked / newlygranted) and verifies that the per-table
column-ACL HashMap dispatches each column to the
right diff outcome — guarding against off-by-one
mistakes that single-column tests would miss.
mark_serial_columns_handles_dotted_identifier_namesdrives the
mark_serial_columnspath with aschema, table, and column name that all contain a
literal
., asserting that the new tuple keystill locates the target column. The pre-fix
splitn(3, '.')parser would have failed on thisinput.