fix(sanitize): anchor sandbox-comment regex to avoid dropping real rows - #110
Merged
Merged
Conversation
The sandbox-comment sanitizer regex (`.*999999.*sandbox.*`) matched any line containing both substrings anywhere, not just the mysqlbinlog "sandbox mode" artifact comment it was meant to strip. Since mysqldump uses --extended-insert by default, a whole table's rows can share one dump line, so real data containing "999999" (e.g. an ID) and "sandbox" (e.g. a PayPal/Braintree environment setting) anywhere on that line caused SanitizeSQLDump to silently drop the entire INSERT statement. This explained why `govard sync --db` (which sanitizes the stream) imported less data than `db dump` + `db import -f` (which doesn't sanitize file-based imports). Anchor the pattern to the actual `/*!999999 ... */` comment delimiters so it only matches the intended artifact. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sync --dbanddb import --stream-dbwere silently dropping rows becauseSanitizeSQLDump's sandbox-comment regex (.*999999.*sandbox.*) matched anydump line containing both substrings, not just the specific mysqlbinlog artifact
comment (
/*!999999\- enable the sandbox mode */) it was meant to strip.mysqldump --extended-insert(the default here) puts a whole table's rowson one line, real data containing
"999999"(e.g. an ID) and"sandbox"(e.g. a PayPal/Braintree environment setting) anywhere on that line caused the
entire INSERT statement to be dropped with no error.
db import -f <file>doesn't sanitize file-based imports at all, which is why itimported full data while the streaming paths (
sync --db,db import --stream-db)came up short — same root symptom, same root cause.
sqlSandboxPatternto the real/*!999999 ... */comment delimiters, andchanged the fast-path substring gate from
"sandbox"to the distinctive"/*!999999"marker.Closes #109
Test plan
--extended-insert-style line (tests/sql_sanitize_test.go)format (
/*!999999\- enable the sandbox mode */) and confirmed it's stilldropped
go build ./...,go vet ./...,go test ./...all passgofmt -s -l .shows no drift on changed files