diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 1fd612f..806a615 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -20,7 +20,7 @@ jobs: - uses: actions/checkout@v6 - uses: actions/setup-go@v6 with: - go-version: '1.21' + go-version: '1.24' cache: false - name: golangci-lint uses: golangci/golangci-lint-action@v9 diff --git a/.github/workflows/run-tests.yml b/.github/workflows/run-tests.yml index 2a3e4c4..fd728b0 100644 --- a/.github/workflows/run-tests.yml +++ b/.github/workflows/run-tests.yml @@ -45,7 +45,7 @@ jobs: - name: Set up Go uses: actions/setup-go@v3 with: - go-version: 1.21 + go-version: '1.24' - name: Debug environment variables run: printenv | sort @@ -67,7 +67,7 @@ jobs: - name: Set up Go uses: actions/setup-go@v4 with: - go-version: 1.21 + go-version: '1.24' - name: Install dependencies run: go mod tidy diff --git a/CLAUDE.md b/CLAUDE.md index 2cdbe62..f537cef 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -62,6 +62,6 @@ Handler → Service → Repository → DB **Request/Response** (`internal/request/`) — `GetRequestParams` parses pagination, filtering, and sorting from query params. Filters are declared per-handler as `[]req.FilterRule`; sortable fields are whitelisted. Response helpers (`GetSingleItemResp`, `GetListResp`, `GetValidateErrResp`, etc.) standardise the JSON envelope. -**Migrations** (`migrations/`) — raw SQL files named `_.up.sql` / `.down.sql`. Multiple statements are separated by `------------------` on its own line. Tracking is via a `migrations` table in the DB. +**Migrations** (`migrations/`) — raw SQL files named `_.up.sql` / `.down.sql` where timestamp is `YYYYMMDDHHmmss` (14 digits, no separators). Multiple statements per file are separated by `;`. Tracking is via a `schema_migrations` table managed by [golang-migrate/migrate](https://github.com/golang-migrate/migrate). If a migration fails mid-run the DB is marked dirty; resolve with `migrate force ` before retrying. **Scaffolding** (`internal/scaffold/`, `cmd/scaffold/`) — reads a live DB table schema and generates the four layers above. Output paths are configured in `cmd/scaffold/config.json`. diff --git a/Makefile b/Makefile index 93a67bb..0a95941 100644 --- a/Makefile +++ b/Makefile @@ -15,7 +15,7 @@ lint: .PHONY: test test: - go test ./... -cover + go test -p 1 ./... -cover SHELL := /bin/bash docker_up: diff --git a/README.md b/README.md index e45a96f..bfc97be 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,8 @@ The basic requirements from a RESTful API framework are: - ~~easy filtering in collection requests~~ - ~~sorting and ordering in collection requests~~ - ~~CRUD scaffolding~~ -- user model, login and authentication, etc... -- ci/cd +- ~~user model, login and authentication, etc...~~ +- ~~ci/cd~~ - documentation Good to have: @@ -29,12 +29,43 @@ Good to have: - `./internal/models/` # Contain data models - `./internal/service/` # Service layer that contains business logic of each model/endpoints - `./internal/repository/` # Contains data retrieval functions -- `./internal/migrations/` # Contains database migration files +- `./migrations/` # Contains database migration files - `./internal/middleware/` # Contains API middleware - `./config.json` # Application configuration -## Migration -Migration will run SQL commands to update the database. It has its own [README](README_MIGRATION.md). +## Migrations + +Migration files live in `./migrations/` and are named `_.up.sql` / `_.down.sql` where the timestamp is 14 digits (`YYYYMMDDHHmmss`). Multiple SQL statements in one file are separated by `;`. Run state is tracked in a `schema_migrations` table managed by [golang-migrate/migrate](https://github.com/golang-migrate/migrate). + +**Create a new migration:** +```bash +go run ./cmd/migrate/migrate.go -action create -name +``` + +**Run all pending migrations:** +```bash +go run ./cmd/migrate/migrate.go -action up +``` + +**Run a specific number of migrations:** +```bash +go run ./cmd/migrate/migrate.go -action up -number 1 +``` + +**Rollback one migration:** +```bash +go run ./cmd/migrate/migrate.go -action down +``` + +**Rollback a specific number of migrations:** +```bash +go run ./cmd/migrate/migrate.go -action down -number 2 +``` + +If a migration fails mid-run the database is marked dirty. Resolve with: +```bash +migrate force +``` ## Scaffolding Scaffolding is a tool that will read the table definition in the database and create the CRUD routes, handlers, service and diff --git a/README_MIGRATION.md b/README_MIGRATION.md deleted file mode 100644 index 50f49dd..0000000 --- a/README_MIGRATION.md +++ /dev/null @@ -1,41 +0,0 @@ -## Golang Database Migration Tool - -This is a tool that is designed to automate database migrations. It is based on -the design of migrations of Yii2 framework. - -### Usage -`go run a -action <-name|-number> XXXX` - -Create new migration file: -`go run ./cmd/migrate/migrate.go -action create -name add_user_table` - -Migrate pending migrations: -`go run ./cmd/migrate/migrate.go -action up -number 1` // The number is optional. Will run all migrations if -value for number is not included. - -Rollback past migrations: -`go run ./cmd/migrate/migrate.go -action down -number 1` // number is optional. Will only rollback one migration if no number value is included. - -The library keeps track of which migrations have been run by creating and populating a table named `migrations`. - -### Getting Started -The library can be included into your application with the following example. A database connection and -path to the migration files need to be passed in. - -An example of usage can be seen in the file `cmd/main.go` which can be modified and added into your own project. - -The script can be run with `go run cmd/main.go -action create -name new_user_table` - -Migration files will be saved to the location defined by the second parameter in call to `migrate.New()`. Two files will be created. -One for migration up and another for down. Once created, you should edit the files with the raw SQL queries. Multiple -queries should be separated with a string of `------------------` -on a line of its own. - -### Additional - -The migrations are run inside a transaction and will attempt to rollback all statements if one fails. However, some -MySQL statements force a commit, preventing this from working as intended with MySQL/MariaDB databases. - -### Todo -- Add tests -- split the files up into separate packages. diff --git a/ci/.golangci-lint.yml b/ci/.golangci-lint.yml index c47b7b4..3629d19 100644 --- a/ci/.golangci-lint.yml +++ b/ci/.golangci-lint.yml @@ -58,6 +58,11 @@ linters: gosec: excludes: - G101 + exclusions: + rules: + - path: "_test.go" + linters: + - goconst formatters: enable: diff --git a/cmd/migrate/migrate.go b/cmd/migrate/migrate.go index ed6b1e0..a45ab2d 100644 --- a/cmd/migrate/migrate.go +++ b/cmd/migrate/migrate.go @@ -2,17 +2,19 @@ package main import ( + "database/sql" "flag" "fmt" "log" "os" + "time" _ "github.com/go-sql-driver/mysql" + gm "github.com/golang-migrate/migrate/v4" + "github.com/golang-migrate/migrate/v4/database/mysql" + _ "github.com/golang-migrate/migrate/v4/source/file" "github.com/doublehops/dh-go-framework/internal/config" - "github.com/doublehops/dh-go-framework/internal/db" - "github.com/doublehops/dh-go-framework/internal/logga" - migrate "github.com/doublehops/dh-go-framework/internal/migration" ) func main() { @@ -23,68 +25,121 @@ func main() { } func run() error { - var args migrate.Action + action := flag.String("action", "", "action: up, down, create") + name := flag.String("name", "", "migration name (required for create)") + number := flag.Int("number", 0, "number of migrations to run (0 = all for up, 1 for down)") + configFile := flag.String("config", "config.json", "config file") + flag.Parse() - flag.StringVar(&args.Action, "action", "", "the intended action") - flag.StringVar(&args.Name, "name", "", "the name of the migration") - flag.IntVar(&args.Number, "number", 0, "The number of migrations to run") + if *action == "" { + fmt.Fprintln(os.Stderr, "Usage: -action [up|down|create] [-number N] [-name ]") + os.Exit(1) + } - configFile := flag.String("config", "config.json", "Config file to use") - flag.Parse() + if *action == "create" { + if *name == "" { + fmt.Fprintln(os.Stderr, "-name is required for the create action") + os.Exit(1) + } - args = setFlags(args) + return createMigration(*name) + } - // Setup config. cfg, err := config.New(*configFile) if err != nil { - return fmt.Errorf("error starting main. %s", err.Error()) + return fmt.Errorf("error loading config: %w", err) } - // Setup logger. - l, err := logga.New(&cfg.Logging) + m, cleanup, err := newMigrator(cfg) if err != nil { - return fmt.Errorf("error configuring logger. %s", err.Error()) + return err + } + + defer cleanup() + + return runAction(m, *action, *number) +} + +func runAction(m *gm.Migrate, action string, number int) error { + version, dirty, _ := m.Version() //nolint:errcheck + log.Printf("current version: %d (dirty: %v)", version, dirty) + + var err error + + switch action { + case "up": + if number == 0 { + err = m.Up() + } else { + err = m.Steps(number) + } + case "down": + n := number + if n == 0 { + n = 1 + } + + err = m.Steps(-n) + default: + return fmt.Errorf("unknown action %q — use up, down, or create", action) } - // Setup db connection. - DB, err := db.New(l, cfg.DB) + if err == gm.ErrNoChange { + log.Println("no migrations to run") + + return nil + } + + return err +} + +func newMigrator(cfg *config.Config) (*gm.Migrate, func(), error) { + dsn := fmt.Sprintf("%s:%s@(%s:3306)/%s?parseTime=true&multiStatements=true", + cfg.DB.User, cfg.DB.Pass, cfg.DB.Host, cfg.DB.Name) + + sqlDB, err := sql.Open("mysql", dsn) if err != nil { - return fmt.Errorf("error creating database connection. %s", err.Error()) + return nil, nil, fmt.Errorf("error opening database: %w", err) } - dir, err := os.Getwd() + driver, err := mysql.WithInstance(sqlDB, &mysql.Config{}) if err != nil { - return fmt.Errorf("there was an error with os.Getwd(). %s", err.Error()) + _ = sqlDB.Close() + return nil, nil, fmt.Errorf("error creating mysql driver: %w", err) } - args.Path = dir + "/migrations" - args.DB = DB + dir, err := os.Getwd() + if err != nil { + _ = sqlDB.Close() + return nil, nil, fmt.Errorf("error getting working directory: %w", err) + } - err = args.Migrate() + m, err := gm.NewWithDatabaseInstance("file://"+dir+"/migrations", "mysql", driver) if err != nil { - return fmt.Errorf("there was an error initialising migration. %s", err.Error()) + _ = sqlDB.Close() + return nil, nil, fmt.Errorf("error creating migrator: %w", err) } - return nil + return m, func() { _ = sqlDB.Close() }, nil } -// setFlags will check that the flags received are valid and assign default ones if not supplied. -func setFlags(args migrate.Action) migrate.Action { - if found := args.IsValidAction(args.Action); !found { - args.PrintHelp() +func createMigration(name string) error { + dir, err := os.Getwd() + if err != nil { + return fmt.Errorf("error getting working directory: %w", err) } - if args.Action == "create" && args.Name == "" { - args.PrintHelp() - } + ts := time.Now().Format("20060102150405") + base := fmt.Sprintf("%s/migrations/%s_%s", dir, ts, name) - if args.Action == "up" && args.Number == 0 { - args.Number = 9999 // run them all if none defined. + if err := os.WriteFile(base+".up.sql", []byte("-- Write your up migration SQL here\n"), 0o600); err != nil { + return fmt.Errorf("error creating up migration file: %w", err) } - - if args.Action == "down" && args.Number == 0 { - args.Number = 1 // run just one if none defined. + if err := os.WriteFile(base+".down.sql", []byte("-- Write your rollback SQL here\n"), 0o600); err != nil { + return fmt.Errorf("error creating down migration file: %w", err) } - return args + log.Printf("created:\n %s.up.sql\n %s.down.sql", base, base) + + return nil } diff --git a/go.mod b/go.mod index 102e9f5..b1204d4 100644 --- a/go.mod +++ b/go.mod @@ -1,27 +1,27 @@ module github.com/doublehops/dh-go-framework -go 1.21 +go 1.24.0 // @todo - this should be removed when dhapi is pushed to Github. replace github.com/doublehops/dhapi => /home/b/workspace/dhapi-2 require ( github.com/doublehops/go-common v0.0.0-20230910011642-8556bd635e3f - github.com/doublehops/go-migration v0.0.3 github.com/go-sql-driver/mysql v1.8.1 + github.com/golang-migrate/migrate/v4 v4.19.1 github.com/jinzhu/copier v0.4.0 github.com/jmoiron/sqlx v1.4.0 github.com/joho/godotenv v1.5.1 github.com/julienschmidt/httprouter v1.3.0 github.com/mythrnr/httprouter-group v0.9.1 - github.com/stretchr/testify v1.9.0 - golang.org/x/crypto v0.27.0 - golang.org/x/text v0.18.0 + github.com/stretchr/testify v1.10.0 + golang.org/x/crypto v0.45.0 + golang.org/x/text v0.31.0 ) require ( filippo.io/edwards25519 v1.1.0 // indirect - github.com/davecgh/go-spew v1.1.1 // indirect - github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect + github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect -) \ No newline at end of file +) diff --git a/go.sum b/go.sum index 8c9dea9..4ca392e 100644 --- a/go.sum +++ b/go.sum @@ -1,13 +1,39 @@ filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA= filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4= -github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= -github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0= +github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= +github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= +github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= +github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= +github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= +github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= +github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dhui/dktest v0.4.6 h1:+DPKyScKSEp3VLtbMDHcUq6V5Lm5zfZZVb0Sk7Ahom4= +github.com/dhui/dktest v0.4.6/go.mod h1:JHTSYDtKkvFNFHJKqCzVzqXecyv+tKt8EzceOmQOgbU= +github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= +github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= +github.com/docker/docker v28.3.3+incompatible h1:Dypm25kh4rmk49v1eiVbsAtpAsYURjYkaKubwuBdxEI= +github.com/docker/docker v28.3.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= +github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c= +github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc= +github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= +github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/doublehops/go-common v0.0.0-20230910011642-8556bd635e3f h1:OjNpxc9tgEFEF3u6UYqsFJ7gNGUZF2u0LPI1vWCi8qU= github.com/doublehops/go-common v0.0.0-20230910011642-8556bd635e3f/go.mod h1:UaQiCwB7vJZfppOfn6sobNIjDcxqtYfB86f/YV6A7Cw= -github.com/doublehops/go-migration v0.0.3 h1:H/f10b7CKs5tZJYw7tZXePnXRkwhNeF/8z1Gl2PJGuM= -github.com/doublehops/go-migration v0.0.3/go.mod h1:Afh3EwDNAuSn9dKXvoerO1LN7RIfG3aE5eC5N7RhIQY= +github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= +github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= +github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= +github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= github.com/go-sql-driver/mysql v1.8.1 h1:LedoTUt/eveggdHS9qUFC1EFSa8bU2+1pZjSRpvNJ1Y= github.com/go-sql-driver/mysql v1.8.1/go.mod h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg= +github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= +github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/golang-migrate/migrate/v4 v4.19.1 h1:OCyb44lFuQfYXYLx1SCxPZQGU7mcaZ7gH9yH4jSFbBA= +github.com/golang-migrate/migrate/v4 v4.19.1/go.mod h1:CTcgfjxhaUtsLipnLoQRWCrjYXycRz/g5+RWDuYgPrE= github.com/jinzhu/copier v0.4.0 h1:w3ciUoD19shMCRargcpm0cm91ytaBhDvuRpz1ODO/U8= github.com/jinzhu/copier v0.4.0/go.mod h1:DfbEm0FYsaqBcKcFuvmOZb218JkPGtvSHsKg8S8hyyg= github.com/jmoiron/sqlx v1.4.0 h1:1PLqN7S1UYp5t4SrVVnt4nUVNemrDAtxlulVe+Qgm3o= @@ -20,17 +46,41 @@ github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw= github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o= github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o4kU= github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y= +github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0= +github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo= +github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0= +github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y= +github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A= +github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc= github.com/mythrnr/httprouter-group v0.9.1 h1:sjAYahYhd4WlV+CpBTFaNS08qBjLCCLks8aQHaDotLs= github.com/mythrnr/httprouter-group v0.9.1/go.mod h1:wotPIQ25+D2etonS126LG2Yby+nD6koTr3IRRoxzwt4= -github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= -github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= -github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= -golang.org/x/crypto v0.27.0 h1:GXm2NjJrPaiv/h1tb2UH8QfgC/hOf/+z0p6PT8o1w7A= -golang.org/x/crypto v0.27.0/go.mod h1:1Xngt8kV6Dvbssa53Ziq6Eqn0HqbZi5Z6R0ZpwQzt70= -golang.org/x/text v0.18.0 h1:XvMDiNzPAl0jr17s6W9lcaIhGUfUORdGCNsuLmPG224= -golang.org/x/text v0.18.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY= +github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= +github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= +github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug= +github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA= +go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q= +go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ= +go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I= +go.opentelemetry.io/otel/metric v1.37.0 h1:mvwbQS5m0tbmqML4NqK+e3aDiO02vsf/WgbsdpcPoZE= +go.opentelemetry.io/otel/metric v1.37.0/go.mod h1:04wGrZurHYKOc+RKeye86GwKiTb9FKm1WHtO+4EVr2E= +go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mxVK7z4= +go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0= +golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= +golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= +golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc= +golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= +golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= -gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= \ No newline at end of file +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/internal/config/config.go b/internal/config/config.go index 8d48eec..d50ff71 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -115,7 +115,7 @@ func loadEnv() error { func resolveEnvInStruct(s interface{}) { val := reflect.ValueOf(s) - if val.Kind() != reflect.Ptr || val.Elem().Kind() != reflect.Struct { + if val.Kind() != reflect.Pointer || val.Elem().Kind() != reflect.Struct { return } val = val.Elem() diff --git a/internal/logga/logga.go b/internal/logga/logga.go index bc8b0a1..8b8a34c 100644 --- a/internal/logga/logga.go +++ b/internal/logga/logga.go @@ -10,6 +10,16 @@ import ( "github.com/doublehops/dh-go-framework/test/testbuffer" ) +const ( + logFormatJSON = "json" + logWriterStdout = "stdout" + logWriterTest = "testwriter" + logLevelDebug = "DEBUG" + logLevelInfo = "INFO" + logLevelWarn = "WARN" + logLevelError = "ERROR" +) + // ErrInvalidLogWriter and ErrInvalidLogLevelValue are errors returned when the logger config is invalid. var ( ErrInvalidLogWriter = errors.New("a valid writer was not defined in configuration") @@ -42,7 +52,7 @@ func New(cfg *config.Logging) (*Logga, error) { var logger *slog.Logger switch cfg.OutputFormat { - case "json": + case logFormatJSON: logger = slog.New(slog.NewJSONHandler(writer, &slog.HandlerOptions{Level: logLevel})) case "text": logger = slog.New(slog.NewTextHandler(writer, &slog.HandlerOptions{Level: logLevel})) @@ -55,11 +65,11 @@ func New(cfg *config.Logging) (*Logga, error) { func getWriterFromConfig(configuredWriter string) (io.Writer, error) { switch configuredWriter { - case "stdout": + case logWriterStdout: return os.Stdout, nil case "": // Default to stdout if none is defined. return os.Stdout, nil - case "testwriter": // Used for testing. + case logWriterTest: // Used for testing. return testbuffer.TestBuffer{}, nil } @@ -68,13 +78,13 @@ func getWriterFromConfig(configuredWriter string) (io.Writer, error) { func getLogLevelFromConfig(configuredLevel string) (slog.Level, error) { switch configuredLevel { - case "DEBUG": + case logLevelDebug: return slog.LevelDebug, nil - case "INFO": + case logLevelInfo: return slog.LevelInfo, nil - case "WARN": + case logLevelWarn: return slog.LevelWarn, nil - case "ERROR": + case logLevelError: return slog.LevelError, nil } diff --git a/internal/migration/Makefile b/internal/migration/Makefile deleted file mode 100644 index d30c2f7..0000000 --- a/internal/migration/Makefile +++ /dev/null @@ -1,12 +0,0 @@ - -.PHONY: gofmt -gofmt: ## Run gofumpt over the codebase. gofumpt must be installed and in your path. - gofumpt -l -w . - -.PHONY: lint -lint: ## Run golangci-lint. golangci-lint must be installed and in your path. - golangci-lint run --modules-download-mode vendor - -.PHONY: test -test: - go test ./... diff --git a/internal/migration/README.md b/internal/migration/README.md deleted file mode 100644 index d018c59..0000000 --- a/internal/migration/README.md +++ /dev/null @@ -1,41 +0,0 @@ -## Golang Database Migration Tool - -This is a tool that is designed to automate database migrations. It is based on -the design of migrations of Yii2 framework. - -### Usage -` -action <-name|-number> XXXX` - -Create new migration file: -`./migrate -action create -name add_user_table` - -Migrate pending migrations: -`./migrate -action up -number 1` // The number is optional. Will run all migrations if -value for number is not included. - -Rollback past migrations: -`./migrate -action down -number 1` // number is optional. Will only rollback one migration if no number value is included. - -The library keeps track of which migrations have been run by creating and populating a table named `migrations`. - -### Getting Started -The library can be included into your application with the following example. A database connection and -path to the migration files need to be passed in. - -An example of usage can be seen in the file `cmd/main.go` which can be modified and added into your own project. - -The script can be ran with `go run cmd/main.go -action create -name new_user_table` - -Migration files will be saved to the location defined by the second parameter in call to `migrate.New()`. Two files will be created. -One for migration up and another for down. Once created, you should edit the files with the raw SQL queries. Multiple -queries should be separated with a string of `------------------` -on a line of its own. - -### Additional - -The migrations are run inside a transaction and will attempt to rollback all statements if one fails. However, some -MySQL statements force a commit, preventing this from working as intended with MySQL/MariaDB databases. - -### Todo -- Add tests -- split the files up into separate packages. diff --git a/internal/migration/actioncreate.go b/internal/migration/actioncreate.go deleted file mode 100644 index a93795a..0000000 --- a/internal/migration/actioncreate.go +++ /dev/null @@ -1,51 +0,0 @@ -// Package gomigration provides database migration management including creating, applying, and rolling back SQL migrations. -package gomigration - -import ( - "fmt" - "os" - "time" - - "github.com/doublehops/go-migration/helpers" -) - -// CreateMigration will copy template file into new fil -func (a *Action) CreateMigration(path string) error { - currentTime := time.Now() - curTime := currentTime.Format("20060102_150405_") - name := curTime + a.Name - upName := name + ".up.sql" - downName := name + ".down.sql" - upPath := path + "/" + upName - downPath := path + "/" + downName - - separatorMessage := fmt.Sprintf("-- You need to separate multiple queries with this dotted line: %s\n\n", QuerySeparator) - - exampleUp := `CREATE TABLE news ( - id INT(11) NOT NULL, - currency_id INT(11) NOT NULL, - created_at DATETIME, - updated_at DATETIME, - deleted_at DATETIME, - PRIMARY KEY (id), - FOREIGN KEY (currency_id) REFERENCES currency(id)); - ` - - exampleUp = separatorMessage + exampleUp - exampleDown := separatorMessage + "DROP TABLE news;\n\n" - - err := os.WriteFile(upPath, []byte(exampleUp), 0o644) //nolint:gosec - if err != nil { - return fmt.Errorf("unable to write template file: %s. %s", upPath, err) - } - - err = os.WriteFile(downPath, []byte(exampleDown), 0o644) //nolint:gosec - if err != nil { - return fmt.Errorf("unable to write template file: %s. %s", downPath, err) - } - - helpers.PrintMsg("Migration file created: " + upPath + "\n") - helpers.PrintMsg("Migration file created: " + downPath + "\n") - - return nil -} diff --git a/internal/migration/actiondown.go b/internal/migration/actiondown.go deleted file mode 100644 index 566ae82..0000000 --- a/internal/migration/actiondown.go +++ /dev/null @@ -1,52 +0,0 @@ -package gomigration - -import ( - "fmt" - "strings" - - "github.com/doublehops/go-migration/helpers" -) - -// MigrateDown will rollback migration/s. -func (a *Action) MigrateDown(migrationFiles []File) error { - var err error - - for _, file := range migrationFiles { - err = a.processFileDown(file) - if err != nil { - return err - } - } - - return nil -} - -// processFileDown will process the down queries in the given file. It will attempt to rollback when there is -// and error in one of the queries. -func (a *Action) processFileDown(file File) error { - tx, err := a.DB.Begin() - if err != nil { - return fmt.Errorf("error starting transaction. %w", err) - } - defer tx.Rollback() // nolint - - helpers.PrintMsg(fmt.Sprintf("Migrating down queries from: %s", file.Filename)) - for _, q := range file.Queries { - _, err = tx.Exec(q) - if err != nil { - return fmt.Errorf("\nthere was an error executing query. File: %s; Error: %s", file.Filename, err) - } - } - filename := strings.Replace(file.Filename, ".down.sql", ".up.sql", 1) - _, err = tx.Exec(RemoveMigrationRecordFromTableSQL, filename) - if err != nil { - return fmt.Errorf("unable to remove from migration table with newly ran migration record. %w", err) - } - helpers.PrintMsg(" - Success\n") - - if err = tx.Commit(); err != nil { - return fmt.Errorf("there was an error committing transaction. %w", err) - } - - return nil -} diff --git a/internal/migration/actionup.go b/internal/migration/actionup.go deleted file mode 100644 index a8ccb16..0000000 --- a/internal/migration/actionup.go +++ /dev/null @@ -1,51 +0,0 @@ -package gomigration - -import ( - "fmt" - - "github.com/doublehops/go-migration/helpers" -) - -// MigrateUp will run new migration/s. -func (a *Action) MigrateUp(migrationFiles []File) error { - var err error - - for _, file := range migrationFiles { - err = a.processFileUp(file) - if err != nil { - return err - } - } - - return nil -} - -// processFile will process the queries in the given file. It will attempt to rollback when there is -// and error in one of the queries. -func (a *Action) processFileUp(file File) error { - tx, err := a.DB.Begin() - if err != nil { - return fmt.Errorf("error starting transaction. %w", err) - } - defer tx.Rollback() // nolint - - helpers.PrintMsg(fmt.Sprintf("Migrating queries from: %s", file.Filename)) - for _, query := range file.Queries { - _, err = tx.Exec(query) - if err != nil { - return fmt.Errorf("\nthere was an error executing query. File: %s; query; %s; Error: %s", file.Filename, file.Queries, err) - } - } - - _, err = tx.Exec(InsertMigrationRecordIntoTableSQL, file.Filename) - if err != nil { - return fmt.Errorf("unable to update migration table with newly ran migration record. %w", err) - } - helpers.PrintMsg(" - Success\n") - - if err = tx.Commit(); err != nil { - return fmt.Errorf("there was an error committing transaction. %w", err) - } - - return nil -} diff --git a/internal/migration/files.go b/internal/migration/files.go deleted file mode 100644 index 6db230e..0000000 --- a/internal/migration/files.go +++ /dev/null @@ -1,133 +0,0 @@ -package gomigration - -import ( - "fmt" - "os" - "sort" - "strings" -) - -// listMigrationFiles will get migration files from the configured path. -func (a *Action) listMigrationFiles() ([]string, error) { - fileFilter := "." + a.Action + ".sql" - - var files []string - f, err := os.ReadDir(a.Path) - if err != nil { - return files, fmt.Errorf("unable to list migration files. %w", err) - } - - for _, file := range f { - if strings.Contains(file.Name(), fileFilter) { - files = append(files, file.Name()) - } - } - - return files, nil -} - -// getPendingMigrationFiles will loop through all migration files and return the ones that haven't been run yet. -func (a *Action) getPendingMigrationFiles() ([]string, error) { - var pendingFiles []string - foundLastRan := false - - lastRanMigration, err := a.getLatestRanMigration() - if err != nil { - return pendingFiles, err - } - allFiles, err := a.listMigrationFiles() - if err != nil { - return pendingFiles, err - } - - if lastRanMigration == "" { // No migrations have run yet. - foundLastRan = true // If no migrations have previously ran, set found as true to start from first file. - } - - i := 0 - for _, file := range allFiles { - if i == a.Number { - break - } - if file == lastRanMigration { - foundLastRan = true - - continue - } - if !foundLastRan { - continue - } - - pendingFiles = append(pendingFiles, file) - i++ - } - - return pendingFiles, nil -} - -// getPreviouslyMigratedFiles will loop through all migration files and return the ones that have already been run. -func (a *Action) getMigrationFilesToRollBack() ([]string, error) { - var migrationsToRollBack []string - foundLastRan := false - - lastRanMigration, err := a.getLatestRanMigration() - if err != nil { - return migrationsToRollBack, err - } - allFiles, err := a.listMigrationFiles() - if err != nil { - return migrationsToRollBack, err - } - - sort.Sort(sort.Reverse(sort.StringSlice(allFiles))) - - if lastRanMigration == "" { // No migrations have run yet. - return migrationsToRollBack, nil - } - - lastRanMigrationShortName := TrimExtension(lastRanMigration) - - i := 0 - for _, file := range allFiles { - shortFileName := TrimExtension(file) - if i == a.Number { - break - } - if shortFileName == lastRanMigrationShortName { - foundLastRan = true - migrationsToRollBack = append(migrationsToRollBack, file) - i++ - - continue - } - if !foundLastRan { - continue - } - - migrationsToRollBack = append(migrationsToRollBack, file) - i++ - } - - return migrationsToRollBack, nil -} - -// parseMigrations will iterate through the files and unmarshal the JSON and add to the files slice. -func (a *Action) parseMigrations(filesToParse []string) ([]File, error) { - var files []File - for _, file := range filesToParse { - - thisFile := File{Filename: file} - data, err := os.ReadFile(a.Path + "/" + file) //nolint:gosec - if err != nil { - return files, fmt.Errorf("unable to read file: %s. %s", file, err) - } - - queries := strings.Split(string(data), QuerySeparator) - - thisFile.Queries = queries - - files = append(files, thisFile) - } - - return files, nil -} diff --git a/internal/migration/helpers/helpers.go b/internal/migration/helpers/helpers.go deleted file mode 100644 index e9cdae0..0000000 --- a/internal/migration/helpers/helpers.go +++ /dev/null @@ -1,9 +0,0 @@ -// Package helpers provides utility functions for the migration package. -package helpers - -import "os" - -// PrintMsg writes a message to stderr. -func PrintMsg(msg string) { - _, _ = os.Stderr.WriteString(msg) -} diff --git a/internal/migration/history.go b/internal/migration/history.go deleted file mode 100644 index c79ba79..0000000 --- a/internal/migration/history.go +++ /dev/null @@ -1,49 +0,0 @@ -package gomigration - -import ( - "database/sql" - "fmt" -) - -// MigrationRecord represents a row in the migrations tracking table. -type MigrationRecord struct { - ID int - Filename string - CreatedAt string -} - -// getLatestRanMigration will find the last processed migration. -func (a *Action) getLatestRanMigration() (string, error) { - var record MigrationRecord - rows, err := a.DB.Query(GetLatestMigrationSQL) - if err != nil { - if err == sql.ErrNoRows { - return "", nil - } - } - defer func() { - _ = rows.Close() - _ = rows.Err() - }() - if rows == nil { - return "", nil - } - - for rows.Next() { - if err = rows.Scan(&record.ID, &record.Filename, &record.CreatedAt); err != nil { - return "", fmt.Errorf("error retrieving rows from migration table. %s", err) - } - } - - return record.Filename, nil -} - -// addMigrationTable will add a `migration` table to the database to track what has been -func (a *Action) addMigrationTable() error { - _, err := a.DB.Exec(CreateMigrationsTable) - if err != nil { - return fmt.Errorf("error creating migrations table. %s", err) - } - - return nil -} diff --git a/internal/migration/migrate.go b/internal/migration/migrate.go deleted file mode 100644 index e097b3c..0000000 --- a/internal/migration/migrate.go +++ /dev/null @@ -1,170 +0,0 @@ -package gomigration - -import ( - "fmt" - "os" - "strings" - - _ "github.com/go-sql-driver/mysql" // nolint:revive - "github.com/jmoiron/sqlx" - - "github.com/doublehops/dh-go-framework/internal/migration/helpers" -) - -// Action holds the configuration for a migration run including direction, number, and DB connection. -type Action struct { - Action string - Number int - Name string - - DB *sqlx.DB - Path string -} - -// File represents a migration file with its parsed SQL queries. -type File struct { - Filename string - Queries []string -} - -// TrimExtension removes the .up.sql or .down.sql suffix from a migration filename. -func TrimExtension(filename string) string { - var str string - - str = strings.Replace(filename, ".up.sql", "", 1) - str = strings.Replace(str, ".down.sql", "", 1) - - return str -} - -// TableList is a slice of Table records used when inspecting database tables. -type TableList []Table - -// Table represents a database table name, used to check for the migrations table. -type Table struct { - Name string -} - -// Migrate runs the migration action (create, up, or down) as specified in a.Action. -// -//nolint:cyclop -func (a *Action) Migrate() error { - var err error - - err = a.ensureMigrationsTableExists() - if err != nil { - return err - } - - if a.Action == "create" { - err = a.CreateMigration(a.Path) - - return err - } - - if a.Action == "up" { - pendingFiles, err := a.getPendingMigrationFiles() - if err != nil { - return err - } - if len(pendingFiles) == 0 { - helpers.PrintMsg("There are no pending migrations\n") - - return nil - } - migrationFiles, err := a.parseMigrations(pendingFiles) - if err != nil { - return err - } - - if err := a.MigrateUp(migrationFiles); err != nil { - return err - } - } - - if a.Action == "down" { - previousFiles, err := a.getMigrationFilesToRollBack() - if err != nil { - return err - } - if len(previousFiles) == 0 { - helpers.PrintMsg("There are no previous migrations to rollback\n") - - return nil - } - migrationFiles, err := a.parseMigrations(previousFiles) - if err != nil { - return err - } - - if err := a.MigrateDown(migrationFiles); err != nil { - return err - } - } - - return nil -} - -// IsValidAction returns true if the key is one of the supported migration actions. -func (a *Action) IsValidAction(key string) bool { - validActions := []string{ - "create", - "up", - "down", - } - - for _, item := range validActions { - if item == key { - return true - } - } - - return false -} - -// PrintHelp prints usage instructions for the migration CLI tool and exits. -func (a *Action) PrintHelp() { - helpMsg := ` -Usage: -action= -number= -Examples: -./main.go -action create -name add_user_table // Will create a new migration file with template. -./main.go -action up -number 1 // number is optional. Will run all migrations if not included. -./main.go -action down -number 1 // number is optional. Will run only one migration if not included. -` - _, _ = os.Stderr.WriteString(helpMsg) - os.Exit(1) -} - -// ensureMigrationsTableExists to create table to track migrations. -func (a *Action) ensureMigrationsTableExists() error { - var tableList TableList - rows, err := a.DB.Query(CheckMigrationsTableExistsSQL) - defer func() { - _ = rows.Close() - _ = rows.Err() - }() - if err != nil { - return fmt.Errorf("++++>>>> Error: %w", err) - } - - for rows.Next() { - var t Table - if err := rows.Scan(&t.Name); err != nil { - return err - } - tableList = append(tableList, t) - } - - for _, tbl := range tableList { - if tbl.Name == "migrations" { // Already exists - return nil - } - } - - err = a.addMigrationTable() - if err != nil { - return err - } - - return nil -} diff --git a/internal/migration/sql.go b/internal/migration/sql.go deleted file mode 100644 index dcd42bd..0000000 --- a/internal/migration/sql.go +++ /dev/null @@ -1,31 +0,0 @@ -package gomigration - -// QuerySeparator is the delimiter used to split multiple SQL statements within a migration file. -const QuerySeparator = "------------------" - -// GetLatestMigrationSQL and related vars are the SQL statements used to manage the migrations table. -var ( - GetLatestMigrationSQL = `SELECT * FROM migrations - ORDER BY id DESC - LIMIT 1 -` - - CreateMigrationsTable = `CREATE TABLE migrations ( - id INT(11) NOT NULL AUTO_INCREMENT, - filename VARCHAR(255), - created_at DATETIME, - PRIMARY KEY(id) -)` - - CheckMigrationsTableExistsSQL = `SHOW TABLES` - - InsertMigrationRecordIntoTableSQL = `INSERT INTO migrations - (filename,created_at) - VALUES - (?,NOW()) -` - - RemoveMigrationRecordFromTableSQL = `DELETE FROM migrations - WHERE filename = ? -` -) diff --git a/internal/request/responses.go b/internal/request/responses.go index 2d69eb4..760563a 100644 --- a/internal/request/responses.go +++ b/internal/request/responses.go @@ -18,6 +18,11 @@ var ( NotFoundMsg = ErrorMessage{"message": "not found"} ) +const ( + errResponseName = "there was an error processing request" + errResponseStatus = "error" +) + // ErrorMessage is a map of field names to error message strings for API error responses. type ErrorMessage map[string]string @@ -31,10 +36,10 @@ func GetSingleItemResp(data interface{}) SingleItemResp { // GeneralErrResp builds a general error response with the provided message and HTTP status code. func GeneralErrResp(msg string, statusCode int) GeneralErrorResp { return GeneralErrorResp{ - Name: "there was an error processing request", + Name: errResponseName, Message: msg, Code: statusCode, - Status: "error", + Status: errResponseStatus, Errors: nil, } } @@ -42,10 +47,10 @@ func GeneralErrResp(msg string, statusCode int) GeneralErrorResp { // ServerErrResp builds a 500 internal server error response. func ServerErrResp(msg string) GeneralErrorResp { return GeneralErrorResp{ - Name: "there was an error processing request", + Name: errResponseName, Message: msg, Code: http.StatusInternalServerError, - Status: "error", + Status: errResponseStatus, Errors: nil, } } @@ -53,10 +58,10 @@ func ServerErrResp(msg string) GeneralErrorResp { // GetNotAuthorisedResp builds a 403 forbidden response. func GetNotAuthorisedResp() GeneralErrorResp { return GeneralErrorResp{ - Name: "there was an error processing request", + Name: errResponseName, Message: ErrNotAuthorised.Error(), Code: http.StatusForbidden, - Status: "error", + Status: errResponseStatus, Errors: nil, } } @@ -67,7 +72,7 @@ func GetNotFoundResp() GeneralErrorResp { Name: ErrRecordNotFound.Error(), Message: ErrRecordNotFound.Error(), Code: http.StatusNotFound, - Status: "error", + Status: errResponseStatus, Errors: nil, } } @@ -92,7 +97,7 @@ func GetValidateErrResp(errors ErrMsgs, errs ...string) GeneralErrorResp { Name: "Validation failed", Message: err, Code: http.StatusBadRequest, - Status: "error", + Status: errResponseStatus, Errors: errors, } } @@ -103,7 +108,7 @@ func UnableToParseResp() GeneralErrorResp { Name: "Parsing error", Message: ErrCouldNotParseRequest.Error(), Code: http.StatusBadRequest, - Status: "error", + Status: errResponseStatus, Errors: nil, } } @@ -114,7 +119,7 @@ func ErrorProcessingRequestResp() GeneralErrorResp { Name: "Parsing error", Message: ErrProcessingRequest.Error(), Code: http.StatusInternalServerError, - Status: "error", + Status: errResponseStatus, Errors: nil, } } diff --git a/internal/testtools/migrate.go b/internal/testtools/migrate.go index fbe1cf7..ddc272c 100644 --- a/internal/testtools/migrate.go +++ b/internal/testtools/migrate.go @@ -1,49 +1,72 @@ package testtools import ( + "database/sql" "fmt" "os" "path/filepath" + _ "github.com/go-sql-driver/mysql" // registers the MySQL driver with database/sql + gm "github.com/golang-migrate/migrate/v4" + "github.com/golang-migrate/migrate/v4/database/mysql" + _ "github.com/golang-migrate/migrate/v4/source/file" // registers the file source driver with golang-migrate + "github.com/doublehops/dh-go-framework/internal/config" - "github.com/doublehops/dh-go-framework/internal/db" - "github.com/doublehops/dh-go-framework/internal/logga" - migrate "github.com/doublehops/dh-go-framework/internal/migration" ) -// RunMigrations runs all pending "up" migrations against the database configured in cfg. -// It locates the migrations directory by walking up from the current working directory -// until it finds the project root (identified by go.mod). +// RunMigrations drops all tables and re-applies every migration so each test +// run starts from a known-clean state. func RunMigrations(cfg *config.Config) error { projectRoot, err := findProjectRoot() if err != nil { return fmt.Errorf("RunMigrations: %w", err) } - logCfg := &config.Logging{ - Writer: "stdout", - LogLevel: "DEBUG", - OutputFormat: "JSON", - } + dsn := fmt.Sprintf("%s:%s@(%s:3306)/%s?parseTime=true&multiStatements=true", + cfg.DB.User, cfg.DB.Pass, cfg.DB.Host, cfg.DB.Name) - l, err := logga.New(logCfg) - if err != nil { - return fmt.Errorf("RunMigrations: failed to create logger: %w", err) + migrationsPath := "file://" + filepath.Join(projectRoot, "migrations") + + // openMigrator opens its own *sql.DB so that m.Close() (which closes the DB) + // does not affect a subsequent migrator instance. + openMigrator := func() (*gm.Migrate, error) { + sqlDB, err := sql.Open("mysql", dsn) + if err != nil { + return nil, fmt.Errorf("failed to open db: %w", err) + } + drv, err := mysql.WithInstance(sqlDB, &mysql.Config{}) + if err != nil { + _ = sqlDB.Close() + return nil, fmt.Errorf("failed to create driver: %w", err) + } + m, err := gm.NewWithDatabaseInstance(migrationsPath, "mysql", drv) + if err != nil { + _ = sqlDB.Close() + return nil, fmt.Errorf("failed to create migrator: %w", err) + } + + return m, nil } - database, err := db.New(l, cfg.DB) + // Drop all tables so the test run starts from scratch. + m, err := openMigrator() if err != nil { - return fmt.Errorf("RunMigrations: failed to connect to database: %w", err) + return fmt.Errorf("RunMigrations: %w", err) } + if err := m.Drop(); err != nil { + _, _ = m.Close() + return fmt.Errorf("RunMigrations: drop failed: %w", err) + } + _, _ = m.Close() - action := &migrate.Action{ - Action: "up", - Number: 9999, - DB: database, - Path: filepath.Join(projectRoot, "migrations"), + // Fresh connection so the driver re-initialises schema_migrations, then apply all migrations. + m, err = openMigrator() + if err != nil { + return fmt.Errorf("RunMigrations: %w", err) } + defer func() { _, _ = m.Close() }() - if err := action.Migrate(); err != nil { + if err := m.Up(); err != nil && err != gm.ErrNoChange { return fmt.Errorf("RunMigrations: migration failed: %w", err) } diff --git a/migrations/20231011_220958_author.down.sql b/migrations/20231011220958_author.down.sql similarity index 100% rename from migrations/20231011_220958_author.down.sql rename to migrations/20231011220958_author.down.sql diff --git a/migrations/20231011_220958_author.up.sql b/migrations/20231011220958_author.up.sql similarity index 100% rename from migrations/20231011_220958_author.up.sql rename to migrations/20231011220958_author.up.sql diff --git a/migrations/20240210172504_add_temp_table.down.sql b/migrations/20240210172504_add_temp_table.down.sql new file mode 100644 index 0000000..40bde01 --- /dev/null +++ b/migrations/20240210172504_add_temp_table.down.sql @@ -0,0 +1 @@ +DROP TABLE my_new_table; diff --git a/migrations/20240210_172504_add_temp_table.up.sql b/migrations/20240210172504_add_temp_table.up.sql similarity index 69% rename from migrations/20240210_172504_add_temp_table.up.sql rename to migrations/20240210172504_add_temp_table.up.sql index 76f9471..5b7d69f 100644 --- a/migrations/20240210_172504_add_temp_table.up.sql +++ b/migrations/20240210172504_add_temp_table.up.sql @@ -1,5 +1,3 @@ --- You need to separate multiple queries with this dotted line: ------------------ - CREATE TABLE my_new_table ( id INT(11) NOT NULL AUTO_INCREMENT, currency_id INT(11), @@ -8,5 +6,4 @@ CREATE TABLE my_new_table ( updated_at DATETIME, deleted_at DATETIME, PRIMARY KEY (id) - ); - \ No newline at end of file +); diff --git a/migrations/20240210_172504_add_temp_table.down.sql b/migrations/20240210_172504_add_temp_table.down.sql deleted file mode 100644 index 31815f7..0000000 --- a/migrations/20240210_172504_add_temp_table.down.sql +++ /dev/null @@ -1,4 +0,0 @@ --- You need to separate multiple queries with this dotted line: ------------------ - -DROP TABLE my_new_table; - diff --git a/migrations/20240923153040_create_user_table.down.sql b/migrations/20240923153040_create_user_table.down.sql new file mode 100644 index 0000000..5c4c72a --- /dev/null +++ b/migrations/20240923153040_create_user_table.down.sql @@ -0,0 +1,2 @@ +DROP TABLE user; +DROP TABLE organisation; diff --git a/migrations/20240923_153040_create_user_table.up.sql b/migrations/20240923153040_create_user_table.up.sql similarity index 90% rename from migrations/20240923_153040_create_user_table.up.sql rename to migrations/20240923153040_create_user_table.up.sql index 282cfc9..47d56f9 100644 --- a/migrations/20240923_153040_create_user_table.up.sql +++ b/migrations/20240923153040_create_user_table.up.sql @@ -7,10 +7,7 @@ CREATE TABLE organisation ( PRIMARY KEY (id) ); ------------------- - -INSERT INTO organisation (name) VALUES ('org1') ------------------- +INSERT INTO organisation (name) VALUES ('org1'); CREATE TABLE user ( id INT(11) NOT NULL AUTO_INCREMENT, diff --git a/migrations/20240923_153040_create_user_table.down.sql b/migrations/20240923_153040_create_user_table.down.sql deleted file mode 100644 index 07d044f..0000000 --- a/migrations/20240923_153040_create_user_table.down.sql +++ /dev/null @@ -1,6 +0,0 @@ --- You need to separate multiple queries with this dotted line: ------------------ - -DROP TABLE user; ------------------- -DROP TABLE organisation; - diff --git a/migrations/20260405123832_user_sessions_table.down.sql b/migrations/20260405123832_user_sessions_table.down.sql new file mode 100644 index 0000000..66ccd87 --- /dev/null +++ b/migrations/20260405123832_user_sessions_table.down.sql @@ -0,0 +1 @@ +DROP TABLE user_session; diff --git a/migrations/20260405_123832_user_sessions_table.up.sql b/migrations/20260405123832_user_sessions_table.up.sql similarity index 65% rename from migrations/20260405_123832_user_sessions_table.up.sql rename to migrations/20260405123832_user_sessions_table.up.sql index 4aab41f..3c2c2b6 100644 --- a/migrations/20260405_123832_user_sessions_table.up.sql +++ b/migrations/20260405123832_user_sessions_table.up.sql @@ -1,5 +1,3 @@ --- You need to separate multiple queries with this dotted line: ------------------ - CREATE TABLE user_session ( id INT(11) NOT NULL AUTO_INCREMENT, user_id INT(11) NOT NULL, @@ -9,5 +7,5 @@ CREATE TABLE user_session ( updated_at DATETIME, deleted_at DATETIME, PRIMARY KEY (id), - FOREIGN KEY (user_id) REFERENCES user(id)); - \ No newline at end of file + FOREIGN KEY (user_id) REFERENCES user(id) +); diff --git a/migrations/20260405_123832_user_sessions_table.down.sql b/migrations/20260405_123832_user_sessions_table.down.sql deleted file mode 100644 index e56394b..0000000 --- a/migrations/20260405_123832_user_sessions_table.down.sql +++ /dev/null @@ -1,4 +0,0 @@ --- You need to separate multiple queries with this dotted line: ------------------ - -DROP TABLE user_session; - diff --git a/vendor/github.com/doublehops/go-migration/.gitignore b/vendor/github.com/doublehops/go-migration/.gitignore deleted file mode 100644 index d24622e..0000000 --- a/vendor/github.com/doublehops/go-migration/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -go.sum -vendor diff --git a/vendor/github.com/doublehops/go-migration/Makefile b/vendor/github.com/doublehops/go-migration/Makefile deleted file mode 100644 index d30c2f7..0000000 --- a/vendor/github.com/doublehops/go-migration/Makefile +++ /dev/null @@ -1,12 +0,0 @@ - -.PHONY: gofmt -gofmt: ## Run gofumpt over the codebase. gofumpt must be installed and in your path. - gofumpt -l -w . - -.PHONY: lint -lint: ## Run golangci-lint. golangci-lint must be installed and in your path. - golangci-lint run --modules-download-mode vendor - -.PHONY: test -test: - go test ./... diff --git a/vendor/github.com/doublehops/go-migration/README.md b/vendor/github.com/doublehops/go-migration/README.md deleted file mode 100644 index d018c59..0000000 --- a/vendor/github.com/doublehops/go-migration/README.md +++ /dev/null @@ -1,41 +0,0 @@ -## Golang Database Migration Tool - -This is a tool that is designed to automate database migrations. It is based on -the design of migrations of Yii2 framework. - -### Usage -` -action <-name|-number> XXXX` - -Create new migration file: -`./migrate -action create -name add_user_table` - -Migrate pending migrations: -`./migrate -action up -number 1` // The number is optional. Will run all migrations if -value for number is not included. - -Rollback past migrations: -`./migrate -action down -number 1` // number is optional. Will only rollback one migration if no number value is included. - -The library keeps track of which migrations have been run by creating and populating a table named `migrations`. - -### Getting Started -The library can be included into your application with the following example. A database connection and -path to the migration files need to be passed in. - -An example of usage can be seen in the file `cmd/main.go` which can be modified and added into your own project. - -The script can be ran with `go run cmd/main.go -action create -name new_user_table` - -Migration files will be saved to the location defined by the second parameter in call to `migrate.New()`. Two files will be created. -One for migration up and another for down. Once created, you should edit the files with the raw SQL queries. Multiple -queries should be separated with a string of `------------------` -on a line of its own. - -### Additional - -The migrations are run inside a transaction and will attempt to rollback all statements if one fails. However, some -MySQL statements force a commit, preventing this from working as intended with MySQL/MariaDB databases. - -### Todo -- Add tests -- split the files up into separate packages. diff --git a/vendor/github.com/doublehops/go-migration/actioncreate.go b/vendor/github.com/doublehops/go-migration/actioncreate.go deleted file mode 100644 index c8bfaf2..0000000 --- a/vendor/github.com/doublehops/go-migration/actioncreate.go +++ /dev/null @@ -1,51 +0,0 @@ -package go_migration - -import ( - "fmt" - "io/ioutil" - "time" - - "github.com/doublehops/go-migration/helpers" -) - -// CreateMigration will copy template file into new fil -func (a *Action) CreateMigration(path string) error { - currentTime := time.Now() - curTime := currentTime.Format("20060102_150405_") - name := curTime + a.Name - upName := name + ".up.sql" - downName := name + ".down.sql" - upPath := path + "/" + upName - downPath := path + "/" + downName - - separatorMessage := fmt.Sprintf("-- You need to separate multiple queries with this dotted line: %s\n\n", QuerySeparator) - - exampleUp := -`CREATE TABLE news ( - id INT(11) NOT NULL, - currency_id INT(11) NOT NULL, - created_at DATETIME, - updated_at DATETIME, - deleted_at DATETIME, - PRIMARY KEY (id), - FOREIGN KEY (currency_id) REFERENCES currency(id)); - ` - - exampleUp = separatorMessage + exampleUp - exampleDown := separatorMessage +"DROP TABLE news;\n\n" - - err := ioutil.WriteFile(upPath, []byte(exampleUp), 0644) - if err != nil { - return fmt.Errorf("unable to write template file: %s. %s", upPath, err) - } - - err = ioutil.WriteFile(downPath, []byte(exampleDown), 0644) - if err != nil { - return fmt.Errorf("unable to write template file: %s. %s", downPath, err) - } - - helpers.PrintMsg("Migration file created: " + upPath + "\n") - helpers.PrintMsg("Migration file created: " + downPath + "\n") - - return nil -} diff --git a/vendor/github.com/doublehops/go-migration/actiondown.go b/vendor/github.com/doublehops/go-migration/actiondown.go deleted file mode 100644 index 6a2fed7..0000000 --- a/vendor/github.com/doublehops/go-migration/actiondown.go +++ /dev/null @@ -1,53 +0,0 @@ -package go_migration - -import ( - "fmt" - "strings" - - "github.com/doublehops/go-migration/helpers" -) - -// MigrateDown will rollback migration/s. -func (a *Action) MigrateDown(migrationFiles []File) error { - var err error - - for _, file := range migrationFiles { - err = a.processFileDown(file) - if err != nil { - return err - } - } - - return nil -} - -// processFileDown will process the down queries in the given file. It will attempt to rollback when there is -// and error in one of the queries. -func (a *Action) processFileDown(file File) error { - - tx, err := a.DB.Begin() - if err != nil { - return fmt.Errorf("error starting transaction. %w", err) - } - defer tx.Rollback() // nolint - - helpers.PrintMsg(fmt.Sprintf("Migrating down queries from: %s", file.Filename)) - for _, q := range file.Queries { - _, err = tx.Exec(q) - if err != nil { - return fmt.Errorf("\nthere was an error executing query. File: %s; Error: %s", file.Filename, err) - } - } - filename := strings.Replace(file.Filename, ".down.sql", ".up.sql", 1) - _, err = tx.Exec(RemoveMigrationRecordFromTableSQL, filename) - if err != nil { - return fmt.Errorf("unable to remove from migration table with newly ran migration record. %w", err) - } - helpers.PrintMsg(" - Success\n") - - if err = tx.Commit(); err != nil { - return fmt.Errorf("there was an error committing transaction. %w", err) - } - - return nil -} \ No newline at end of file diff --git a/vendor/github.com/doublehops/go-migration/actionup.go b/vendor/github.com/doublehops/go-migration/actionup.go deleted file mode 100644 index e81e8cc..0000000 --- a/vendor/github.com/doublehops/go-migration/actionup.go +++ /dev/null @@ -1,51 +0,0 @@ -package go_migration - -import ( - "fmt" - "github.com/doublehops/go-migration/helpers" -) - -// MigrateUp will run new migration/s. -func (a *Action) MigrateUp(migrationFiles []File) error { - var err error - - for _, file := range migrationFiles { - err = a.processFileUp(file) - if err != nil { - return err - } - } - - return nil -} - -// processFile will process the queries in the given file. It will attempt to rollback when there is -// and error in one of the queries. -func (a *Action) processFileUp(file File) error { - - tx, err := a.DB.Begin() - if err != nil { - return fmt.Errorf("error starting transaction. %w", err) - } - defer tx.Rollback() // nolint - - helpers.PrintMsg(fmt.Sprintf("Migrating queries from: %s", file.Filename)) - for _, query := range file.Queries { - _, err = tx.Exec(query) - if err != nil { - return fmt.Errorf("\nthere was an error executing query. File: %s; query; %s; Error: %s", file.Filename, file.Queries, err) - } - } - - _, err = tx.Exec(InsertMigrationRecordIntoTableSQL, file.Filename) - if err != nil { - return fmt.Errorf("unable to update migration table with newly ran migration record. %w\n", err) - } - helpers.PrintMsg(" - Success\n") - - if err = tx.Commit(); err != nil { - return fmt.Errorf("there was an error committing transaction. %w", err) - } - - return nil -} diff --git a/vendor/github.com/doublehops/go-migration/files.go b/vendor/github.com/doublehops/go-migration/files.go deleted file mode 100644 index 9dc41bb..0000000 --- a/vendor/github.com/doublehops/go-migration/files.go +++ /dev/null @@ -1,133 +0,0 @@ -package go_migration - -import ( - "fmt" - "io/ioutil" - "os" - "sort" - "strings" -) - -// listMigrationFiles will get migration files from the configured path. -func (a *Action) listMigrationFiles() ([]string, error) { - - fileFilter := "."+ a.Action +".sql" - - var files []string - f, err := ioutil.ReadDir(a.Path) - if err != nil { - return files, fmt.Errorf("unable to list migration files. %w", err) - } - - for _, file := range f { - if strings.Contains(file.Name(), fileFilter) { - files = append(files, file.Name()) - } - } - - return files, nil -} - -// getPendingMigrationFiles will loop through all migration files and return the ones that haven't been run yet. -func (a *Action) getPendingMigrationFiles() ([]string, error) { - var pendingFiles []string - var foundLastRan = false - - lastRanMigration, err := a.getLatestRanMigration() - if err != nil { - return pendingFiles, err - } - allFiles, err := a.listMigrationFiles() - if err != nil { - return pendingFiles, err - } - - if lastRanMigration == "" { // No migrations have run yet. - foundLastRan = true // If no migrations have previously ran, set found as true to start from first file. - } - - var i = 0 - for _, file := range allFiles { - if i == a.Number { - break - } - if file == lastRanMigration { - foundLastRan = true - continue - } - if !foundLastRan { - continue - } - - pendingFiles = append(pendingFiles, file) - i++ - } - - return pendingFiles, nil -} - -// getPreviouslyMigratedFiles will loop through all migration files and return the ones that have already been run. -func (a *Action) getMigrationFilesToRollBack() ([]string, error) { - var migrationsToRollBack []string - var foundLastRan = false - - lastRanMigration, err := a.getLatestRanMigration() - if err != nil { - return migrationsToRollBack, err - } - allFiles, err := a.listMigrationFiles() - if err != nil { - return migrationsToRollBack, err - } - - sort.Sort(sort.Reverse(sort.StringSlice(allFiles))) - - if lastRanMigration == "" { // No migrations have run yet. - return migrationsToRollBack, nil - } - - lastRanMigrationShortName := TrimExtension(lastRanMigration) - - var i = 0 - for _, file := range allFiles { - shortFileName := TrimExtension(file) - if i == a.Number { - break - } - if shortFileName == lastRanMigrationShortName { - foundLastRan = true - migrationsToRollBack = append(migrationsToRollBack, file) - i++ - continue - } - if !foundLastRan { - continue - } - - migrationsToRollBack = append(migrationsToRollBack, file) - i++ - } - - return migrationsToRollBack, nil -} - -// parseMigrations will iterate through the files and unmarshal the JSON and add to the files slice. -func (a *Action) parseMigrations(filesToParse []string) ([]File, error) { - var files []File - for _, file := range filesToParse { - - thisFile := File{Filename: file} - data, err := os.ReadFile(a.Path+"/"+file) - if err != nil { - return files, fmt.Errorf("unable to read file: %s. %s", file, err) - } - - queries := strings.Split(string(data), QuerySeparator) - - thisFile.Queries = queries - - files = append(files, thisFile) - } - - return files, nil -} \ No newline at end of file diff --git a/vendor/github.com/doublehops/go-migration/helpers/helpers.go b/vendor/github.com/doublehops/go-migration/helpers/helpers.go deleted file mode 100644 index 8ae4542..0000000 --- a/vendor/github.com/doublehops/go-migration/helpers/helpers.go +++ /dev/null @@ -1,7 +0,0 @@ -package helpers - -import "os" - -func PrintMsg(msg string) { - os.Stderr.WriteString(msg) -} diff --git a/vendor/github.com/doublehops/go-migration/history.go b/vendor/github.com/doublehops/go-migration/history.go deleted file mode 100644 index 18fbaa3..0000000 --- a/vendor/github.com/doublehops/go-migration/history.go +++ /dev/null @@ -1,47 +0,0 @@ -package go_migration - -import ( - "database/sql" - "fmt" -) - -type MigrationRecord struct { - ID int - Filename string - CreatedAt string -} - -// getLatestRanMigration will find the last processed migration. -func (a *Action) getLatestRanMigration() (string, error) { - - var record MigrationRecord - rows, err := a.DB.Query(GetLatestMigrationSQL) - if err != nil { - if err == sql.ErrNoRows { - return "", nil - } - } - defer rows.Close() - if rows == nil { - return "", nil - } - - for rows.Next() { - if err = rows.Scan(&record.ID, &record.Filename, &record.CreatedAt); err != nil { - return "", fmt.Errorf("error retrieving rows from migration table. %s", err) - } - } - - return record.Filename, nil -} - -// addMigrationTable will add a `migration` table to the database to track what has been -func (a *Action) addMigrationTable() error { - - _, err := a.DB.Exec(CreateMigrationsTable) - if err != nil { - return fmt.Errorf("error creating migrations table. %s", err) - } - - return nil -} diff --git a/vendor/github.com/doublehops/go-migration/migrate.go b/vendor/github.com/doublehops/go-migration/migrate.go deleted file mode 100644 index 3005ea0..0000000 --- a/vendor/github.com/doublehops/go-migration/migrate.go +++ /dev/null @@ -1,152 +0,0 @@ -package go_migration - -import ( - "database/sql" - "fmt" - "os" - "strings" - - "github.com/doublehops/go-migration/helpers" -) - -type Action struct { - Action string - Number int - Name string - - DB *sql.DB - Path string -} - -type File struct { - Filename string - Queries []string -} - -func TrimExtension(filename string) string { - var str string - - str = strings.Replace(filename, ".up.sql", "", 1) - str = strings.Replace(str, ".down.sql", "", 1) - - return str -} - -type TableList []Table - -type Table struct { - Name string -} - -func (a *Action) Migrate() error { - var err error - - err = a.ensureMigrationsTableExists() - if err != nil { - return err - } - - if a.Action == "create" { - err = a.CreateMigration(a.Path) - return err - } - - if a.Action == "up" { - pendingFiles, err := a.getPendingMigrationFiles() - if err != nil { - return err - } - if len(pendingFiles) == 0 { - helpers.PrintMsg("There are no pending migrations\n") - return nil - } - migrationFiles, err := a.parseMigrations(pendingFiles) - if err != nil { - return err - } - - if err = a.MigrateUp(migrationFiles); err != nil { - return err - } - } - - if a.Action == "down" { - previousFiles, err := a.getMigrationFilesToRollBack() - if err != nil { - return err - } - if len(previousFiles) == 0 { - helpers.PrintMsg("There are no previous migrations to rollback\n") - return nil - } - migrationFiles, err := a.parseMigrations(previousFiles) - if err != nil { - return err - } - - if err = a.MigrateDown(migrationFiles); err != nil { - return err - } - } - - return nil -} - -func (a *Action) IsValidAction(key string) bool { - - validActions := []string{ - "create", - "up", - "down", - } - - for _, item := range validActions { - if item == key { - return true - } - } - - return false -} - -func (a *Action) PrintHelp() { - var helpMsg = ` -Usage: -action= -number= -Examples: -./main.go -action create -name add_user_table // Will create a new migration file with template. -./main.go -action up -number 1 // number is optional. Will run all migrations if not included. -./main.go -action down -number 1 // number is optional. Will run only one migration if not included. -` - os.Stderr.WriteString(helpMsg) - os.Exit(1) -} - -// ensureMigrationsTableExists to create table to track migrations. -func (a *Action) ensureMigrationsTableExists() error { - var tableList TableList - rows, err := a.DB.Query(CheckMigrationsTableExistsSQL) - if err != nil { - return fmt.Errorf("++++>>>> Error: %w", err) - } - - for rows.Next() { - var t Table - if err = rows.Scan(&t.Name); err != nil { - return err - } - tableList = append(tableList, t) - } - - for _, tbl := range tableList { - if tbl.Name == "migrations" { // Already exists - return nil - } - } - - err = a.addMigrationTable() - if err != nil { - return err - } - - return nil -} diff --git a/vendor/github.com/doublehops/go-migration/sql.go b/vendor/github.com/doublehops/go-migration/sql.go deleted file mode 100644 index 2299a9d..0000000 --- a/vendor/github.com/doublehops/go-migration/sql.go +++ /dev/null @@ -1,27 +0,0 @@ -package go_migration - -const QuerySeparator = "------------------" - -var GetLatestMigrationSQL = `SELECT * FROM migrations - ORDER BY id DESC - LIMIT 1 -` - -var CreateMigrationsTable = `CREATE TABLE migrations ( - id INT(11) NOT NULL AUTO_INCREMENT, - filename VARCHAR(255), - created_at DATETIME, - PRIMARY KEY(id) -)` - -var CheckMigrationsTableExistsSQL = `SHOW TABLES` - -var InsertMigrationRecordIntoTableSQL = `INSERT INTO migrations - (filename,created_at) - VALUES - (?,NOW()) -` - -var RemoveMigrationRecordFromTableSQL = `DELETE FROM migrations - WHERE filename = ? -` diff --git a/vendor/github.com/golang-migrate/migrate/v4/.dockerignore b/vendor/github.com/golang-migrate/migrate/v4/.dockerignore new file mode 100644 index 0000000..f12dc01 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/.dockerignore @@ -0,0 +1,12 @@ +# Project +FAQ.md +README.md +LICENSE +.gitignore +.travis.yml +CONTRIBUTING.md +MIGRATIONS.md +docker-deploy.sh + +# Golang +testing diff --git a/vendor/github.com/golang-migrate/migrate/v4/.gitignore b/vendor/github.com/golang-migrate/migrate/v4/.gitignore new file mode 100644 index 0000000..23b5604 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/.gitignore @@ -0,0 +1,10 @@ +.DS_Store +cli/build +cli/cli +cli/migrate +.coverage +.godoc.pid +vendor/ +.vscode/ +.idea +dist/ diff --git a/vendor/github.com/golang-migrate/migrate/v4/.golangci.yml b/vendor/github.com/golang-migrate/migrate/v4/.golangci.yml new file mode 100644 index 0000000..68a8e95 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/.golangci.yml @@ -0,0 +1,30 @@ +run: + # timeout for analysis, e.g. 30s, 5m, default is 1m + timeout: 5m +linters: + enable: + #- golint + #- interfacer + - unconvert + #- dupl + - goconst + - gofmt + - misspell + - unparam + - nakedret + - prealloc + - revive + #- gosec +linters-settings: + misspell: + locale: US + revive: + rules: + - name: redundant-build-tag +issues: + max-same-issues: 0 + max-issues-per-linter: 0 + exclude-use-default: false + exclude: + # gosec: Duplicated errcheck checks + - G104 diff --git a/vendor/github.com/golang-migrate/migrate/v4/.goreleaser.yml b/vendor/github.com/golang-migrate/migrate/v4/.goreleaser.yml new file mode 100644 index 0000000..682248f --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/.goreleaser.yml @@ -0,0 +1,102 @@ +project_name: migrate +before: + hooks: + - go mod tidy +builds: + - env: + - CGO_ENABLED=0 + goos: + - linux + - windows + - darwin + goarch: + - amd64 + - arm + - arm64 + - 386 + goarm: + - 7 + main: ./cmd/migrate + ldflags: + - '-w -s -X main.Version={{ .Version }} -extldflags "static"' + flags: + - "-tags={{ .Env.DATABASE }} {{ .Env.SOURCE }}" + - "-trimpath" +nfpms: + - homepage: "https://github.com/golang-migrate/migrate" + maintainer: "dhui@users.noreply.github.com" + license: MIT + description: "Database migrations" + formats: + - deb + file_name_template: "{{ .ProjectName }}.{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}" +dockers: + - goos: linux + goarch: amd64 + dockerfile: Dockerfile.github-actions + use: buildx + ids: + - migrate + image_templates: + - 'migrate/migrate:{{ .Tag }}-amd64' + build_flag_templates: + - '--label=org.opencontainers.image.created={{ .Date }}' + - '--label=org.opencontainers.image.title={{ .ProjectName }}' + - '--label=org.opencontainers.image.revision={{ .FullCommit }}' + - '--label=org.opencontainers.image.version={{ .Version }}' + - "--label=org.opencontainers.image.source={{ .GitURL }}" + - "--platform=linux/amd64" + - goos: linux + goarch: arm64 + dockerfile: Dockerfile.github-actions + use: buildx + ids: + - migrate + image_templates: + - 'migrate/migrate:{{ .Tag }}-arm64' + build_flag_templates: + - '--label=org.opencontainers.image.created={{ .Date }}' + - '--label=org.opencontainers.image.title={{ .ProjectName }}' + - '--label=org.opencontainers.image.revision={{ .FullCommit }}' + - '--label=org.opencontainers.image.version={{ .Version }}' + - "--label=org.opencontainers.image.source={{ .GitURL }}" + - "--platform=linux/arm64" + +docker_manifests: +- name_template: 'migrate/migrate:{{ .Tag }}' + image_templates: + - 'migrate/migrate:{{ .Tag }}-amd64' + - 'migrate/migrate:{{ .Tag }}-arm64' +- name_template: 'migrate/migrate:{{ .Major }}' + image_templates: + - 'migrate/migrate:{{ .Tag }}-amd64' + - 'migrate/migrate:{{ .Tag }}-arm64' +- name_template: 'migrate/migrate:latest' + image_templates: + - 'migrate/migrate:{{ .Tag }}-amd64' + - 'migrate/migrate:{{ .Tag }}-arm64' +archives: + - name_template: "{{ .ProjectName }}.{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}" + format_overrides: + - goos: windows + format: zip +checksum: + name_template: 'sha256sum.txt' +release: + draft: true + prerelease: auto +source: + enabled: true + format: zip +changelog: + skip: false + sort: asc + filters: + exclude: + - '^docs:' + - '^test:' + - Merge pull request + - Merge branch + - go mod tidy +snapshot: + name_template: "{{ .Tag }}-next" diff --git a/vendor/github.com/golang-migrate/migrate/v4/.travis.yml b/vendor/github.com/golang-migrate/migrate/v4/.travis.yml new file mode 100644 index 0000000..fdaea8c --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/.travis.yml @@ -0,0 +1,138 @@ +language: go +sudo: required + +matrix: + allow_failures: + - go: master + include: + # Supported versions of Go: https://golang.org/dl/ + - go: "1.14.x" + - go: "1.15.x" + - go: master + +go_import_path: github.com/golang-migrate/migrate + +env: + global: + - GO111MODULE=on + - MIGRATE_TEST_CONTAINER_BOOT_TIMEOUT=60 + - DOCKER_USERNAME=golangmigrate + - secure: "oSOznzUrgr5h45qW4PONkREpisPAt40tnM+KFWtS/Ggu5UI2Ie0CmyYXWuBjbt7B97a4yN9Qzmn8FxJHJ7kk+ABOi3muhkxeIhr6esXbzHhX/Jhv0mj1xkzX7KoVN9oHBz3cOI/QeRyEAO68xjDHNE2kby4RTT9VBt6TQUakKVkqI5qkqLBTADepCjVC+9XhxVxUNyeWKU8ormaUfJBjoNVoDlwXekUPnJenfmfZqXxUInvBCfUyp7Pq+kurBORmg4yc6qOlRYuK67Xw+i5xpjbZouNlXPk0rq7pPy5zjhmZQ3kImoFPvNMeKViDcI6kSIJKtjdhms9/g/6MgXS9HlL5kFy8tYKbsyiHnHB1BsvaLAKXctbUZFDPstgMPADfnad2kZXPrNqIhfWKZrGRWidawCYJ1sKKwYxLMKrtA0umqgMoL90MmBOELhuGmvMV0cFJB+zo+K2YWjEiMGd8xRb5mC5aAy0ZcCehO46jGtpr217EJmMF8Ywr7cFqM2Shg5U2jev9qUpYiXwmPnJKDuoT2ZHuHmPgFIkYiWC5yeJnnmG5bed1sKBp93AFrJX+1Rx5oC4BpNegewmBZKpOSwls/D1uMAeQK3dPmQHLsT6o2VBLfeDGr+zY0R85ywwPZCv00vGol02zYoTqN7eFqr6Qhjr/qx5K1nnxJdFK3Ts=" + +services: + - docker + +cache: + directories: + - $GOPATH/pkg + + +before_install: + # Update docker to latest version: https://docs.travis-ci.com/user/docker/#installing-a-newer-docker-version + - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - + - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" + - sudo apt-get update + - sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce + # Install golangci-lint + - curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin v1.30.0 + - echo "TRAVIS_GO_VERSION=${TRAVIS_GO_VERSION}" + +install: + - go get github.com/mattn/goveralls + +script: + - golangci-lint run + - make test COVERAGE_DIR=/tmp/coverage + +after_success: + - goveralls -service=travis-ci -coverprofile /tmp/coverage/combined.txt + - make list-external-deps > dependency_tree.txt && cat dependency_tree.txt + - make build-cli + - gem install --no-document fpm + - fpm -s dir -t deb -n migrate -v "$(git describe --tags 2>/dev/null | cut -c 2-)" --license MIT -m dhui@users.noreply.github.com --url https://github.com/golang-migrate/migrate --description='Database migrations' -a amd64 -p migrate.$(git describe --tags 2>/dev/null | cut -c 2-).deb --deb-no-default-config-files -f -C cli/build migrate.linux-amd64=/usr/local/bin/migrate + +deploy: + - provider: releases + api_key: + secure: hWH1HLPpzpfA8pXQ93T1qKQVFSpQp0as/JLQ7D91jHuJ8p+RxVeqblDrR6HQY/95R/nyiE9GJmvUolSuw5h449LSrGxPtVWhdh6EnkxlQHlen5XeMhVjRjFV0sE9qGe8v7uAkiTfRO61ktTWHrEAvw5qpyqnNISodmZS78XIasPODQbNlzwINhWhDTHIjXGb4FpizYaL3OGCanrxfR9fQyCaqKGGBjRq3Mfq8U6Yd4mApmsE+uJxgaZV8K5zBqpkSzQRWhcVGNL5DuLsU3gfSJOo7kZeA2G71SHffH577dBoqtCZ4VFv169CoUZehLWCb+7XKJZmHXVujCURATSySLGUOPc6EoLFAn3YtsCA04mS4bZVo5FZPWVwfhjmkhtDR4f6wscKp7r1HsFHSOgm59QfETQdrn4MnZ44H2Jd39axqndn5DvK9EcZVjPHynOPnueXP2u6mTuUgh2VyyWBCDO3CNo0fGlo7VJI69IkIWNSD87K9cHZWYMClyKZkUzS+PmRAhHRYbVd+9ZjKOmnU36kUHNDG/ft1D4ogsY+rhVtXB4lgWDM5adri+EIScYdYnB1/pQexLBigcJY9uE7nQTR0U6QgVNYvun7uRNs40E0c4voSfmPdFO0FlOD2y1oQhnaXfWLbu9nMcTcs4RFGrcC7NzkUN4/WjG8s285V6w= + skip_cleanup: true + on: + go: "1.15.x" + repo: golang-migrate/migrate + tags: true + file: + - cli/build/migrate.linux-amd64.tar.gz + - cli/build/migrate.linux-armv7.tar.gz + - cli/build/migrate.linux-arm64.tar.gz + - cli/build/migrate.darwin-amd64.tar.gz + - cli/build/migrate.windows-amd64.exe.tar.gz + - cli/build/migrate.windows-386.exe.tar.gz + - cli/build/sha256sum.txt + - dependency_tree.txt + - provider: packagecloud + repository: migrate + username: golang-migrate + token: + secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o= + dist: ubuntu/xenial + package_glob: '*.deb' + skip_cleanup: true + on: + go: "1.15.x" + repo: golang-migrate/migrate + tags: true + - provider: packagecloud + repository: migrate + username: golang-migrate + token: + secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o= + dist: ubuntu/bionic + package_glob: '*.deb' + skip_cleanup: true + on: + go: "1.15.x" + repo: golang-migrate/migrate + tags: true + - provider: packagecloud + repository: migrate + username: golang-migrate + token: + secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o= + dist: ubuntu/focal + package_glob: '*.deb' + skip_cleanup: true + on: + go: "1.15.x" + repo: golang-migrate/migrate + tags: true + - provider: packagecloud + repository: migrate + username: golang-migrate + token: + secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o= + dist: debian/stretch + package_glob: '*.deb' + skip_cleanup: true + on: + go: "1.15.x" + repo: golang-migrate/migrate + tags: true + - provider: packagecloud + repository: migrate + username: golang-migrate + token: + secure: aICwu3gJ1sJ1QVCD3elpg+Jxzt4P+Zj1uoh5f0sOwnjDNIZ4FwUT1cMrWloP8P2KD0iyCOawuZER27o/kQ21oX2OxHvQbYPReA2znLm7lHzCmypAAOHPxpgnQ4rMGHHJXd+OsxtdclGs67c+EbdBfoRRbK400Qz/vjPJEDeH4mh02ZHC2nw4Nk/wV4jjBIkIt9dGEx6NgOA17FCMa3MaPHlHeFIzU7IfTlDHbS0mCCYbg/wafWBWcbGqtZLWAYtJDmfjrAStmDLdAX5J5PsB7taGSGPZHmPmpGoVgrKt/tb9Xz1rFBGslTpGROOiO4CiMAvkEKFn8mxrBGjfSBqp7Dp3eeSalKXB1DJAbEXx2sEbMcvmnoR9o43meaAn+ZRts8lRL8S/skBloe6Nk8bx3NlJCGB9WPK1G56b7c/fZnJxQbrCw6hxDfbZwm8S2YPviFTo/z1BfZDhRsL74reKsN2kgnGo2W/k38vvzIpsssQ9DHN1b0TLCxolCNPtQ7oHcQ1ohcjP2UgYXk0FhqDoL+9LQva/DU4N9sKH0UbAaqsMVSErLeG8A4aauuFcVrWRBaDYyTag4dQqzTulEy7iru2kDDIBgSQ1gMW/yoBOIPK4oi6MtbTf1X39fzXFLS1cDd3LW61yAu3YrbjAetpfx2frIvrRAiL9TxWA1gnrs5o= + dist: debian/buster + package_glob: '*.deb' + skip_cleanup: true + on: + go: "1.15.x" + repo: golang-migrate/migrate + tags: true + - provider: script + script: ./docker-deploy.sh + skip_cleanup: true + on: + go: "1.15.x" + repo: golang-migrate/migrate + tags: true diff --git a/vendor/github.com/golang-migrate/migrate/v4/CONTRIBUTING.md b/vendor/github.com/golang-migrate/migrate/v4/CONTRIBUTING.md new file mode 100644 index 0000000..84fb823 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/CONTRIBUTING.md @@ -0,0 +1,24 @@ +# Development, Testing and Contributing + + 1. Make sure you have a running Docker daemon + (Install for [MacOS](https://docs.docker.com/docker-for-mac/)) + 1. Use a version of Go that supports [modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) (e.g. Go 1.11+) + 1. Fork this repo and `git clone` somewhere to `$GOPATH/src/github.com/golang-migrate/migrate` + * Ensure that [Go modules are enabled](https://golang.org/cmd/go/#hdr-Preliminary_module_support) (e.g. your repo path or the `GO111MODULE` environment variable are set correctly) + 1. Install [golangci-lint](https://github.com/golangci/golangci-lint#install) + 1. Run the linter: `golangci-lint run` + 1. Confirm tests are working: `make test-short` + 1. Write awesome code ... + 1. `make test` to run all tests against all database versions + 1. Push code and open Pull Request + +Some more helpful commands: + + * You can specify which database/ source tests to run: + `make test-short SOURCE='file go_bindata' DATABASE='postgres cassandra'` + * After `make test`, run `make html-coverage` which opens a shiny test coverage overview. + * `make build-cli` builds the CLI in directory `cli/build/`. + * `make list-external-deps` lists all external dependencies for each package + * `make docs && make open-docs` opens godoc in your browser, `make kill-docs` kills the godoc server. + Repeatedly call `make docs` to refresh the server. + * Set the `DOCKER_API_VERSION` environment variable to the latest supported version if you get errors regarding the docker client API version being too new. diff --git a/vendor/github.com/golang-migrate/migrate/v4/Dockerfile b/vendor/github.com/golang-migrate/migrate/v4/Dockerfile new file mode 100644 index 0000000..eef2a47 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/Dockerfile @@ -0,0 +1,26 @@ +FROM golang:1.25-alpine3.21 AS builder +ARG VERSION + +RUN apk add --no-cache git gcc musl-dev make + +WORKDIR /go/src/github.com/golang-migrate/migrate + +ENV GO111MODULE=on + +COPY go.mod go.sum ./ + +RUN go mod download + +COPY . ./ + +RUN make build-docker + +FROM alpine:3.21 + +RUN apk add --no-cache ca-certificates + +COPY --from=builder /go/src/github.com/golang-migrate/migrate/build/migrate.linux-386 /usr/local/bin/migrate +RUN ln -s /usr/local/bin/migrate /migrate + +ENTRYPOINT ["migrate"] +CMD ["--help"] diff --git a/vendor/github.com/golang-migrate/migrate/v4/Dockerfile.circleci b/vendor/github.com/golang-migrate/migrate/v4/Dockerfile.circleci new file mode 100644 index 0000000..b6b244d --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/Dockerfile.circleci @@ -0,0 +1,17 @@ +ARG DOCKER_IMAGE +FROM $DOCKER_IMAGE + +RUN apk add --no-cache git gcc musl-dev make + +WORKDIR /go/src/github.com/golang-migrate/migrate + +ENV GO111MODULE=on +ENV COVERAGE_DIR=/tmp/coverage + +COPY go.mod go.sum ./ + +RUN go mod download + +COPY . ./ + +CMD ["make", "test"] diff --git a/vendor/github.com/golang-migrate/migrate/v4/Dockerfile.github-actions b/vendor/github.com/golang-migrate/migrate/v4/Dockerfile.github-actions new file mode 100644 index 0000000..9786e12 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/Dockerfile.github-actions @@ -0,0 +1,11 @@ +FROM alpine:3.19 + +RUN apk add --no-cache ca-certificates + +COPY migrate /usr/local/bin/migrate + +RUN ln -s /usr/local/bin/migrate /usr/bin/migrate +RUN ln -s /usr/local/bin/migrate /migrate + +ENTRYPOINT ["migrate"] +CMD ["--help"] diff --git a/vendor/github.com/golang-migrate/migrate/v4/FAQ.md b/vendor/github.com/golang-migrate/migrate/v4/FAQ.md new file mode 100644 index 0000000..88f3130 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/FAQ.md @@ -0,0 +1,79 @@ +# FAQ + +#### How is the code base structured? + ``` + / package migrate (the heart of everything) + /cli the CLI wrapper + /database database driver and sub directories have the actual driver implementations + /source source driver and sub directories have the actual driver implementations + ``` + +#### Why is there no `source/driver.go:Last()`? + It's not needed. And unless the source has a "native" way to read a directory in reversed order, + it might be expensive to do a full directory scan in order to get the last element. + +#### What is a NilMigration? NilVersion? + NilMigration defines a migration without a body. NilVersion is defined as const -1. + +#### What is the difference between uint(version) and int(targetVersion)? + version refers to an existing migration version coming from a source and therefore can never be negative. + targetVersion can either be a version OR represent a NilVersion, which equals -1. + +#### What's the difference between Next/Previous and Up/Down? + ``` + 1_first_migration.up.extension next -> 2_second_migration.up.extension ... + 1_first_migration.down.extension <- previous 2_second_migration.down.extension ... + ``` + +#### Why two separate files (up and down) for a migration? + It makes all of our lives easier. No new markup/syntax to learn for users + and existing database utility tools continue to work as expected. + +#### How many migrations can migrate handle? + Whatever the maximum positive signed integer value is for your platform. + For 32bit it would be 2,147,483,647 migrations. Migrate only keeps references to + the currently run and pre-fetched migrations in memory. Please note that some + source drivers need to do build a full "directory" tree first, which puts some + heat on the memory consumption. + +#### Are the table tests in migrate_test.go bloated? + Yes and no. There are duplicate test cases for sure but they don't hurt here. In fact + the tests are very visual now and might help new users understand expected behaviors quickly. + Migrate from version x to y and y is the last migration? Just check out the test for + that particular case and know what's going on instantly. + +#### What is Docker being used for? + Only for testing. See [testing/docker.go](testing/docker.go) + +#### Why not just use docker-compose? + It doesn't give us enough runtime control for testing. We want to be able to bring up containers fast + and whenever we want, not just once at the beginning of all tests. + +#### Can I maintain my driver in my own repository? + Yes, technically thats possible. We want to encourage you to contribute your driver to this repository though. + The driver's functionality is dictated by migrate's interfaces. That means there should really + just be one driver for a database/ source. We want to prevent a future where several drivers doing the exact same thing, + just implemented a bit differently, co-exist somewhere on GitHub. If users have to do research first to find the + "best" available driver for a database in order to get started, we would have failed as an open source community. + +#### Can I mix multiple sources during a batch of migrations? + No. + +#### What does "dirty" database mean? + Before a migration runs, each database sets a dirty flag. Execution stops if a migration fails and the dirty state persists, + which prevents attempts to run more migrations on top of a failed migration. You need to manually fix the error + and then "force" the expected version. + +#### What happens if two programs try and update the database at the same time? + Database-specific locking features are used by *some* database drivers to prevent multiple instances of migrate from running migrations on + the same database at the same time. For example, the MySQL driver uses the `GET_LOCK` function, while the Postgres driver uses + the `pg_advisory_lock` function. + +#### Do I need to create a table for tracking migration version used? +No, it is done automatically. + +#### Can I use migrate with a non-Go project? +Yes, you can use the migrate CLI in a non-Go project, but there are probably other libraries/frameworks available that offer better test and deploy integrations in that language/framework. + +#### I have got an error `Dirty database version 1. Fix and force version`. What should I do? +Keep calm and refer to [the getting started docs](GETTING_STARTED.md#forcing-your-database-version). diff --git a/vendor/github.com/golang-migrate/migrate/v4/GETTING_STARTED.md b/vendor/github.com/golang-migrate/migrate/v4/GETTING_STARTED.md new file mode 100644 index 0000000..45e9a4e --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/GETTING_STARTED.md @@ -0,0 +1,53 @@ +# Getting started +Before you start, you should understand the concept of forward/up and reverse/down database migrations. + +Configure a database for your application. Make sure that your database driver is supported [here](README.md#databases). + +## Create migrations +Create some migrations using migrate CLI. Here is an example: +``` +migrate create -ext sql -dir db/migrations -seq create_users_table +``` +Once you create your files, you should fill them. + +**IMPORTANT:** In a project developed by more than one person there is a chance of migrations inconsistency - e.g. two developers can create conflicting migrations, and the developer that created their migration later gets it merged to the repository first. +Developers and Teams should keep an eye on such cases (especially during code review). +[Here](https://github.com/golang-migrate/migrate/issues/179#issuecomment-475821264) is the issue summary if you would like to read more. + +Consider making your migrations idempotent - we can run the same sql code twice in a row with the same result. This makes our migrations more robust. On the other hand, it causes slightly less control over database schema - e.g. let's say you forgot to drop the table in down migration. You run down migration - the table is still there. When you run up migration again - `CREATE TABLE` would return an error, helping you find an issue in down migration, while `CREATE TABLE IF NOT EXISTS` would not. Use those conditions wisely. + +In case you would like to run several commands/queries in one migration, you should wrap them in a transaction (if your database supports it). +This way if one of commands fails, our database will remain unchanged. + +## Run migrations +Run your migrations through the CLI or your app and check if they applied expected changes. +Just to give you an idea: +``` +migrate -database YOUR_DATABASE_URL -path PATH_TO_YOUR_MIGRATIONS up +``` + +Just add the code to your app and you're ready to go! + +Before committing your migrations you should run your migrations up, down, and then up again to see if migrations are working properly both ways. +(e.g. if you created a table in a migration but reverse migration did not delete it, you will encounter an error when running the forward migration again) +It's also worth checking your migrations in a separate, containerized environment. You can find some tools at the [end of this document](#further-reading). + +**IMPORTANT:** If you would like to run multiple instances of your app on different machines be sure to use a database that supports locking when running migrations. Otherwise you may encounter issues. + +## Forcing your database version +In case you run a migration that contained an error, migrate will not let you run other migrations on the same database. You will see an error like `Dirty database version 1. Fix and force version`, even when you fix the erred migration. This means your database was marked as 'dirty'. +You need to investigate the migration error - was your migration applied partially, or was it not applied at all? Once you know, you should force your database to a version reflecting it's real state. You can do so with `force` command: +``` +migrate -path PATH_TO_YOUR_MIGRATIONS -database YOUR_DATABASE_URL force VERSION +``` +Once you force the version and your migration was fixed, your database is 'clean' again and you can proceed with your migrations. + +For details and example of usage see [this comment](https://github.com/golang-migrate/migrate/issues/282#issuecomment-530743258). + +## Further reading: +- [PostgreSQL tutorial](database/postgres/TUTORIAL.md) +- [Best practices](MIGRATIONS.md) +- [FAQ](FAQ.md) +- Tools for testing your migrations in a container: + - https://github.com/dhui/dktest + - https://github.com/ory/dockertest diff --git a/vendor/github.com/golang-migrate/migrate/v4/LICENSE b/vendor/github.com/golang-migrate/migrate/v4/LICENSE new file mode 100644 index 0000000..d03742c --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/LICENSE @@ -0,0 +1,28 @@ +The MIT License (MIT) + +Original Work +Copyright (c) 2016 Matthias Kadenbach +https://github.com/mattes/migrate + +Modified Work +Copyright (c) 2018 Dale Hui +https://github.com/golang-migrate/migrate + + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. diff --git a/vendor/github.com/golang-migrate/migrate/v4/MIGRATIONS.md b/vendor/github.com/golang-migrate/migrate/v4/MIGRATIONS.md new file mode 100644 index 0000000..3475d8e --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/MIGRATIONS.md @@ -0,0 +1,86 @@ +# Migrations + +## Migration Filename Format + +A single logical migration is represented as two separate migration files, one +to migrate "up" to the specified version from the previous version, and a second +to migrate back "down" to the previous version. These migrations can be provided +by any one of the supported [migration sources](./README.md#migration-sources). + +The ordering and direction of the migration files is determined by the filenames +used for them. `migrate` expects the filenames of migrations to have the format: + + {version}_{title}.up.{extension} + {version}_{title}.down.{extension} + +The `title` of each migration is unused, and is only for readability. Similarly, +the `extension` of the migration files is not checked by the library, and should +be an appropriate format for the database in use (`.sql` for SQL variants, for +instance). + +Versions of migrations may be represented as any 64 bit unsigned integer. +All migrations are applied upward in order of increasing version number, and +downward by decreasing version number. + +Common versioning schemes include incrementing integers: + + 1_initialize_schema.down.sql + 1_initialize_schema.up.sql + 2_add_table.down.sql + 2_add_table.up.sql + ... + +Or timestamps at an appropriate resolution: + + 1500360784_initialize_schema.down.sql + 1500360784_initialize_schema.up.sql + 1500445949_add_table.down.sql + 1500445949_add_table.up.sql + ... + +But any scheme resulting in distinct, incrementing integers as versions is valid. + +It is suggested that the version number of corresponding `up` and `down` migration +files be equivalent for clarity, but they are allowed to differ so long as the +relative ordering of the migrations is preserved. + +The migration files are permitted to be "empty", in the event that a migration +is a no-op or is irreversible. It is recommended to still include both migration +files by making the whole migration file consist of a comment. +If your database does not support comments, then deleting the migration file will also work. +Note, an actual empty file (e.g. a 0 byte file) may cause issues with your database since migrate +will attempt to run an empty query. In this case, deleting the migration file will also work. +For the rational of this behavior see: +[#244 (comment)](https://github.com/golang-migrate/migrate/issues/244#issuecomment-510758270) + +## Migration Content Format + +The format of the migration files themselves varies between database systems. +Different databases have different semantics around schema changes and when and +how they are allowed to occur +(for instance, [if schema changes can occur within a transaction](https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis)). + +As such, the `migrate` library has little to no checking around the format of +migration sources. The migration files are generally processed directly by the +drivers as raw operations. + +## Reversibility of Migrations + +Best practice for writing schema migration is that all migrations should be +reversible. It should in theory be possible for run migrations down and back up +through any and all versions with the state being fully cleaned and recreated +by doing so. + +By adhering to this recommended practice, development and deployment of new code +is cleaner and easier (cleaning database state for a new feature should be as +easy as migrating down to a prior version, and back up to the latest). + +As opposed to some other migration libraries, `migrate` represents up and down +migrations as separate files. This prevents any non-standard file syntax from +being introduced which may result in unintended behavior or errors, depending +on what database is processing the file. + +While it is technically possible for an up or down migration to exist on its own +without an equivalently versioned counterpart, it is strongly recommended to +always include a down migration which cleans up the state of the corresponding +up migration. diff --git a/vendor/github.com/golang-migrate/migrate/v4/Makefile b/vendor/github.com/golang-migrate/migrate/v4/Makefile new file mode 100644 index 0000000..8e23a43 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/Makefile @@ -0,0 +1,120 @@ +SOURCE ?= file go_bindata github github_ee bitbucket aws_s3 google_cloud_storage godoc_vfs gitlab +DATABASE ?= postgres mysql redshift cassandra spanner cockroachdb yugabytedb clickhouse mongodb sqlserver firebird neo4j pgx pgx5 rqlite +DATABASE_TEST ?= $(DATABASE) sqlite sqlite3 sqlcipher +VERSION ?= $(shell git describe --tags 2>/dev/null | cut -c 2-) +TEST_FLAGS ?= +REPO_OWNER ?= $(shell cd .. && basename "$$(pwd)") +COVERAGE_DIR ?= .coverage + +build: + CGO_ENABLED=0 go build -ldflags='-X main.Version=$(VERSION)' -tags '$(DATABASE) $(SOURCE)' ./cmd/migrate + +build-docker: + CGO_ENABLED=0 go build -a -o build/migrate.linux-386 -ldflags="-s -w -X main.Version=${VERSION}" -tags "$(DATABASE) $(SOURCE)" ./cmd/migrate + +build-cli: clean + -mkdir ./cli/build + cd ./cmd/migrate && CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o ../../cli/build/migrate.linux-amd64 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' . + cd ./cmd/migrate && CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -a -o ../../cli/build/migrate.linux-armv7 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' . + cd ./cmd/migrate && CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -o ../../cli/build/migrate.linux-arm64 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' . + cd ./cmd/migrate && CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -a -o ../../cli/build/migrate.darwin-amd64 -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' . + cd ./cmd/migrate && CGO_ENABLED=0 GOOS=windows GOARCH=386 go build -a -o ../../cli/build/migrate.windows-386.exe -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' . + cd ./cmd/migrate && CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -a -o ../../cli/build/migrate.windows-amd64.exe -ldflags='-X main.Version=$(VERSION) -extldflags "-static"' -tags '$(DATABASE) $(SOURCE)' . + cd ./cli/build && find . -name 'migrate*' | xargs -I{} tar czf {}.tar.gz {} + cd ./cli/build && shasum -a 256 * > sha256sum.txt + cat ./cli/build/sha256sum.txt + + +clean: + -rm -r ./cli/build + + +test-short: + make test-with-flags --ignore-errors TEST_FLAGS='-short' + + +test: + @-rm -r $(COVERAGE_DIR) + @mkdir $(COVERAGE_DIR) + make test-with-flags TEST_FLAGS='-v -race -covermode atomic -coverprofile $$(COVERAGE_DIR)/combined.txt -bench=. -benchmem -timeout 20m' + + +test-with-flags: + @echo SOURCE: $(SOURCE) + @echo DATABASE_TEST: $(DATABASE_TEST) + + @go test $(TEST_FLAGS) ./... + + +kill-orphaned-docker-containers: + docker rm -f $(shell docker ps -aq --filter label=migrate_test) + + +html-coverage: + go tool cover -html=$(COVERAGE_DIR)/combined.txt + + +list-external-deps: + $(call external_deps,'.') + $(call external_deps,'./cli/...') + $(call external_deps,'./testing/...') + + $(foreach v, $(SOURCE), $(call external_deps,'./source/$(v)/...')) + $(call external_deps,'./source/testing/...') + $(call external_deps,'./source/stub/...') + + $(foreach v, $(DATABASE), $(call external_deps,'./database/$(v)/...')) + $(call external_deps,'./database/testing/...') + $(call external_deps,'./database/stub/...') + + +restore-import-paths: + find . -name '*.go' -type f -execdir sed -i '' s%\"github.com/$(REPO_OWNER)/migrate%\"github.com/mattes/migrate%g '{}' \; + + +rewrite-import-paths: + find . -name '*.go' -type f -execdir sed -i '' s%\"github.com/mattes/migrate%\"github.com/$(REPO_OWNER)/migrate%g '{}' \; + + +# example: fswatch -0 --exclude .godoc.pid --event Updated . | xargs -0 -n1 -I{} make docs +docs: + -make kill-docs + nohup godoc -play -http=127.0.0.1:6064 /dev/null 2>&1 & echo $$! > .godoc.pid + cat .godoc.pid + + +kill-docs: + @cat .godoc.pid + kill -9 $$(cat .godoc.pid) + rm .godoc.pid + + +open-docs: + open http://localhost:6064/pkg/github.com/$(REPO_OWNER)/migrate + + +# example: make release V=0.0.0 +release: + git tag v$(V) + @read -p "Press enter to confirm and push to origin ..." && git push origin v$(V) + +echo-source: + @echo "$(SOURCE)" + +echo-database: + @echo "$(DATABASE)" + + +define external_deps + @echo '-- $(1)'; go list -f '{{join .Deps "\n"}}' $(1) | grep -v github.com/$(REPO_OWNER)/migrate | xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' + +endef + + +.PHONY: build build-docker build-cli clean test-short test test-with-flags html-coverage \ + restore-import-paths rewrite-import-paths list-external-deps release \ + docs kill-docs open-docs kill-orphaned-docker-containers echo-source echo-database + +SHELL = /bin/sh +RAND = $(shell echo $$RANDOM) + diff --git a/vendor/github.com/golang-migrate/migrate/v4/README.md b/vendor/github.com/golang-migrate/migrate/v4/README.md new file mode 100644 index 0000000..9b5b4b6 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/README.md @@ -0,0 +1,196 @@ +[![GitHub Workflow Status (branch)](https://img.shields.io/github/actions/workflow/status/golang-migrate/migrate/ci.yaml?branch=master)](https://github.com/golang-migrate/migrate/actions/workflows/ci.yaml?query=branch%3Amaster) +[![GoDoc](https://pkg.go.dev/badge/github.com/golang-migrate/migrate)](https://pkg.go.dev/github.com/golang-migrate/migrate/v4) +[![Coverage Status](https://img.shields.io/coveralls/github/golang-migrate/migrate/master.svg)](https://coveralls.io/github/golang-migrate/migrate?branch=master) +[![packagecloud.io](https://img.shields.io/badge/deb-packagecloud.io-844fec.svg)](https://packagecloud.io/golang-migrate/migrate?filter=debs) +[![Docker Pulls](https://img.shields.io/docker/pulls/migrate/migrate.svg)](https://hub.docker.com/r/migrate/migrate/) +![Supported Go Versions](https://img.shields.io/badge/Go-1.24%2C%201.25-lightgrey.svg) +[![GitHub Release](https://img.shields.io/github/release/golang-migrate/migrate.svg)](https://github.com/golang-migrate/migrate/releases) +[![Go Report Card](https://goreportcard.com/badge/github.com/golang-migrate/migrate/v4)](https://goreportcard.com/report/github.com/golang-migrate/migrate/v4) + +# migrate + +__Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).__ + +* Migrate reads migrations from [sources](#migration-sources) + and applies them in correct order to a [database](#databases). +* Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. + (Keeps the drivers lightweight, too.) +* Database drivers don't assume things or try to correct user input. When in doubt, fail. + +Forked from [mattes/migrate](https://github.com/mattes/migrate) + +## Databases + +Database drivers run migrations. [Add a new database?](database/driver.go) + +* [PostgreSQL](database/postgres) +* [PGX v4](database/pgx) +* [PGX v5](database/pgx/v5) +* [Redshift](database/redshift) +* [Ql](database/ql) +* [Cassandra / ScyllaDB](database/cassandra) +* [SQLite](database/sqlite) +* [SQLite3](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165)) +* [SQLCipher](database/sqlcipher) +* [MySQL / MariaDB](database/mysql) +* [Neo4j](database/neo4j) +* [MongoDB](database/mongodb) +* [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170)) +* [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171)) +* [Google Cloud Spanner](database/spanner) +* [CockroachDB](database/cockroachdb) +* [YugabyteDB](database/yugabytedb) +* [ClickHouse](database/clickhouse) +* [Firebird](database/firebird) +* [MS SQL Server](database/sqlserver) +* [rqlite](database/rqlite) + +### Database URLs + +Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: `dbdriver://username:password@host:port/dbname?param1=true¶m2=false` + +Any [reserved URL characters](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters) need to be escaped. Note, the `%` character also [needs to be escaped](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_the_percent_character) + +Explicitly, the following characters need to be escaped: +`!`, `#`, `$`, `%`, `&`, `'`, `(`, `)`, `*`, `+`, `,`, `/`, `:`, `;`, `=`, `?`, `@`, `[`, `]` + +It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below: + +```bash +$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))' +String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] +FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D +$ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")' +String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] +FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D +$ +``` + +## Migration Sources + +Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go) + +* [Filesystem](source/file) - read from filesystem +* [io/fs](source/iofs) - read from a Go [io/fs](https://pkg.go.dev/io/fs#FS) +* [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata)) +* [pkger](source/pkger) - read from embedded binary data ([markbates/pkger](https://github.com/markbates/pkger)) +* [GitHub](source/github) - read from remote GitHub repositories +* [GitHub Enterprise](source/github_ee) - read from remote GitHub Enterprise repositories +* [Bitbucket](source/bitbucket) - read from remote Bitbucket repositories +* [Gitlab](source/gitlab) - read from remote Gitlab repositories +* [AWS S3](source/aws_s3) - read from Amazon Web Services S3 +* [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage + +## CLI usage + +* Simple wrapper around this library. +* Handles ctrl+c (SIGINT) gracefully. +* No config search paths, no config files, no magic ENV var injections. + +[CLI Documentation](cmd/migrate) (includes CLI install instructions) + +### Basic usage + +```bash +$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2 +``` + +### Docker usage + +```bash +$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate + -path=/migrations/ -database postgres://localhost:5432/database up 2 +``` + +## Use in your Go project + +* API is stable and frozen for this release (v3 & v4). +* Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies. +* To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`. +* Bring your own logger. +* Uses `io.Reader` streams internally for low memory overhead. +* Thread-safe and no goroutine leaks. + +__[Go Documentation](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)__ + +```go +import ( + "github.com/golang-migrate/migrate/v4" + _ "github.com/golang-migrate/migrate/v4/database/postgres" + _ "github.com/golang-migrate/migrate/v4/source/github" +) + +func main() { + m, err := migrate.New( + "github://mattes:personal-access-token@mattes/migrate_test", + "postgres://localhost:5432/database?sslmode=enable") + m.Steps(2) +} +``` + +Want to use an existing database client? + +```go +import ( + "database/sql" + _ "github.com/lib/pq" + "github.com/golang-migrate/migrate/v4" + "github.com/golang-migrate/migrate/v4/database/postgres" + _ "github.com/golang-migrate/migrate/v4/source/file" +) + +func main() { + db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable") + driver, err := postgres.WithInstance(db, &postgres.Config{}) + m, err := migrate.NewWithDatabaseInstance( + "file:///migrations", + "postgres", driver) + m.Up() // or m.Steps(2) if you want to explicitly set the number of migrations to run +} +``` + +## Getting started + +Go to [getting started](GETTING_STARTED.md) + +## Tutorials + +* [CockroachDB](database/cockroachdb/TUTORIAL.md) +* [PostgreSQL](database/postgres/TUTORIAL.md) + +(more tutorials to come) + +## Migration files + +Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-files-up-and-down-for-a-migration) + +```bash +1481574547_create_users_table.up.sql +1481574547_create_users_table.down.sql +``` + +[Best practices: How to write migrations.](MIGRATIONS.md) + +## Coming from another db migration tool? + +Check out [migradaptor](https://github.com/musinit/migradaptor/). +*Note: migradaptor is not affiliated or supported by this project* + +## Versions + +Version | Supported? | Import | Notes +--------|------------|--------|------ +**master** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | New features and bug fixes arrive here first | +**v4** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | Used for stable releases | +**v3** | :x: | `import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended) | **DO NOT USE** - No longer supported | + +## Development and Contributing + +Yes, please! [`Makefile`](Makefile) is your friend, +read the [development guide](CONTRIBUTING.md). + +Also have a look at the [FAQ](FAQ.md). + +--- + +Looking for alternatives? [https://awesome-go.com/#database](https://awesome-go.com/#database). diff --git a/vendor/github.com/golang-migrate/migrate/v4/SECURITY.md b/vendor/github.com/golang-migrate/migrate/v4/SECURITY.md new file mode 100644 index 0000000..1d7146f --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/SECURITY.md @@ -0,0 +1,16 @@ +# Security Policy + +## Supported Versions + +| Version | Supported | +| ------- | ------------------ | +| master | :white_check_mark: | +| 4.x | :white_check_mark: | +| 3.x | :x: | +| < 3.0 | :x: | + +## Reporting a Vulnerability + +We prefer [coordinated disclosures](https://en.wikipedia.org/wiki/Coordinated_vulnerability_disclosure). To start one, create a GitHub security advisory following [these instructions](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing/privately-reporting-a-security-vulnerability) + +Please suggest potential impact and urgency in your reports. diff --git a/vendor/github.com/golang-migrate/migrate/v4/database/driver.go b/vendor/github.com/golang-migrate/migrate/v4/database/driver.go new file mode 100644 index 0000000..11268e6 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/database/driver.go @@ -0,0 +1,123 @@ +// Package database provides the Driver interface. +// All database drivers must implement this interface, register themselves, +// optionally provide a `WithInstance` function and pass the tests +// in package database/testing. +package database + +import ( + "fmt" + "io" + "sync" + + iurl "github.com/golang-migrate/migrate/v4/internal/url" +) + +var ( + ErrLocked = fmt.Errorf("can't acquire lock") + ErrNotLocked = fmt.Errorf("can't unlock, as not currently locked") +) + +const NilVersion int = -1 + +var driversMu sync.RWMutex +var drivers = make(map[string]Driver) + +// Driver is the interface every database driver must implement. +// +// How to implement a database driver? +// 1. Implement this interface. +// 2. Optionally, add a function named `WithInstance`. +// This function should accept an existing DB instance and a Config{} struct +// and return a driver instance. +// 3. Add a test that calls database/testing.go:Test() +// 4. Add own tests for Open(), WithInstance() (when provided) and Close(). +// All other functions are tested by tests in database/testing. +// Saves you some time and makes sure all database drivers behave the same way. +// 5. Call Register in init(). +// 6. Create a internal/cli/build_.go file +// 7. Add driver name in 'DATABASE' variable in Makefile +// +// Guidelines: +// - Don't try to correct user input. Don't assume things. +// When in doubt, return an error and explain the situation to the user. +// - All configuration input must come from the URL string in func Open() +// or the Config{} struct in WithInstance. Don't os.Getenv(). +type Driver interface { + // Open returns a new driver instance configured with parameters + // coming from the URL string. Migrate will call this function + // only once per instance. + Open(url string) (Driver, error) + + // Close closes the underlying database instance managed by the driver. + // Migrate will call this function only once per instance. + Close() error + + // Lock should acquire a database lock so that only one migration process + // can run at a time. Migrate will call this function before Run is called. + // If the implementation can't provide this functionality, return nil. + // Return database.ErrLocked if database is already locked. + Lock() error + + // Unlock should release the lock. Migrate will call this function after + // all migrations have been run. + Unlock() error + + // Run applies a migration to the database. migration is guaranteed to be not nil. + Run(migration io.Reader) error + + // SetVersion saves version and dirty state. + // Migrate will call this function before and after each call to Run. + // version must be >= -1. -1 means NilVersion. + SetVersion(version int, dirty bool) error + + // Version returns the currently active version and if the database is dirty. + // When no migration has been applied, it must return version -1. + // Dirty means, a previous migration failed and user interaction is required. + Version() (version int, dirty bool, err error) + + // Drop deletes everything in the database. + // Note that this is a breaking action, a new call to Open() is necessary to + // ensure subsequent calls work as expected. + Drop() error +} + +// Open returns a new driver instance. +func Open(url string) (Driver, error) { + scheme, err := iurl.SchemeFromURL(url) + if err != nil { + return nil, err + } + + driversMu.RLock() + d, ok := drivers[scheme] + driversMu.RUnlock() + if !ok { + return nil, fmt.Errorf("database driver: unknown driver %v (forgotten import?)", scheme) + } + + return d.Open(url) +} + +// Register globally registers a driver. +func Register(name string, driver Driver) { + driversMu.Lock() + defer driversMu.Unlock() + if driver == nil { + panic("Register driver is nil") + } + if _, dup := drivers[name]; dup { + panic("Register called twice for driver " + name) + } + drivers[name] = driver +} + +// List lists the registered drivers +func List() []string { + driversMu.RLock() + defer driversMu.RUnlock() + names := make([]string, 0, len(drivers)) + for n := range drivers { + names = append(names, n) + } + return names +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/database/error.go b/vendor/github.com/golang-migrate/migrate/v4/database/error.go new file mode 100644 index 0000000..eb802c7 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/database/error.go @@ -0,0 +1,27 @@ +package database + +import ( + "fmt" +) + +// Error should be used for errors involving queries ran against the database +type Error struct { + // Optional: the line number + Line uint + + // Query is a query excerpt + Query []byte + + // Err is a useful/helping error message for humans + Err string + + // OrigErr is the underlying error + OrigErr error +} + +func (e Error) Error() string { + if len(e.Err) == 0 { + return fmt.Sprintf("%v in line %v: %s", e.OrigErr, e.Line, e.Query) + } + return fmt.Sprintf("%v in line %v: %s (details: %v)", e.Err, e.Line, e.Query, e.OrigErr) +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/database/mysql/README.md b/vendor/github.com/golang-migrate/migrate/v4/database/mysql/README.md new file mode 100644 index 0000000..a687b1d --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/database/mysql/README.md @@ -0,0 +1,56 @@ +# MySQL + +`mysql://user:password@tcp(host:port)/dbname?query` + +| URL Query | WithInstance Config | Description | +|------------|---------------------|-------------| +| `x-migrations-table` | `MigrationsTable` | Name of the migrations table | +| `x-no-lock` | `NoLock` | Set to `true` to skip `GET_LOCK`/`RELEASE_LOCK` statements. Useful for [multi-master MySQL flavors](https://www.percona.com/doc/percona-xtradb-cluster/LATEST/features/pxc-strict-mode.html#explicit-table-locking). Only run migrations from one host when this is enabled. | +| `x-statement-timeout` | `StatementTimeout` | Abort any statement that takes more than the specified number of milliseconds, functionally similar to [Server-side SELECT statement timeouts](https://dev.mysql.com/blog-archive/server-side-select-statement-timeouts/) but enforced by the client. Available for all versions of MySQL, not just >=5.7. | +| `dbname` | `DatabaseName` | The name of the database to connect to | +| `user` | | The user to sign in as | +| `password` | | The user's password | +| `host` | | The host to connect to. | +| `port` | | The port to bind to. | +| `tls` | | TLS / SSL encrypted connection parameter; see [go-sql-driver](https://github.com/go-sql-driver/mysql#tls). Use any name (e.g. `migrate`) if you want to use a custom TLS config (`x-tls-` queries). | +| `x-tls-ca` | | The location of the CA (certificate authority) file. | +| `x-tls-cert` | | The location of the client certificate file. Must be used with `x-tls-key`. | +| `x-tls-key` | | The location of the private key file. Must be used with `x-tls-cert`. | +| `x-tls-insecure-skip-verify` | | Whether or not to use SSL (true\|false) | + +## Use with existing client + +If you use the MySQL driver with existing database client, you must create the client with parameter `multiStatements=true`: + +```go +package main + +import ( + "database/sql" + + _ "github.com/go-sql-driver/mysql" + "github.com/golang-migrate/migrate/v4" + "github.com/golang-migrate/migrate/v4/database/mysql" + _ "github.com/golang-migrate/migrate/v4/source/file" +) + +func main() { + db, _ := sql.Open("mysql", "user:password@tcp(host:port)/dbname?multiStatements=true") + driver, _ := mysql.WithInstance(db, &mysql.Config{}) + m, _ := migrate.NewWithDatabaseInstance( + "file:///migrations", + "mysql", + driver, + ) + + m.Steps(2) +} +``` + +## Upgrading from v1 + +1. Write down the current migration version from schema_migrations +1. `DROP TABLE schema_migrations` +2. Wrap your existing migrations in transactions ([BEGIN/COMMIT](https://dev.mysql.com/doc/refman/5.7/en/commit.html)) if you use multiple statements within one migration. +3. Download and install the latest migrate version. +4. Force the current migration version with `migrate force `. diff --git a/vendor/github.com/golang-migrate/migrate/v4/database/mysql/mysql.go b/vendor/github.com/golang-migrate/migrate/v4/database/mysql/mysql.go new file mode 100644 index 0000000..61a59d2 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/database/mysql/mysql.go @@ -0,0 +1,508 @@ +//go:build go1.9 + +package mysql + +import ( + "context" + "crypto/tls" + "crypto/x509" + "database/sql" + "errors" + "fmt" + "io" + nurl "net/url" + "os" + "strconv" + "strings" + "sync/atomic" + "time" + + "github.com/go-sql-driver/mysql" + "github.com/golang-migrate/migrate/v4/database" +) + +var _ database.Driver = (*Mysql)(nil) // explicit compile time type check + +func init() { + database.Register("mysql", &Mysql{}) +} + +var DefaultMigrationsTable = "schema_migrations" + +var ( + ErrDatabaseDirty = fmt.Errorf("database is dirty") + ErrNilConfig = fmt.Errorf("no config") + ErrNoDatabaseName = fmt.Errorf("no database name") + ErrAppendPEM = fmt.Errorf("failed to append PEM") + ErrTLSCertKeyConfig = fmt.Errorf("To use TLS client authentication, both x-tls-cert and x-tls-key must not be empty") +) + +type Config struct { + MigrationsTable string + DatabaseName string + NoLock bool + StatementTimeout time.Duration +} + +type Mysql struct { + // mysql RELEASE_LOCK must be called from the same conn, so + // just do everything over a single conn anyway. + conn *sql.Conn + db *sql.DB + isLocked atomic.Bool + + config *Config +} + +// connection instance must have `multiStatements` set to true +func WithConnection(ctx context.Context, conn *sql.Conn, config *Config) (*Mysql, error) { + if config == nil { + return nil, ErrNilConfig + } + + if err := conn.PingContext(ctx); err != nil { + return nil, err + } + + mx := &Mysql{ + conn: conn, + db: nil, + config: config, + } + + if config.DatabaseName == "" { + query := `SELECT DATABASE()` + var databaseName sql.NullString + if err := conn.QueryRowContext(ctx, query).Scan(&databaseName); err != nil { + return nil, &database.Error{OrigErr: err, Query: []byte(query)} + } + + if len(databaseName.String) == 0 { + return nil, ErrNoDatabaseName + } + + config.DatabaseName = databaseName.String + } + + if len(config.MigrationsTable) == 0 { + config.MigrationsTable = DefaultMigrationsTable + } + + if err := mx.ensureVersionTable(); err != nil { + return nil, err + } + + return mx, nil +} + +// instance must have `multiStatements` set to true +func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) { + ctx := context.Background() + + if err := instance.Ping(); err != nil { + return nil, err + } + + conn, err := instance.Conn(ctx) + if err != nil { + return nil, err + } + + mx, err := WithConnection(ctx, conn, config) + if err != nil { + return nil, err + } + + mx.db = instance + + return mx, nil +} + +// extractCustomQueryParams extracts the custom query params (ones that start with "x-") from +// mysql.Config.Params (connection parameters) as to not interfere with connecting to MySQL +func extractCustomQueryParams(c *mysql.Config) (map[string]string, error) { + if c == nil { + return nil, ErrNilConfig + } + customQueryParams := map[string]string{} + + for k, v := range c.Params { + if strings.HasPrefix(k, "x-") { + customQueryParams[k] = v + delete(c.Params, k) + } + } + return customQueryParams, nil +} + +func urlToMySQLConfig(url string) (*mysql.Config, error) { + // Need to parse out custom TLS parameters and call + // mysql.RegisterTLSConfig() before mysql.ParseDSN() is called + // which consumes the registered tls.Config + // Fixes: https://github.com/golang-migrate/migrate/issues/411 + // + // Can't use url.Parse() since it fails to parse MySQL DSNs + // mysql.ParseDSN() also searches for "?" to find query parameters: + // https://github.com/go-sql-driver/mysql/blob/46351a8/dsn.go#L344 + if idx := strings.LastIndex(url, "?"); idx > 0 { + rawParams := url[idx+1:] + parsedParams, err := nurl.ParseQuery(rawParams) + if err != nil { + return nil, err + } + + ctls := parsedParams.Get("tls") + if len(ctls) > 0 { + if _, isBool := readBool(ctls); !isBool && strings.ToLower(ctls) != "skip-verify" { + rootCertPool := x509.NewCertPool() + pem, err := os.ReadFile(parsedParams.Get("x-tls-ca")) + if err != nil { + return nil, err + } + + if ok := rootCertPool.AppendCertsFromPEM(pem); !ok { + return nil, ErrAppendPEM + } + + clientCert := make([]tls.Certificate, 0, 1) + if ccert, ckey := parsedParams.Get("x-tls-cert"), parsedParams.Get("x-tls-key"); ccert != "" || ckey != "" { + if ccert == "" || ckey == "" { + return nil, ErrTLSCertKeyConfig + } + certs, err := tls.LoadX509KeyPair(ccert, ckey) + if err != nil { + return nil, err + } + clientCert = append(clientCert, certs) + } + + insecureSkipVerify := false + insecureSkipVerifyStr := parsedParams.Get("x-tls-insecure-skip-verify") + if len(insecureSkipVerifyStr) > 0 { + x, err := strconv.ParseBool(insecureSkipVerifyStr) + if err != nil { + return nil, err + } + insecureSkipVerify = x + } + + err = mysql.RegisterTLSConfig(ctls, &tls.Config{ + RootCAs: rootCertPool, + Certificates: clientCert, + InsecureSkipVerify: insecureSkipVerify, + }) + if err != nil { + return nil, err + } + } + } + } + + config, err := mysql.ParseDSN(strings.TrimPrefix(url, "mysql://")) + if err != nil { + return nil, err + } + + config.MultiStatements = true + + // Keep backwards compatibility from when we used net/url.Parse() to parse the DSN. + // net/url.Parse() would automatically unescape it for us. + // See: https://play.golang.org/p/q9j1io-YICQ + user, err := nurl.QueryUnescape(config.User) + if err != nil { + return nil, err + } + config.User = user + + password, err := nurl.QueryUnescape(config.Passwd) + if err != nil { + return nil, err + } + config.Passwd = password + + return config, nil +} + +func (m *Mysql) Open(url string) (database.Driver, error) { + config, err := urlToMySQLConfig(url) + if err != nil { + return nil, err + } + + customParams, err := extractCustomQueryParams(config) + if err != nil { + return nil, err + } + + noLockParam, noLock := customParams["x-no-lock"], false + if noLockParam != "" { + noLock, err = strconv.ParseBool(noLockParam) + if err != nil { + return nil, fmt.Errorf("could not parse x-no-lock as bool: %w", err) + } + } + + statementTimeoutParam := customParams["x-statement-timeout"] + statementTimeout := 0 + if statementTimeoutParam != "" { + statementTimeout, err = strconv.Atoi(statementTimeoutParam) + if err != nil { + return nil, fmt.Errorf("could not parse x-statement-timeout as float: %w", err) + } + } + + db, err := sql.Open("mysql", config.FormatDSN()) + if err != nil { + return nil, err + } + + mx, err := WithInstance(db, &Config{ + DatabaseName: config.DBName, + MigrationsTable: customParams["x-migrations-table"], + NoLock: noLock, + StatementTimeout: time.Duration(statementTimeout) * time.Millisecond, + }) + if err != nil { + return nil, err + } + + return mx, nil +} + +func (m *Mysql) Close() error { + connErr := m.conn.Close() + var dbErr error + if m.db != nil { + dbErr = m.db.Close() + } + + if connErr != nil || dbErr != nil { + return fmt.Errorf("conn: %v, db: %v", connErr, dbErr) + } + return nil +} + +func (m *Mysql) Lock() error { + return database.CasRestoreOnErr(&m.isLocked, false, true, database.ErrLocked, func() error { + if m.config.NoLock { + return nil + } + aid, err := database.GenerateAdvisoryLockId( + fmt.Sprintf("%s:%s", m.config.DatabaseName, m.config.MigrationsTable)) + if err != nil { + return err + } + + query := "SELECT GET_LOCK(?, 10)" + var success bool + if err := m.conn.QueryRowContext(context.Background(), query, aid).Scan(&success); err != nil { + return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)} + } + + if !success { + return database.ErrLocked + } + + return nil + }) +} + +func (m *Mysql) Unlock() error { + return database.CasRestoreOnErr(&m.isLocked, true, false, database.ErrNotLocked, func() error { + if m.config.NoLock { + return nil + } + + aid, err := database.GenerateAdvisoryLockId( + fmt.Sprintf("%s:%s", m.config.DatabaseName, m.config.MigrationsTable)) + if err != nil { + return err + } + + query := `SELECT RELEASE_LOCK(?)` + if _, err := m.conn.ExecContext(context.Background(), query, aid); err != nil { + return &database.Error{OrigErr: err, Query: []byte(query)} + } + + // NOTE: RELEASE_LOCK could return NULL or (or 0 if the code is changed), + // in which case isLocked should be true until the timeout expires -- synchronizing + // these states is likely not worth trying to do; reconsider the necessity of isLocked. + + return nil + }) +} + +func (m *Mysql) Run(migration io.Reader) error { + migr, err := io.ReadAll(migration) + if err != nil { + return err + } + + ctx := context.Background() + if m.config.StatementTimeout != 0 { + var cancel context.CancelFunc + ctx, cancel = context.WithTimeout(ctx, m.config.StatementTimeout) + defer cancel() + } + + query := string(migr[:]) + if _, err := m.conn.ExecContext(ctx, query); err != nil { + return database.Error{OrigErr: err, Err: "migration failed", Query: migr} + } + + return nil +} + +func (m *Mysql) SetVersion(version int, dirty bool) error { + tx, err := m.conn.BeginTx(context.Background(), &sql.TxOptions{Isolation: sql.LevelSerializable}) + if err != nil { + return &database.Error{OrigErr: err, Err: "transaction start failed"} + } + + query := "DELETE FROM `" + m.config.MigrationsTable + "` LIMIT 1" + if _, err := tx.ExecContext(context.Background(), query); err != nil { + if errRollback := tx.Rollback(); errRollback != nil { + err = errors.Join(err, errRollback) + } + return &database.Error{OrigErr: err, Query: []byte(query)} + } + + // Also re-write the schema version for nil dirty versions to prevent + // empty schema version for failed down migration on the first migration + // See: https://github.com/golang-migrate/migrate/issues/330 + if version >= 0 || (version == database.NilVersion && dirty) { + query := "INSERT INTO `" + m.config.MigrationsTable + "` (version, dirty) VALUES (?, ?)" + if _, err := tx.ExecContext(context.Background(), query, version, dirty); err != nil { + if errRollback := tx.Rollback(); errRollback != nil { + err = errors.Join(err, errRollback) + } + return &database.Error{OrigErr: err, Query: []byte(query)} + } + } + + if err := tx.Commit(); err != nil { + return &database.Error{OrigErr: err, Err: "transaction commit failed"} + } + + return nil +} + +func (m *Mysql) Version() (version int, dirty bool, err error) { + query := "SELECT version, dirty FROM `" + m.config.MigrationsTable + "` LIMIT 1" + err = m.conn.QueryRowContext(context.Background(), query).Scan(&version, &dirty) + switch { + case err == sql.ErrNoRows: + return database.NilVersion, false, nil + + case err != nil: + if e, ok := err.(*mysql.MySQLError); ok { + if e.Number == 0 { + return database.NilVersion, false, nil + } + } + return 0, false, &database.Error{OrigErr: err, Query: []byte(query)} + + default: + return version, dirty, nil + } +} + +func (m *Mysql) Drop() (err error) { + // select all tables + query := `SHOW TABLES LIKE '%'` + tables, err := m.conn.QueryContext(context.Background(), query) + if err != nil { + return &database.Error{OrigErr: err, Query: []byte(query)} + } + defer func() { + if errClose := tables.Close(); errClose != nil { + err = errors.Join(err, errClose) + } + }() + + // delete one table after another + tableNames := make([]string, 0) + for tables.Next() { + var tableName string + if err := tables.Scan(&tableName); err != nil { + return err + } + if len(tableName) > 0 { + tableNames = append(tableNames, tableName) + } + } + if err := tables.Err(); err != nil { + return &database.Error{OrigErr: err, Query: []byte(query)} + } + + if len(tableNames) > 0 { + // disable checking foreign key constraints until finished + query = `SET foreign_key_checks = 0` + if _, err := m.conn.ExecContext(context.Background(), query); err != nil { + return &database.Error{OrigErr: err, Query: []byte(query)} + } + + defer func() { + // enable foreign key checks + _, _ = m.conn.ExecContext(context.Background(), `SET foreign_key_checks = 1`) + }() + + // delete one by one ... + for _, t := range tableNames { + query = "DROP TABLE IF EXISTS `" + t + "`" + if _, err := m.conn.ExecContext(context.Background(), query); err != nil { + return &database.Error{OrigErr: err, Query: []byte(query)} + } + } + } + + return nil +} + +// ensureVersionTable checks if versions table exists and, if not, creates it. +// Note that this function locks the database, which deviates from the usual +// convention of "caller locks" in the Mysql type. +func (m *Mysql) ensureVersionTable() (err error) { + if err = m.Lock(); err != nil { + return err + } + + defer func() { + if e := m.Unlock(); e != nil { + err = errors.Join(err, e) + } + }() + + // check if migration table exists + var result string + query := `SHOW TABLES LIKE '` + m.config.MigrationsTable + `'` + if err := m.conn.QueryRowContext(context.Background(), query).Scan(&result); err != nil { + if err != sql.ErrNoRows { + return &database.Error{OrigErr: err, Query: []byte(query)} + } + } else { + return nil + } + + // if not, create the empty migration table + query = "CREATE TABLE `" + m.config.MigrationsTable + "` (version bigint not null primary key, dirty boolean not null)" + if _, err := m.conn.ExecContext(context.Background(), query); err != nil { + return &database.Error{OrigErr: err, Query: []byte(query)} + } + return nil +} + +// Returns the bool value of the input. +// The 2nd return value indicates if the input was a valid bool value +// See https://github.com/go-sql-driver/mysql/blob/a059889267dc7170331388008528b3b44479bffb/utils.go#L71 +func readBool(input string) (value bool, valid bool) { + switch input { + case "1", "true", "TRUE", "True": + return true, true + case "0", "false", "FALSE", "False": + return false, true + } + + // Not a valid bool value + return +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/database/util.go b/vendor/github.com/golang-migrate/migrate/v4/database/util.go new file mode 100644 index 0000000..c60fafb --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/database/util.go @@ -0,0 +1,33 @@ +package database + +import ( + "fmt" + "hash/crc32" + "strings" + "sync/atomic" +) + +const advisoryLockIDSalt uint = 1486364155 + +// GenerateAdvisoryLockId inspired by rails migrations, see https://goo.gl/8o9bCT +func GenerateAdvisoryLockId(databaseName string, additionalNames ...string) (string, error) { // nolint: golint + if len(additionalNames) > 0 { + databaseName = strings.Join(append(additionalNames, databaseName), "\x00") + } + sum := crc32.ChecksumIEEE([]byte(databaseName)) + sum = sum * uint32(advisoryLockIDSalt) + return fmt.Sprint(sum), nil +} + +// CasRestoreOnErr CAS wrapper to automatically restore the lock state on error +func CasRestoreOnErr(lock *atomic.Bool, o, n bool, casErr error, f func() error) error { + if !lock.CompareAndSwap(o, n) { + return casErr + } + if err := f(); err != nil { + // Automatically unlock/lock on error + lock.Store(o) + return err + } + return nil +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/docker-deploy.sh b/vendor/github.com/golang-migrate/migrate/v4/docker-deploy.sh new file mode 100644 index 0000000..558ea79 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/docker-deploy.sh @@ -0,0 +1,5 @@ +#!/bin/bash + +echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin && \ +docker build --build-arg VERSION="$TRAVIS_TAG" . -t migrate/migrate -t migrate/migrate:"$TRAVIS_TAG" && \ +docker push migrate/migrate:"$TRAVIS_TAG" && docker push migrate/migrate diff --git a/vendor/github.com/golang-migrate/migrate/v4/internal/url/url.go b/vendor/github.com/golang-migrate/migrate/v4/internal/url/url.go new file mode 100644 index 0000000..e793fa8 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/internal/url/url.go @@ -0,0 +1,25 @@ +package url + +import ( + "errors" + "strings" +) + +var errNoScheme = errors.New("no scheme") +var errEmptyURL = errors.New("URL cannot be empty") + +// schemeFromURL returns the scheme from a URL string +func SchemeFromURL(url string) (string, error) { + if url == "" { + return "", errEmptyURL + } + + i := strings.Index(url, ":") + + // No : or : is the first character. + if i < 1 { + return "", errNoScheme + } + + return url[0:i], nil +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/log.go b/vendor/github.com/golang-migrate/migrate/v4/log.go new file mode 100644 index 0000000..cb00b77 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/log.go @@ -0,0 +1,12 @@ +package migrate + +// Logger is an interface so you can pass in your own +// logging implementation. +type Logger interface { + + // Printf is like fmt.Printf + Printf(format string, v ...interface{}) + + // Verbose should return true when verbose logging output is wanted + Verbose() bool +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/migrate.go b/vendor/github.com/golang-migrate/migrate/v4/migrate.go new file mode 100644 index 0000000..7cac0ba --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/migrate.go @@ -0,0 +1,979 @@ +// Package migrate reads migrations from sources and runs them against databases. +// Sources are defined by the `source.Driver` and databases by the `database.Driver` +// interface. The driver interfaces are kept "dumb", all migration logic is kept +// in this package. +package migrate + +import ( + "errors" + "fmt" + "os" + "sync" + "time" + + "github.com/golang-migrate/migrate/v4/database" + iurl "github.com/golang-migrate/migrate/v4/internal/url" + "github.com/golang-migrate/migrate/v4/source" +) + +// DefaultPrefetchMigrations sets the number of migrations to pre-read +// from the source. This is helpful if the source is remote, but has little +// effect for a local source (i.e. file system). +// Please note that this setting has a major impact on the memory usage, +// since each pre-read migration is buffered in memory. See DefaultBufferSize. +var DefaultPrefetchMigrations = uint(10) + +// DefaultLockTimeout sets the max time a database driver has to acquire a lock. +var DefaultLockTimeout = 15 * time.Second + +var ( + ErrNoChange = errors.New("no change") + ErrNilVersion = errors.New("no migration") + ErrInvalidVersion = errors.New("version must be >= -1") + ErrLocked = errors.New("database locked") + ErrLockTimeout = errors.New("timeout: can't acquire database lock") +) + +// ErrShortLimit is an error returned when not enough migrations +// can be returned by a source for a given limit. +type ErrShortLimit struct { + Short uint +} + +// Error implements the error interface. +func (e ErrShortLimit) Error() string { + return fmt.Sprintf("limit %v short", e.Short) +} + +type ErrDirty struct { + Version int +} + +func (e ErrDirty) Error() string { + return fmt.Sprintf("Dirty database version %v. Fix and force version.", e.Version) +} + +type Migrate struct { + sourceName string + sourceDrv source.Driver + databaseDriverName string + databaseDrv database.Driver + + // Log accepts a Logger interface + Log Logger + + // GracefulStop accepts `true` and will stop executing migrations + // as soon as possible at a safe break point, so that the database + // is not corrupted. + GracefulStop chan bool + isLockedMu *sync.Mutex + + isGracefulStop bool + isLocked bool + + // PrefetchMigrations defaults to DefaultPrefetchMigrations, + // but can be set per Migrate instance. + PrefetchMigrations uint + + // LockTimeout defaults to DefaultLockTimeout, + // but can be set per Migrate instance. + LockTimeout time.Duration +} + +// New returns a new Migrate instance from a source URL and a database URL. +// The URL scheme is defined by each driver. +func New(sourceURL, databaseURL string) (*Migrate, error) { + m := newCommon() + + sourceName, err := iurl.SchemeFromURL(sourceURL) + if err != nil { + return nil, fmt.Errorf("failed to parse scheme from source URL: %w", err) + } + m.sourceName = sourceName + + databaseDriverName, err := iurl.SchemeFromURL(databaseURL) + if err != nil { + return nil, fmt.Errorf("failed to parse scheme from database URL: %w", err) + } + m.databaseDriverName = databaseDriverName + + sourceDrv, err := source.Open(sourceURL) + if err != nil { + return nil, fmt.Errorf("failed to open source, %q: %w", sourceURL, err) + } + m.sourceDrv = sourceDrv + + databaseDrv, err := database.Open(databaseURL) + if err != nil { + return nil, fmt.Errorf("failed to open database: %w", err) + } + m.databaseDrv = databaseDrv + + return m, nil +} + +// NewWithDatabaseInstance returns a new Migrate instance from a source URL +// and an existing database instance. The source URL scheme is defined by each driver. +// Use any string that can serve as an identifier during logging as databaseDriverName. +// You are responsible for closing the underlying database client if necessary. +func NewWithDatabaseInstance(sourceURL string, databaseDriverName string, databaseInstance database.Driver) (*Migrate, error) { + m := newCommon() + + sourceName, err := iurl.SchemeFromURL(sourceURL) + if err != nil { + return nil, err + } + m.sourceName = sourceName + + m.databaseDriverName = databaseDriverName + + sourceDrv, err := source.Open(sourceURL) + if err != nil { + return nil, fmt.Errorf("failed to open source, %q: %w", sourceURL, err) + } + m.sourceDrv = sourceDrv + + m.databaseDrv = databaseInstance + + return m, nil +} + +// NewWithSourceInstance returns a new Migrate instance from an existing source instance +// and a database URL. The database URL scheme is defined by each driver. +// Use any string that can serve as an identifier during logging as sourceName. +// You are responsible for closing the underlying source client if necessary. +func NewWithSourceInstance(sourceName string, sourceInstance source.Driver, databaseURL string) (*Migrate, error) { + m := newCommon() + + databaseDriverName, err := iurl.SchemeFromURL(databaseURL) + if err != nil { + return nil, fmt.Errorf("failed to parse scheme from database URL: %w", err) + } + m.databaseDriverName = databaseDriverName + + m.sourceName = sourceName + + databaseDrv, err := database.Open(databaseURL) + if err != nil { + return nil, fmt.Errorf("failed to open database: %w", err) + } + m.databaseDrv = databaseDrv + + m.sourceDrv = sourceInstance + + return m, nil +} + +// NewWithInstance returns a new Migrate instance from an existing source and +// database instance. Use any string that can serve as an identifier during logging +// as sourceName and databaseDriverName. You are responsible for closing down +// the underlying source and database client if necessary. +func NewWithInstance(sourceName string, sourceInstance source.Driver, databaseDriverName string, databaseInstance database.Driver) (*Migrate, error) { + m := newCommon() + + m.sourceName = sourceName + m.databaseDriverName = databaseDriverName + + m.sourceDrv = sourceInstance + m.databaseDrv = databaseInstance + + return m, nil +} + +func newCommon() *Migrate { + return &Migrate{ + GracefulStop: make(chan bool, 1), + PrefetchMigrations: DefaultPrefetchMigrations, + LockTimeout: DefaultLockTimeout, + isLockedMu: &sync.Mutex{}, + } +} + +// Close closes the source and the database. +func (m *Migrate) Close() (source error, database error) { + databaseSrvClose := make(chan error) + sourceSrvClose := make(chan error) + + m.logVerbosePrintf("Closing source and database\n") + + go func() { + databaseSrvClose <- m.databaseDrv.Close() + }() + + go func() { + sourceSrvClose <- m.sourceDrv.Close() + }() + + return <-sourceSrvClose, <-databaseSrvClose +} + +// Migrate looks at the currently active migration version, +// then migrates either up or down to the specified version. +func (m *Migrate) Migrate(version uint) error { + if err := m.lock(); err != nil { + return err + } + + curVersion, dirty, err := m.databaseDrv.Version() + if err != nil { + return m.unlockErr(err) + } + + if dirty { + return m.unlockErr(ErrDirty{curVersion}) + } + + ret := make(chan interface{}, m.PrefetchMigrations) + go m.read(curVersion, int(version), ret) + + return m.unlockErr(m.runMigrations(ret)) +} + +// Steps looks at the currently active migration version. +// It will migrate up if n > 0, and down if n < 0. +func (m *Migrate) Steps(n int) error { + if n == 0 { + return ErrNoChange + } + + if err := m.lock(); err != nil { + return err + } + + curVersion, dirty, err := m.databaseDrv.Version() + if err != nil { + return m.unlockErr(err) + } + + if dirty { + return m.unlockErr(ErrDirty{curVersion}) + } + + ret := make(chan interface{}, m.PrefetchMigrations) + + if n > 0 { + go m.readUp(curVersion, n, ret) + } else { + go m.readDown(curVersion, -n, ret) + } + + return m.unlockErr(m.runMigrations(ret)) +} + +// Up looks at the currently active migration version +// and will migrate all the way up (applying all up migrations). +func (m *Migrate) Up() error { + if err := m.lock(); err != nil { + return err + } + + curVersion, dirty, err := m.databaseDrv.Version() + if err != nil { + return m.unlockErr(err) + } + + if dirty { + return m.unlockErr(ErrDirty{curVersion}) + } + + ret := make(chan interface{}, m.PrefetchMigrations) + + go m.readUp(curVersion, -1, ret) + return m.unlockErr(m.runMigrations(ret)) +} + +// Down looks at the currently active migration version +// and will migrate all the way down (applying all down migrations). +func (m *Migrate) Down() error { + if err := m.lock(); err != nil { + return err + } + + curVersion, dirty, err := m.databaseDrv.Version() + if err != nil { + return m.unlockErr(err) + } + + if dirty { + return m.unlockErr(ErrDirty{curVersion}) + } + + ret := make(chan interface{}, m.PrefetchMigrations) + go m.readDown(curVersion, -1, ret) + return m.unlockErr(m.runMigrations(ret)) +} + +// Drop deletes everything in the database. +func (m *Migrate) Drop() error { + if err := m.lock(); err != nil { + return err + } + if err := m.databaseDrv.Drop(); err != nil { + return m.unlockErr(err) + } + return m.unlock() +} + +// Run runs any migration provided by you against the database. +// It does not check any currently active version in database. +// Usually you don't need this function at all. Use Migrate, +// Steps, Up or Down instead. +func (m *Migrate) Run(migration ...*Migration) error { + if len(migration) == 0 { + return ErrNoChange + } + + if err := m.lock(); err != nil { + return err + } + + curVersion, dirty, err := m.databaseDrv.Version() + if err != nil { + return m.unlockErr(err) + } + + if dirty { + return m.unlockErr(ErrDirty{curVersion}) + } + + ret := make(chan interface{}, m.PrefetchMigrations) + + go func() { + defer close(ret) + for _, migr := range migration { + if m.PrefetchMigrations > 0 && migr.Body != nil { + m.logVerbosePrintf("Start buffering %v\n", migr.LogString()) + } else { + m.logVerbosePrintf("Scheduled %v\n", migr.LogString()) + } + + ret <- migr + go func(migr *Migration) { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }(migr) + } + }() + + return m.unlockErr(m.runMigrations(ret)) +} + +// Force sets a migration version. +// It does not check any currently active version in database. +// It resets the dirty state to false. +func (m *Migrate) Force(version int) error { + if version < -1 { + return ErrInvalidVersion + } + + if err := m.lock(); err != nil { + return err + } + + if err := m.databaseDrv.SetVersion(version, false); err != nil { + return m.unlockErr(err) + } + + return m.unlock() +} + +// Version returns the currently active migration version. +// If no migration has been applied, yet, it will return ErrNilVersion. +func (m *Migrate) Version() (version uint, dirty bool, err error) { + v, d, err := m.databaseDrv.Version() + if err != nil { + return 0, false, err + } + + if v == database.NilVersion { + return 0, false, ErrNilVersion + } + + return suint(v), d, nil +} + +// read reads either up or down migrations from source `from` to `to`. +// Each migration is then written to the ret channel. +// If an error occurs during reading, that error is written to the ret channel, too. +// Once read is done reading it will close the ret channel. +func (m *Migrate) read(from int, to int, ret chan<- interface{}) { + defer close(ret) + + // check if from version exists + if from >= 0 { + if err := m.versionExists(suint(from)); err != nil { + ret <- err + return + } + } + + // check if to version exists + if to >= 0 { + if err := m.versionExists(suint(to)); err != nil { + ret <- err + return + } + } + + // no change? + if from == to { + ret <- ErrNoChange + return + } + + if from < to { + // it's going up + // apply first migration if from is nil version + if from == -1 { + firstVersion, err := m.sourceDrv.First() + if err != nil { + ret <- err + return + } + + migr, err := m.newMigration(firstVersion, int(firstVersion)) + if err != nil { + ret <- err + return + } + + ret <- migr + go func() { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }() + + from = int(firstVersion) + } + + // run until we reach target ... + for from < to { + if m.stop() { + return + } + + next, err := m.sourceDrv.Next(suint(from)) + if err != nil { + ret <- err + return + } + + migr, err := m.newMigration(next, int(next)) + if err != nil { + ret <- err + return + } + + ret <- migr + go func() { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }() + + from = int(next) + } + + } else { + // it's going down + // run until we reach target ... + for from > to && from >= 0 { + if m.stop() { + return + } + + prev, err := m.sourceDrv.Prev(suint(from)) + if errors.Is(err, os.ErrNotExist) && to == -1 { + // apply nil migration + migr, err := m.newMigration(suint(from), -1) + if err != nil { + ret <- err + return + } + ret <- migr + go func() { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }() + + return + + } else if err != nil { + ret <- err + return + } + + migr, err := m.newMigration(suint(from), int(prev)) + if err != nil { + ret <- err + return + } + + ret <- migr + go func() { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }() + + from = int(prev) + } + } +} + +// readUp reads up migrations from `from` limited by `limit`. +// limit can be -1, implying no limit and reading until there are no more migrations. +// Each migration is then written to the ret channel. +// If an error occurs during reading, that error is written to the ret channel, too. +// Once readUp is done reading it will close the ret channel. +func (m *Migrate) readUp(from int, limit int, ret chan<- interface{}) { + defer close(ret) + + // check if from version exists + if from >= 0 { + if err := m.versionExists(suint(from)); err != nil { + ret <- err + return + } + } + + if limit == 0 { + ret <- ErrNoChange + return + } + + count := 0 + for count < limit || limit == -1 { + if m.stop() { + return + } + + // apply first migration if from is nil version + if from == -1 { + firstVersion, err := m.sourceDrv.First() + if err != nil { + ret <- err + return + } + + migr, err := m.newMigration(firstVersion, int(firstVersion)) + if err != nil { + ret <- err + return + } + + ret <- migr + go func() { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }() + from = int(firstVersion) + count++ + continue + } + + // apply next migration + next, err := m.sourceDrv.Next(suint(from)) + if errors.Is(err, os.ErrNotExist) { + // no limit, but no migrations applied? + if limit == -1 && count == 0 { + ret <- ErrNoChange + return + } + + // no limit, reached end + if limit == -1 { + return + } + + // reached end, and didn't apply any migrations + if limit > 0 && count == 0 { + ret <- os.ErrNotExist + return + } + + // applied less migrations than limit? + if count < limit { + ret <- ErrShortLimit{suint(limit - count)} + return + } + } + if err != nil { + ret <- err + return + } + + migr, err := m.newMigration(next, int(next)) + if err != nil { + ret <- err + return + } + + ret <- migr + go func() { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }() + from = int(next) + count++ + } +} + +// readDown reads down migrations from `from` limited by `limit`. +// limit can be -1, implying no limit and reading until there are no more migrations. +// Each migration is then written to the ret channel. +// If an error occurs during reading, that error is written to the ret channel, too. +// Once readDown is done reading it will close the ret channel. +func (m *Migrate) readDown(from int, limit int, ret chan<- interface{}) { + defer close(ret) + + // check if from version exists + if from >= 0 { + if err := m.versionExists(suint(from)); err != nil { + ret <- err + return + } + } + + if limit == 0 { + ret <- ErrNoChange + return + } + + // no change if already at nil version + if from == -1 && limit == -1 { + ret <- ErrNoChange + return + } + + // can't go over limit if already at nil version + if from == -1 && limit > 0 { + ret <- os.ErrNotExist + return + } + + count := 0 + for count < limit || limit == -1 { + if m.stop() { + return + } + + prev, err := m.sourceDrv.Prev(suint(from)) + if errors.Is(err, os.ErrNotExist) { + // no limit or haven't reached limit, apply "first" migration + if limit == -1 || limit-count > 0 { + firstVersion, err := m.sourceDrv.First() + if err != nil { + ret <- err + return + } + + migr, err := m.newMigration(firstVersion, -1) + if err != nil { + ret <- err + return + } + ret <- migr + go func() { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }() + count++ + } + + if count < limit { + ret <- ErrShortLimit{suint(limit - count)} + } + return + } + if err != nil { + ret <- err + return + } + + migr, err := m.newMigration(suint(from), int(prev)) + if err != nil { + ret <- err + return + } + + ret <- migr + go func() { + if err := migr.Buffer(); err != nil { + m.logErr(err) + } + }() + from = int(prev) + count++ + } +} + +// runMigrations reads *Migration and error from a channel. Any other type +// sent on this channel will result in a panic. Each migration is then +// proxied to the database driver and run against the database. +// Before running a newly received migration it will check if it's supposed +// to stop execution because it might have received a stop signal on the +// GracefulStop channel. +func (m *Migrate) runMigrations(ret <-chan interface{}) error { + for r := range ret { + + if m.stop() { + return nil + } + + switch r := r.(type) { + case error: + return r + + case *Migration: + migr := r + + // set version with dirty state + if err := m.databaseDrv.SetVersion(migr.TargetVersion, true); err != nil { + return err + } + + if migr.Body != nil { + m.logVerbosePrintf("Read and execute %v\n", migr.LogString()) + if err := m.databaseDrv.Run(migr.BufferedBody); err != nil { + return err + } + } + + // set clean state + if err := m.databaseDrv.SetVersion(migr.TargetVersion, false); err != nil { + return err + } + + endTime := time.Now() + readTime := migr.FinishedReading.Sub(migr.StartedBuffering) + runTime := endTime.Sub(migr.FinishedReading) + + // log either verbose or normal + if m.Log != nil { + if m.Log.Verbose() { + m.logPrintf("Finished %v (read %v, ran %v)\n", migr.LogString(), readTime, runTime) + } else { + m.logPrintf("%v (%v)\n", migr.LogString(), readTime+runTime) + } + } + + default: + return fmt.Errorf("unknown type: %T with value: %+v", r, r) + } + } + return nil +} + +// versionExists checks the source if either the up or down migration for +// the specified migration version exists. +func (m *Migrate) versionExists(version uint) (result error) { + // try up migration first + up, _, err := m.sourceDrv.ReadUp(version) + if err == nil { + defer func() { + if errClose := up.Close(); errClose != nil { + result = errors.Join(result, errClose) + } + }() + } + if errors.Is(err, os.ErrExist) { + return nil + } else if !errors.Is(err, os.ErrNotExist) { + return err + } + + // then try down migration + down, _, err := m.sourceDrv.ReadDown(version) + if err == nil { + defer func() { + if errClose := down.Close(); errClose != nil { + result = errors.Join(result, errClose) + } + }() + } + if errors.Is(err, os.ErrExist) { + return nil + } else if !errors.Is(err, os.ErrNotExist) { + return err + } + + err = fmt.Errorf("no migration found for version %d: %w", version, err) + m.logErr(err) + return err +} + +// stop returns true if no more migrations should be run against the database +// because a stop signal was received on the GracefulStop channel. +// Calls are cheap and this function is not blocking. +func (m *Migrate) stop() bool { + if m.isGracefulStop { + return true + } + + select { + case <-m.GracefulStop: + m.isGracefulStop = true + return true + + default: + return false + } +} + +// newMigration is a helper func that returns a *Migration for the +// specified version and targetVersion. +func (m *Migrate) newMigration(version uint, targetVersion int) (*Migration, error) { + var migr *Migration + + if targetVersion >= int(version) { + r, identifier, err := m.sourceDrv.ReadUp(version) + if errors.Is(err, os.ErrNotExist) { + // create "empty" migration + migr, err = NewMigration(nil, "", version, targetVersion) + if err != nil { + return nil, err + } + + } else if err != nil { + return nil, err + + } else { + // create migration from up source + migr, err = NewMigration(r, identifier, version, targetVersion) + if err != nil { + return nil, err + } + } + + } else { + r, identifier, err := m.sourceDrv.ReadDown(version) + if errors.Is(err, os.ErrNotExist) { + // create "empty" migration + migr, err = NewMigration(nil, "", version, targetVersion) + if err != nil { + return nil, err + } + + } else if err != nil { + return nil, err + + } else { + // create migration from down source + migr, err = NewMigration(r, identifier, version, targetVersion) + if err != nil { + return nil, err + } + } + } + + if m.PrefetchMigrations > 0 && migr.Body != nil { + m.logVerbosePrintf("Start buffering %v\n", migr.LogString()) + } else { + m.logVerbosePrintf("Scheduled %v\n", migr.LogString()) + } + + return migr, nil +} + +// lock is a thread safe helper function to lock the database. +// It should be called as late as possible when running migrations. +func (m *Migrate) lock() error { + m.isLockedMu.Lock() + defer m.isLockedMu.Unlock() + + if m.isLocked { + return ErrLocked + } + + // create done channel, used in the timeout goroutine + done := make(chan bool, 1) + defer func() { + done <- true + }() + + // use errchan to signal error back to this context + errchan := make(chan error, 2) + + // start timeout goroutine + timeout := time.After(m.LockTimeout) + go func() { + for { + select { + case <-done: + return + case <-timeout: + errchan <- ErrLockTimeout + return + } + } + }() + + // now try to acquire the lock + go func() { + if err := m.databaseDrv.Lock(); err != nil { + errchan <- err + } else { + errchan <- nil + } + }() + + // wait until we either receive ErrLockTimeout or error from Lock operation + err := <-errchan + if err == nil { + m.isLocked = true + } + return err +} + +// unlock is a thread safe helper function to unlock the database. +// It should be called as early as possible when no more migrations are +// expected to be executed. +func (m *Migrate) unlock() error { + m.isLockedMu.Lock() + defer m.isLockedMu.Unlock() + + if err := m.databaseDrv.Unlock(); err != nil { + // BUG: Can potentially create a deadlock. Add a timeout. + return err + } + + m.isLocked = false + return nil +} + +// unlockErr calls unlock and returns a combined error +// if a prevErr is not nil. +func (m *Migrate) unlockErr(prevErr error) error { + if err := m.unlock(); err != nil { + prevErr = errors.Join(prevErr, err) + } + return prevErr +} + +// logPrintf writes to m.Log if not nil +func (m *Migrate) logPrintf(format string, v ...interface{}) { + if m.Log != nil { + m.Log.Printf(format, v...) + } +} + +// logVerbosePrintf writes to m.Log if not nil. Use for verbose logging output. +func (m *Migrate) logVerbosePrintf(format string, v ...interface{}) { + if m.Log != nil && m.Log.Verbose() { + m.Log.Printf(format, v...) + } +} + +// logErr writes error to m.Log if not nil +func (m *Migrate) logErr(err error) { + if m.Log != nil { + m.Log.Printf("error: %v", err) + } +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/migration.go b/vendor/github.com/golang-migrate/migrate/v4/migration.go new file mode 100644 index 0000000..0e733c6 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/migration.go @@ -0,0 +1,165 @@ +package migrate + +import ( + "bufio" + "errors" + "fmt" + "io" + "time" +) + +// DefaultBufferSize sets the in memory buffer size (in Bytes) for every +// pre-read migration (see DefaultPrefetchMigrations). +var DefaultBufferSize = uint(100000) + +// Migration holds information about a migration. +// It is initially created from data coming from the source and then +// used when run against the database. +type Migration struct { + // Identifier can be any string to help identifying + // the migration in the source. + Identifier string + + // Version is the version of this migration. + Version uint + + // TargetVersion is the migration version after this migration + // has been applied to the database. + // Can be -1, implying that this is a NilVersion. + TargetVersion int + + // Body holds an io.ReadCloser to the source. + Body io.ReadCloser + + // BufferedBody holds an buffered io.Reader to the underlying Body. + BufferedBody io.Reader + + // BufferSize defaults to DefaultBufferSize + BufferSize uint + + // bufferWriter holds an io.WriteCloser and pipes to BufferBody. + // It's an *Closer for flow control. + bufferWriter io.WriteCloser + + // Scheduled is the time when the migration was scheduled/ queued. + Scheduled time.Time + + // StartedBuffering is the time when buffering of the migration source started. + StartedBuffering time.Time + + // FinishedBuffering is the time when buffering of the migration source finished. + FinishedBuffering time.Time + + // FinishedReading is the time when the migration source is fully read. + FinishedReading time.Time + + // BytesRead holds the number of Bytes read from the migration source. + BytesRead int64 +} + +// NewMigration returns a new Migration and sets the body, identifier, +// version and targetVersion. Body can be nil, which turns this migration +// into a "NilMigration". If no identifier is provided, it will default to "". +// targetVersion can be -1, implying it is a NilVersion. +// +// What is a NilMigration? +// Usually each migration version coming from source is expected to have an +// Up and Down migration. This is not a hard requirement though, leading to +// a situation where only the Up or Down migration is present. So let's say +// the user wants to migrate up to a version that doesn't have the actual Up +// migration, in that case we still want to apply the version, but with an empty +// body. We are calling that a NilMigration, a migration with an empty body. +// +// What is a NilVersion? +// NilVersion is a const(-1). When running down migrations and we are at the +// last down migration, there is no next down migration, the targetVersion should +// be nil. Nil in this case is represented by -1 (because type int). +func NewMigration(body io.ReadCloser, identifier string, + version uint, targetVersion int) (*Migration, error) { + tnow := time.Now() + m := &Migration{ + Identifier: identifier, + Version: version, + TargetVersion: targetVersion, + Scheduled: tnow, + } + + if body == nil { + if len(identifier) == 0 { + m.Identifier = "" + } + + m.StartedBuffering = tnow + m.FinishedBuffering = tnow + m.FinishedReading = tnow + return m, nil + } + + br, bw := io.Pipe() + m.Body = body // want to simulate low latency? newSlowReader(body) + m.BufferSize = DefaultBufferSize + m.BufferedBody = br + m.bufferWriter = bw + return m, nil +} + +// String implements string.Stringer and is used in tests. +func (m *Migration) String() string { + return fmt.Sprintf("%v [%v=>%v]", m.Identifier, m.Version, m.TargetVersion) +} + +// LogString returns a string describing this migration to humans. +func (m *Migration) LogString() string { + directionStr := "u" + if m.TargetVersion < int(m.Version) { + directionStr = "d" + } + return fmt.Sprintf("%v/%v %v", m.Version, directionStr, m.Identifier) +} + +// Buffer buffers Body up to BufferSize. +// Calling this function blocks. Call with goroutine. +func (m *Migration) Buffer() (berr error) { + if m.Body == nil { + return nil + } + + m.StartedBuffering = time.Now() + + b := bufio.NewReaderSize(m.Body, int(m.BufferSize)) + + // defer closing buffer writer and body. + defer func() { + // close bufferWriter so Buffer knows that there is no + // more data coming. + if err := m.bufferWriter.Close(); err != nil { + berr = errors.Join(berr, err) + } + + // it's safe to close the Body too. + if err := m.Body.Close(); err != nil { + berr = errors.Join(berr, err) + } + + }() + + // start reading from body, peek won't move the read pointer though + // poor man's solution? + if _, err := b.Peek(int(m.BufferSize)); err != nil && err != io.EOF { + return err + } + + m.FinishedBuffering = time.Now() + + // write to bufferWriter, this will block until + // something starts reading from m.Buffer + n, err := b.WriteTo(m.bufferWriter) + if err != nil { + return err + } + + m.FinishedReading = time.Now() + m.BytesRead = n + + return nil +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/driver.go b/vendor/github.com/golang-migrate/migrate/v4/source/driver.go new file mode 100644 index 0000000..396eabf --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/driver.go @@ -0,0 +1,118 @@ +// Package source provides the Source interface. +// All source drivers must implement this interface, register themselves, +// optionally provide a `WithInstance` function and pass the tests +// in package source/testing. +package source + +import ( + "fmt" + "io" + nurl "net/url" + "sync" +) + +var driversMu sync.RWMutex +var drivers = make(map[string]Driver) + +// Driver is the interface every source driver must implement. +// +// How to implement a source driver? +// 1. Implement this interface. +// 2. Optionally, add a function named `WithInstance`. +// This function should accept an existing source instance and a Config{} struct +// and return a driver instance. +// 3. Add a test that calls source/testing.go:Test() +// 4. Add own tests for Open(), WithInstance() (when provided) and Close(). +// All other functions are tested by tests in source/testing. +// Saves you some time and makes sure all source drivers behave the same way. +// 5. Call Register in init(). +// +// Guidelines: +// - All configuration input must come from the URL string in func Open() +// or the Config{} struct in WithInstance. Don't os.Getenv(). +// - Drivers are supposed to be read only. +// - Ideally don't load any contents (into memory) in Open or WithInstance. +type Driver interface { + // Open returns a new driver instance configured with parameters + // coming from the URL string. Migrate will call this function + // only once per instance. + Open(url string) (Driver, error) + + // Close closes the underlying source instance managed by the driver. + // Migrate will call this function only once per instance. + Close() error + + // First returns the very first migration version available to the driver. + // Migrate will call this function multiple times. + // If there is no version available, it must return os.ErrNotExist. + First() (version uint, err error) + + // Prev returns the previous version for a given version available to the driver. + // Migrate will call this function multiple times. + // If there is no previous version available, it must return os.ErrNotExist. + Prev(version uint) (prevVersion uint, err error) + + // Next returns the next version for a given version available to the driver. + // Migrate will call this function multiple times. + // If there is no next version available, it must return os.ErrNotExist. + Next(version uint) (nextVersion uint, err error) + + // ReadUp returns the UP migration body and an identifier that helps + // finding this migration in the source for a given version. + // If there is no up migration available for this version, + // it must return os.ErrNotExist. + // Do not start reading, just return the ReadCloser! + ReadUp(version uint) (r io.ReadCloser, identifier string, err error) + + // ReadDown returns the DOWN migration body and an identifier that helps + // finding this migration in the source for a given version. + // If there is no down migration available for this version, + // it must return os.ErrNotExist. + // Do not start reading, just return the ReadCloser! + ReadDown(version uint) (r io.ReadCloser, identifier string, err error) +} + +// Open returns a new driver instance. +func Open(url string) (Driver, error) { + u, err := nurl.Parse(url) + if err != nil { + return nil, err + } + + if u.Scheme == "" { + return nil, fmt.Errorf("source driver: invalid URL scheme") + } + + driversMu.RLock() + d, ok := drivers[u.Scheme] + driversMu.RUnlock() + if !ok { + return nil, fmt.Errorf("source driver: unknown driver '%s' (forgotten import?)", u.Scheme) + } + + return d.Open(url) +} + +// Register globally registers a driver. +func Register(name string, driver Driver) { + driversMu.Lock() + defer driversMu.Unlock() + if driver == nil { + panic("Register driver is nil") + } + if _, dup := drivers[name]; dup { + panic("Register called twice for driver " + name) + } + drivers[name] = driver +} + +// List lists the registered drivers +func List() []string { + driversMu.RLock() + defer driversMu.RUnlock() + names := make([]string, 0, len(drivers)) + for n := range drivers { + names = append(names, n) + } + return names +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/errors.go b/vendor/github.com/golang-migrate/migrate/v4/source/errors.go new file mode 100644 index 0000000..93d66e0 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/errors.go @@ -0,0 +1,15 @@ +package source + +import "os" + +// ErrDuplicateMigration is an error type for reporting duplicate migration +// files. +type ErrDuplicateMigration struct { + Migration + os.FileInfo +} + +// Error implements error interface. +func (e ErrDuplicateMigration) Error() string { + return "duplicate migration file: " + e.Name() +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/file/README.md b/vendor/github.com/golang-migrate/migrate/v4/source/file/README.md new file mode 100644 index 0000000..7912eff --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/file/README.md @@ -0,0 +1,4 @@ +# file + +`file:///absolute/path` +`file://relative/path` diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/file/file.go b/vendor/github.com/golang-migrate/migrate/v4/source/file/file.go new file mode 100644 index 0000000..d8b21df --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/file/file.go @@ -0,0 +1,66 @@ +package file + +import ( + nurl "net/url" + "os" + "path/filepath" + + "github.com/golang-migrate/migrate/v4/source" + "github.com/golang-migrate/migrate/v4/source/iofs" +) + +func init() { + source.Register("file", &File{}) +} + +type File struct { + iofs.PartialDriver + url string + path string +} + +func (f *File) Open(url string) (source.Driver, error) { + p, err := parseURL(url) + if err != nil { + return nil, err + } + nf := &File{ + url: url, + path: p, + } + if err := nf.Init(os.DirFS(p), "."); err != nil { + return nil, err + } + return nf, nil +} + +func parseURL(url string) (string, error) { + u, err := nurl.Parse(url) + if err != nil { + return "", err + } + // concat host and path to restore full path + // host might be `.` + p := u.Opaque + if len(p) == 0 { + p = u.Host + u.Path + } + + if len(p) == 0 { + // default to current directory if no path + wd, err := os.Getwd() + if err != nil { + return "", err + } + p = wd + + } else if p[0:1] == "." || p[0:1] != "/" { + // make path absolute if relative + abs, err := filepath.Abs(p) + if err != nil { + return "", err + } + p = abs + } + return p, nil +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/iofs/README.md b/vendor/github.com/golang-migrate/migrate/v4/source/iofs/README.md new file mode 100644 index 0000000..d75b328 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/iofs/README.md @@ -0,0 +1,3 @@ +# iofs + +https://pkg.go.dev/github.com/golang-migrate/migrate/v4/source/iofs diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/iofs/doc.go b/vendor/github.com/golang-migrate/migrate/v4/source/iofs/doc.go new file mode 100644 index 0000000..6b2c862 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/iofs/doc.go @@ -0,0 +1,10 @@ +/* +Package iofs provides the Go 1.16+ io/fs#FS driver. + +It can accept various file systems (like embed.FS, archive/zip#Reader) implementing io/fs#FS. + +This driver cannot be used with Go versions 1.15 and below. + +Also, Opening with a URL scheme is not supported. +*/ +package iofs diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/iofs/iofs.go b/vendor/github.com/golang-migrate/migrate/v4/source/iofs/iofs.go new file mode 100644 index 0000000..a9dc7c4 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/iofs/iofs.go @@ -0,0 +1,175 @@ +//go:build go1.16 + +package iofs + +import ( + "errors" + "fmt" + "io" + "io/fs" + "path" + "strconv" + + "github.com/golang-migrate/migrate/v4/source" +) + +type driver struct { + PartialDriver +} + +// New returns a new Driver from io/fs#FS and a relative path. +func New(fsys fs.FS, path string) (source.Driver, error) { + var i driver + if err := i.Init(fsys, path); err != nil { + return nil, fmt.Errorf("failed to init driver with path %s: %w", path, err) + } + return &i, nil +} + +// Open is part of source.Driver interface implementation. +// Open cannot be called on the iofs passthrough driver. +func (d *driver) Open(url string) (source.Driver, error) { + return nil, errors.New("Open() cannot be called on the iofs passthrough driver") +} + +// PartialDriver is a helper service for creating new source drivers working with +// io/fs.FS instances. It implements all source.Driver interface methods +// except for Open(). New driver could embed this struct and add missing Open() +// method. +// +// To prepare PartialDriver for use Init() function. +type PartialDriver struct { + migrations *source.Migrations + fsys fs.FS + path string +} + +// Init prepares not initialized IoFS instance to read migrations from a +// io/fs#FS instance and a relative path. +func (d *PartialDriver) Init(fsys fs.FS, path string) error { + entries, err := fs.ReadDir(fsys, path) + if err != nil { + return err + } + + ms := source.NewMigrations() + for _, e := range entries { + if e.IsDir() { + continue + } + m, err := source.DefaultParse(e.Name()) + if err != nil { + continue + } + file, err := e.Info() + if err != nil { + return err + } + if !ms.Append(m) { + return source.ErrDuplicateMigration{ + Migration: *m, + FileInfo: file, + } + } + } + + d.fsys = fsys + d.path = path + d.migrations = ms + return nil +} + +// Close is part of source.Driver interface implementation. +// Closes the file system if possible. +func (d *PartialDriver) Close() error { + c, ok := d.fsys.(io.Closer) + if !ok { + return nil + } + return c.Close() +} + +// First is part of source.Driver interface implementation. +func (d *PartialDriver) First() (version uint, err error) { + if version, ok := d.migrations.First(); ok { + return version, nil + } + return 0, &fs.PathError{ + Op: "first", + Path: d.path, + Err: fs.ErrNotExist, + } +} + +// Prev is part of source.Driver interface implementation. +func (d *PartialDriver) Prev(version uint) (prevVersion uint, err error) { + if version, ok := d.migrations.Prev(version); ok { + return version, nil + } + return 0, &fs.PathError{ + Op: "prev for version " + strconv.FormatUint(uint64(version), 10), + Path: d.path, + Err: fs.ErrNotExist, + } +} + +// Next is part of source.Driver interface implementation. +func (d *PartialDriver) Next(version uint) (nextVersion uint, err error) { + if version, ok := d.migrations.Next(version); ok { + return version, nil + } + return 0, &fs.PathError{ + Op: "next for version " + strconv.FormatUint(uint64(version), 10), + Path: d.path, + Err: fs.ErrNotExist, + } +} + +// ReadUp is part of source.Driver interface implementation. +func (d *PartialDriver) ReadUp(version uint) (r io.ReadCloser, identifier string, err error) { + if m, ok := d.migrations.Up(version); ok { + body, err := d.open(path.Join(d.path, m.Raw)) + if err != nil { + return nil, "", err + } + return body, m.Identifier, nil + } + return nil, "", &fs.PathError{ + Op: "read up for version " + strconv.FormatUint(uint64(version), 10), + Path: d.path, + Err: fs.ErrNotExist, + } +} + +// ReadDown is part of source.Driver interface implementation. +func (d *PartialDriver) ReadDown(version uint) (r io.ReadCloser, identifier string, err error) { + if m, ok := d.migrations.Down(version); ok { + body, err := d.open(path.Join(d.path, m.Raw)) + if err != nil { + return nil, "", err + } + return body, m.Identifier, nil + } + return nil, "", &fs.PathError{ + Op: "read down for version " + strconv.FormatUint(uint64(version), 10), + Path: d.path, + Err: fs.ErrNotExist, + } +} + +func (d *PartialDriver) open(path string) (fs.File, error) { + f, err := d.fsys.Open(path) + if err == nil { + return f, nil + } + // Some non-standard file systems may return errors that don't include the path, that + // makes debugging harder. + if !errors.As(err, new(*fs.PathError)) { + err = &fs.PathError{ + Op: "open", + Path: path, + Err: err, + } + } + return nil, err +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/migration.go b/vendor/github.com/golang-migrate/migrate/v4/source/migration.go new file mode 100644 index 0000000..74f6523 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/migration.go @@ -0,0 +1,133 @@ +package source + +import ( + "sort" +) + +// Direction is either up or down. +type Direction string + +const ( + Down Direction = "down" + Up Direction = "up" +) + +// Migration is a helper struct for source drivers that need to +// build the full directory tree in memory. +// Migration is fully independent from migrate.Migration. +type Migration struct { + // Version is the version of this migration. + Version uint + + // Identifier can be any string that helps identifying + // this migration in the source. + Identifier string + + // Direction is either Up or Down. + Direction Direction + + // Raw holds the raw location path to this migration in source. + // ReadUp and ReadDown will use this. + Raw string +} + +// Migrations wraps Migration and has an internal index +// to keep track of Migration order. +type Migrations struct { + index uintSlice + migrations map[uint]map[Direction]*Migration +} + +func NewMigrations() *Migrations { + return &Migrations{ + index: make(uintSlice, 0), + migrations: make(map[uint]map[Direction]*Migration), + } +} + +func (i *Migrations) Append(m *Migration) (ok bool) { + if m == nil { + return false + } + + if i.migrations[m.Version] == nil { + i.migrations[m.Version] = make(map[Direction]*Migration) + } + + // reject duplicate versions + if _, dup := i.migrations[m.Version][m.Direction]; dup { + return false + } + + i.migrations[m.Version][m.Direction] = m + i.buildIndex() + + return true +} + +func (i *Migrations) buildIndex() { + i.index = make(uintSlice, 0, len(i.migrations)) + for version := range i.migrations { + i.index = append(i.index, version) + } + sort.Slice(i.index, func(x, y int) bool { + return i.index[x] < i.index[y] + }) +} + +func (i *Migrations) First() (version uint, ok bool) { + if len(i.index) == 0 { + return 0, false + } + return i.index[0], true +} + +func (i *Migrations) Prev(version uint) (prevVersion uint, ok bool) { + pos := i.findPos(version) + if pos >= 1 && len(i.index) > pos-1 { + return i.index[pos-1], true + } + return 0, false +} + +func (i *Migrations) Next(version uint) (nextVersion uint, ok bool) { + pos := i.findPos(version) + if pos >= 0 && len(i.index) > pos+1 { + return i.index[pos+1], true + } + return 0, false +} + +func (i *Migrations) Up(version uint) (m *Migration, ok bool) { + if _, ok := i.migrations[version]; ok { + if mx, ok := i.migrations[version][Up]; ok { + return mx, true + } + } + return nil, false +} + +func (i *Migrations) Down(version uint) (m *Migration, ok bool) { + if _, ok := i.migrations[version]; ok { + if mx, ok := i.migrations[version][Down]; ok { + return mx, true + } + } + return nil, false +} + +func (i *Migrations) findPos(version uint) int { + if len(i.index) > 0 { + ix := i.index.Search(version) + if ix < len(i.index) && i.index[ix] == version { + return ix + } + } + return -1 +} + +type uintSlice []uint + +func (s uintSlice) Search(x uint) int { + return sort.Search(len(s), func(i int) bool { return s[i] >= x }) +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/source/parse.go b/vendor/github.com/golang-migrate/migrate/v4/source/parse.go new file mode 100644 index 0000000..df085ae --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/source/parse.go @@ -0,0 +1,40 @@ +package source + +import ( + "fmt" + "regexp" + "strconv" +) + +var ( + ErrParse = fmt.Errorf("no match") +) + +var ( + DefaultParse = Parse + DefaultRegex = Regex +) + +// Regex matches the following pattern: +// +// 123_name.up.ext +// 123_name.down.ext +var Regex = regexp.MustCompile(`^([0-9]+)_(.*)\.(` + string(Down) + `|` + string(Up) + `)\.(.*)$`) + +// Parse returns Migration for matching Regex pattern. +func Parse(raw string) (*Migration, error) { + m := Regex.FindStringSubmatch(raw) + if len(m) == 5 { + versionUint64, err := strconv.ParseUint(m[1], 10, 64) + if err != nil { + return nil, err + } + return &Migration{ + Version: uint(versionUint64), + Identifier: m[2], + Direction: Direction(m[3]), + Raw: raw, + }, nil + } + return nil, ErrParse +} diff --git a/vendor/github.com/golang-migrate/migrate/v4/util.go b/vendor/github.com/golang-migrate/migrate/v4/util.go new file mode 100644 index 0000000..c8b6ab7 --- /dev/null +++ b/vendor/github.com/golang-migrate/migrate/v4/util.go @@ -0,0 +1,63 @@ +package migrate + +import ( + "fmt" + nurl "net/url" + "strings" +) + +// MultiError holds multiple errors. +// +// Deprecated: Use stdlib's [errors.Join] et al. instead +// This will be removed in the v5 release. +type MultiError struct { + Errs []error +} + +// NewMultiError returns an error type holding multiple errors. +// +// Deprecated: Use stdlib's [errors.Join] et al. instead +// This will be removed in the v5 release. +func NewMultiError(errs ...error) MultiError { + compactErrs := make([]error, 0) + for _, e := range errs { + if e != nil { + compactErrs = append(compactErrs, e) + } + } + return MultiError{compactErrs} +} + +// Error implements error. Multiple errors are concatenated with 'and's. +func (m MultiError) Error() string { + var strs = make([]string, 0) + for _, e := range m.Errs { + if len(e.Error()) > 0 { + strs = append(strs, e.Error()) + } + } + return strings.Join(strs, " and ") +} + +// suint safely converts int to uint +// see https://goo.gl/wEcqof +// see https://goo.gl/pai7Dr +func suint(n int) uint { + if n < 0 { + panic(fmt.Sprintf("suint(%v) expects input >= 0", n)) + } + return uint(n) +} + +// FilterCustomQuery filters all query values starting with `x-` +func FilterCustomQuery(u *nurl.URL) *nurl.URL { + ux := *u + vx := make(nurl.Values) + for k, v := range ux.Query() { + if len(k) <= 1 || k[0:2] != "x-" { + vx[k] = v + } + } + ux.RawQuery = vx.Encode() + return &ux +} diff --git a/vendor/github.com/stretchr/testify/assert/assertion_compare.go b/vendor/github.com/stretchr/testify/assert/assertion_compare.go index 4d4b4aa..7e19eba 100644 --- a/vendor/github.com/stretchr/testify/assert/assertion_compare.go +++ b/vendor/github.com/stretchr/testify/assert/assertion_compare.go @@ -7,10 +7,13 @@ import ( "time" ) -type CompareType int +// Deprecated: CompareType has only ever been for internal use and has accidentally been published since v1.6.0. Do not use it. +type CompareType = compareResult + +type compareResult int const ( - compareLess CompareType = iota - 1 + compareLess compareResult = iota - 1 compareEqual compareGreater ) @@ -39,7 +42,7 @@ var ( bytesType = reflect.TypeOf([]byte{}) ) -func compare(obj1, obj2 interface{}, kind reflect.Kind) (CompareType, bool) { +func compare(obj1, obj2 interface{}, kind reflect.Kind) (compareResult, bool) { obj1Value := reflect.ValueOf(obj1) obj2Value := reflect.ValueOf(obj2) @@ -325,7 +328,13 @@ func compare(obj1, obj2 interface{}, kind reflect.Kind) (CompareType, bool) { timeObj2 = obj2Value.Convert(timeType).Interface().(time.Time) } - return compare(timeObj1.UnixNano(), timeObj2.UnixNano(), reflect.Int64) + if timeObj1.Before(timeObj2) { + return compareLess, true + } + if timeObj1.Equal(timeObj2) { + return compareEqual, true + } + return compareGreater, true } case reflect.Slice: { @@ -345,7 +354,7 @@ func compare(obj1, obj2 interface{}, kind reflect.Kind) (CompareType, bool) { bytesObj2 = obj2Value.Convert(bytesType).Interface().([]byte) } - return CompareType(bytes.Compare(bytesObj1, bytesObj2)), true + return compareResult(bytes.Compare(bytesObj1, bytesObj2)), true } case reflect.Uintptr: { @@ -381,7 +390,7 @@ func Greater(t TestingT, e1 interface{}, e2 interface{}, msgAndArgs ...interface if h, ok := t.(tHelper); ok { h.Helper() } - return compareTwoValues(t, e1, e2, []CompareType{compareGreater}, "\"%v\" is not greater than \"%v\"", msgAndArgs...) + return compareTwoValues(t, e1, e2, []compareResult{compareGreater}, "\"%v\" is not greater than \"%v\"", msgAndArgs...) } // GreaterOrEqual asserts that the first element is greater than or equal to the second @@ -394,7 +403,7 @@ func GreaterOrEqual(t TestingT, e1 interface{}, e2 interface{}, msgAndArgs ...in if h, ok := t.(tHelper); ok { h.Helper() } - return compareTwoValues(t, e1, e2, []CompareType{compareGreater, compareEqual}, "\"%v\" is not greater than or equal to \"%v\"", msgAndArgs...) + return compareTwoValues(t, e1, e2, []compareResult{compareGreater, compareEqual}, "\"%v\" is not greater than or equal to \"%v\"", msgAndArgs...) } // Less asserts that the first element is less than the second @@ -406,7 +415,7 @@ func Less(t TestingT, e1 interface{}, e2 interface{}, msgAndArgs ...interface{}) if h, ok := t.(tHelper); ok { h.Helper() } - return compareTwoValues(t, e1, e2, []CompareType{compareLess}, "\"%v\" is not less than \"%v\"", msgAndArgs...) + return compareTwoValues(t, e1, e2, []compareResult{compareLess}, "\"%v\" is not less than \"%v\"", msgAndArgs...) } // LessOrEqual asserts that the first element is less than or equal to the second @@ -419,7 +428,7 @@ func LessOrEqual(t TestingT, e1 interface{}, e2 interface{}, msgAndArgs ...inter if h, ok := t.(tHelper); ok { h.Helper() } - return compareTwoValues(t, e1, e2, []CompareType{compareLess, compareEqual}, "\"%v\" is not less than or equal to \"%v\"", msgAndArgs...) + return compareTwoValues(t, e1, e2, []compareResult{compareLess, compareEqual}, "\"%v\" is not less than or equal to \"%v\"", msgAndArgs...) } // Positive asserts that the specified element is positive @@ -431,7 +440,7 @@ func Positive(t TestingT, e interface{}, msgAndArgs ...interface{}) bool { h.Helper() } zero := reflect.Zero(reflect.TypeOf(e)) - return compareTwoValues(t, e, zero.Interface(), []CompareType{compareGreater}, "\"%v\" is not positive", msgAndArgs...) + return compareTwoValues(t, e, zero.Interface(), []compareResult{compareGreater}, "\"%v\" is not positive", msgAndArgs...) } // Negative asserts that the specified element is negative @@ -443,10 +452,10 @@ func Negative(t TestingT, e interface{}, msgAndArgs ...interface{}) bool { h.Helper() } zero := reflect.Zero(reflect.TypeOf(e)) - return compareTwoValues(t, e, zero.Interface(), []CompareType{compareLess}, "\"%v\" is not negative", msgAndArgs...) + return compareTwoValues(t, e, zero.Interface(), []compareResult{compareLess}, "\"%v\" is not negative", msgAndArgs...) } -func compareTwoValues(t TestingT, e1 interface{}, e2 interface{}, allowedComparesResults []CompareType, failMessage string, msgAndArgs ...interface{}) bool { +func compareTwoValues(t TestingT, e1 interface{}, e2 interface{}, allowedComparesResults []compareResult, failMessage string, msgAndArgs ...interface{}) bool { if h, ok := t.(tHelper); ok { h.Helper() } @@ -469,7 +478,7 @@ func compareTwoValues(t TestingT, e1 interface{}, e2 interface{}, allowedCompare return true } -func containsValue(values []CompareType, value CompareType) bool { +func containsValue(values []compareResult, value compareResult) bool { for _, v := range values { if v == value { return true diff --git a/vendor/github.com/stretchr/testify/assert/assertion_format.go b/vendor/github.com/stretchr/testify/assert/assertion_format.go index 3ddab10..1906341 100644 --- a/vendor/github.com/stretchr/testify/assert/assertion_format.go +++ b/vendor/github.com/stretchr/testify/assert/assertion_format.go @@ -104,8 +104,8 @@ func EqualExportedValuesf(t TestingT, expected interface{}, actual interface{}, return EqualExportedValues(t, expected, actual, append([]interface{}{msg}, args...)...) } -// EqualValuesf asserts that two objects are equal or convertible to the same types -// and equal. +// EqualValuesf asserts that two objects are equal or convertible to the larger +// type and equal. // // assert.EqualValuesf(t, uint32(123), int32(123), "error message %s", "formatted") func EqualValuesf(t TestingT, expected interface{}, actual interface{}, msg string, args ...interface{}) bool { @@ -186,7 +186,7 @@ func Eventuallyf(t TestingT, condition func() bool, waitFor time.Duration, tick // assert.EventuallyWithTf(t, func(c *assert.CollectT, "error message %s", "formatted") { // // add assertions as needed; any assertion failure will fail the current tick // assert.True(c, externalValue, "expected 'externalValue' to be true") -// }, 1*time.Second, 10*time.Second, "external state has not changed to 'true'; still false") +// }, 10*time.Second, 1*time.Second, "external state has not changed to 'true'; still false") func EventuallyWithTf(t TestingT, condition func(collect *CollectT), waitFor time.Duration, tick time.Duration, msg string, args ...interface{}) bool { if h, ok := t.(tHelper); ok { h.Helper() @@ -568,6 +568,23 @@ func NotContainsf(t TestingT, s interface{}, contains interface{}, msg string, a return NotContains(t, s, contains, append([]interface{}{msg}, args...)...) } +// NotElementsMatchf asserts that the specified listA(array, slice...) is NOT equal to specified +// listB(array, slice...) ignoring the order of the elements. If there are duplicate elements, +// the number of appearances of each of them in both lists should not match. +// This is an inverse of ElementsMatch. +// +// assert.NotElementsMatchf(t, [1, 1, 2, 3], [1, 1, 2, 3], "error message %s", "formatted") -> false +// +// assert.NotElementsMatchf(t, [1, 1, 2, 3], [1, 2, 3], "error message %s", "formatted") -> true +// +// assert.NotElementsMatchf(t, [1, 2, 3], [1, 2, 4], "error message %s", "formatted") -> true +func NotElementsMatchf(t TestingT, listA interface{}, listB interface{}, msg string, args ...interface{}) bool { + if h, ok := t.(tHelper); ok { + h.Helper() + } + return NotElementsMatch(t, listA, listB, append([]interface{}{msg}, args...)...) +} + // NotEmptyf asserts that the specified object is NOT empty. I.e. not nil, "", false, 0 or either // a slice or a channel with len == 0. // @@ -604,7 +621,16 @@ func NotEqualValuesf(t TestingT, expected interface{}, actual interface{}, msg s return NotEqualValues(t, expected, actual, append([]interface{}{msg}, args...)...) } -// NotErrorIsf asserts that at none of the errors in err's chain matches target. +// NotErrorAsf asserts that none of the errors in err's chain matches target, +// but if so, sets target to that error value. +func NotErrorAsf(t TestingT, err error, target interface{}, msg string, args ...interface{}) bool { + if h, ok := t.(tHelper); ok { + h.Helper() + } + return NotErrorAs(t, err, target, append([]interface{}{msg}, args...)...) +} + +// NotErrorIsf asserts that none of the errors in err's chain matches target. // This is a wrapper for errors.Is. func NotErrorIsf(t TestingT, err error, target error, msg string, args ...interface{}) bool { if h, ok := t.(tHelper); ok { diff --git a/vendor/github.com/stretchr/testify/assert/assertion_forward.go b/vendor/github.com/stretchr/testify/assert/assertion_forward.go index a84e09b..2162908 100644 --- a/vendor/github.com/stretchr/testify/assert/assertion_forward.go +++ b/vendor/github.com/stretchr/testify/assert/assertion_forward.go @@ -186,8 +186,8 @@ func (a *Assertions) EqualExportedValuesf(expected interface{}, actual interface return EqualExportedValuesf(a.t, expected, actual, msg, args...) } -// EqualValues asserts that two objects are equal or convertible to the same types -// and equal. +// EqualValues asserts that two objects are equal or convertible to the larger +// type and equal. // // a.EqualValues(uint32(123), int32(123)) func (a *Assertions) EqualValues(expected interface{}, actual interface{}, msgAndArgs ...interface{}) bool { @@ -197,8 +197,8 @@ func (a *Assertions) EqualValues(expected interface{}, actual interface{}, msgAn return EqualValues(a.t, expected, actual, msgAndArgs...) } -// EqualValuesf asserts that two objects are equal or convertible to the same types -// and equal. +// EqualValuesf asserts that two objects are equal or convertible to the larger +// type and equal. // // a.EqualValuesf(uint32(123), int32(123), "error message %s", "formatted") func (a *Assertions) EqualValuesf(expected interface{}, actual interface{}, msg string, args ...interface{}) bool { @@ -336,7 +336,7 @@ func (a *Assertions) Eventually(condition func() bool, waitFor time.Duration, ti // a.EventuallyWithT(func(c *assert.CollectT) { // // add assertions as needed; any assertion failure will fail the current tick // assert.True(c, externalValue, "expected 'externalValue' to be true") -// }, 1*time.Second, 10*time.Second, "external state has not changed to 'true'; still false") +// }, 10*time.Second, 1*time.Second, "external state has not changed to 'true'; still false") func (a *Assertions) EventuallyWithT(condition func(collect *CollectT), waitFor time.Duration, tick time.Duration, msgAndArgs ...interface{}) bool { if h, ok := a.t.(tHelper); ok { h.Helper() @@ -361,7 +361,7 @@ func (a *Assertions) EventuallyWithT(condition func(collect *CollectT), waitFor // a.EventuallyWithTf(func(c *assert.CollectT, "error message %s", "formatted") { // // add assertions as needed; any assertion failure will fail the current tick // assert.True(c, externalValue, "expected 'externalValue' to be true") -// }, 1*time.Second, 10*time.Second, "external state has not changed to 'true'; still false") +// }, 10*time.Second, 1*time.Second, "external state has not changed to 'true'; still false") func (a *Assertions) EventuallyWithTf(condition func(collect *CollectT), waitFor time.Duration, tick time.Duration, msg string, args ...interface{}) bool { if h, ok := a.t.(tHelper); ok { h.Helper() @@ -1128,6 +1128,40 @@ func (a *Assertions) NotContainsf(s interface{}, contains interface{}, msg strin return NotContainsf(a.t, s, contains, msg, args...) } +// NotElementsMatch asserts that the specified listA(array, slice...) is NOT equal to specified +// listB(array, slice...) ignoring the order of the elements. If there are duplicate elements, +// the number of appearances of each of them in both lists should not match. +// This is an inverse of ElementsMatch. +// +// a.NotElementsMatch([1, 1, 2, 3], [1, 1, 2, 3]) -> false +// +// a.NotElementsMatch([1, 1, 2, 3], [1, 2, 3]) -> true +// +// a.NotElementsMatch([1, 2, 3], [1, 2, 4]) -> true +func (a *Assertions) NotElementsMatch(listA interface{}, listB interface{}, msgAndArgs ...interface{}) bool { + if h, ok := a.t.(tHelper); ok { + h.Helper() + } + return NotElementsMatch(a.t, listA, listB, msgAndArgs...) +} + +// NotElementsMatchf asserts that the specified listA(array, slice...) is NOT equal to specified +// listB(array, slice...) ignoring the order of the elements. If there are duplicate elements, +// the number of appearances of each of them in both lists should not match. +// This is an inverse of ElementsMatch. +// +// a.NotElementsMatchf([1, 1, 2, 3], [1, 1, 2, 3], "error message %s", "formatted") -> false +// +// a.NotElementsMatchf([1, 1, 2, 3], [1, 2, 3], "error message %s", "formatted") -> true +// +// a.NotElementsMatchf([1, 2, 3], [1, 2, 4], "error message %s", "formatted") -> true +func (a *Assertions) NotElementsMatchf(listA interface{}, listB interface{}, msg string, args ...interface{}) bool { + if h, ok := a.t.(tHelper); ok { + h.Helper() + } + return NotElementsMatchf(a.t, listA, listB, msg, args...) +} + // NotEmpty asserts that the specified object is NOT empty. I.e. not nil, "", false, 0 or either // a slice or a channel with len == 0. // @@ -1200,7 +1234,25 @@ func (a *Assertions) NotEqualf(expected interface{}, actual interface{}, msg str return NotEqualf(a.t, expected, actual, msg, args...) } -// NotErrorIs asserts that at none of the errors in err's chain matches target. +// NotErrorAs asserts that none of the errors in err's chain matches target, +// but if so, sets target to that error value. +func (a *Assertions) NotErrorAs(err error, target interface{}, msgAndArgs ...interface{}) bool { + if h, ok := a.t.(tHelper); ok { + h.Helper() + } + return NotErrorAs(a.t, err, target, msgAndArgs...) +} + +// NotErrorAsf asserts that none of the errors in err's chain matches target, +// but if so, sets target to that error value. +func (a *Assertions) NotErrorAsf(err error, target interface{}, msg string, args ...interface{}) bool { + if h, ok := a.t.(tHelper); ok { + h.Helper() + } + return NotErrorAsf(a.t, err, target, msg, args...) +} + +// NotErrorIs asserts that none of the errors in err's chain matches target. // This is a wrapper for errors.Is. func (a *Assertions) NotErrorIs(err error, target error, msgAndArgs ...interface{}) bool { if h, ok := a.t.(tHelper); ok { @@ -1209,7 +1261,7 @@ func (a *Assertions) NotErrorIs(err error, target error, msgAndArgs ...interface return NotErrorIs(a.t, err, target, msgAndArgs...) } -// NotErrorIsf asserts that at none of the errors in err's chain matches target. +// NotErrorIsf asserts that none of the errors in err's chain matches target. // This is a wrapper for errors.Is. func (a *Assertions) NotErrorIsf(err error, target error, msg string, args ...interface{}) bool { if h, ok := a.t.(tHelper); ok { diff --git a/vendor/github.com/stretchr/testify/assert/assertion_order.go b/vendor/github.com/stretchr/testify/assert/assertion_order.go index 00df62a..1d2f718 100644 --- a/vendor/github.com/stretchr/testify/assert/assertion_order.go +++ b/vendor/github.com/stretchr/testify/assert/assertion_order.go @@ -6,7 +6,7 @@ import ( ) // isOrdered checks that collection contains orderable elements. -func isOrdered(t TestingT, object interface{}, allowedComparesResults []CompareType, failMessage string, msgAndArgs ...interface{}) bool { +func isOrdered(t TestingT, object interface{}, allowedComparesResults []compareResult, failMessage string, msgAndArgs ...interface{}) bool { objKind := reflect.TypeOf(object).Kind() if objKind != reflect.Slice && objKind != reflect.Array { return false @@ -50,7 +50,7 @@ func isOrdered(t TestingT, object interface{}, allowedComparesResults []CompareT // assert.IsIncreasing(t, []float{1, 2}) // assert.IsIncreasing(t, []string{"a", "b"}) func IsIncreasing(t TestingT, object interface{}, msgAndArgs ...interface{}) bool { - return isOrdered(t, object, []CompareType{compareLess}, "\"%v\" is not less than \"%v\"", msgAndArgs...) + return isOrdered(t, object, []compareResult{compareLess}, "\"%v\" is not less than \"%v\"", msgAndArgs...) } // IsNonIncreasing asserts that the collection is not increasing @@ -59,7 +59,7 @@ func IsIncreasing(t TestingT, object interface{}, msgAndArgs ...interface{}) boo // assert.IsNonIncreasing(t, []float{2, 1}) // assert.IsNonIncreasing(t, []string{"b", "a"}) func IsNonIncreasing(t TestingT, object interface{}, msgAndArgs ...interface{}) bool { - return isOrdered(t, object, []CompareType{compareEqual, compareGreater}, "\"%v\" is not greater than or equal to \"%v\"", msgAndArgs...) + return isOrdered(t, object, []compareResult{compareEqual, compareGreater}, "\"%v\" is not greater than or equal to \"%v\"", msgAndArgs...) } // IsDecreasing asserts that the collection is decreasing @@ -68,7 +68,7 @@ func IsNonIncreasing(t TestingT, object interface{}, msgAndArgs ...interface{}) // assert.IsDecreasing(t, []float{2, 1}) // assert.IsDecreasing(t, []string{"b", "a"}) func IsDecreasing(t TestingT, object interface{}, msgAndArgs ...interface{}) bool { - return isOrdered(t, object, []CompareType{compareGreater}, "\"%v\" is not greater than \"%v\"", msgAndArgs...) + return isOrdered(t, object, []compareResult{compareGreater}, "\"%v\" is not greater than \"%v\"", msgAndArgs...) } // IsNonDecreasing asserts that the collection is not decreasing @@ -77,5 +77,5 @@ func IsDecreasing(t TestingT, object interface{}, msgAndArgs ...interface{}) boo // assert.IsNonDecreasing(t, []float{1, 2}) // assert.IsNonDecreasing(t, []string{"a", "b"}) func IsNonDecreasing(t TestingT, object interface{}, msgAndArgs ...interface{}) bool { - return isOrdered(t, object, []CompareType{compareLess, compareEqual}, "\"%v\" is not less than or equal to \"%v\"", msgAndArgs...) + return isOrdered(t, object, []compareResult{compareLess, compareEqual}, "\"%v\" is not less than or equal to \"%v\"", msgAndArgs...) } diff --git a/vendor/github.com/stretchr/testify/assert/assertions.go b/vendor/github.com/stretchr/testify/assert/assertions.go index 0b7570f..4e91332 100644 --- a/vendor/github.com/stretchr/testify/assert/assertions.go +++ b/vendor/github.com/stretchr/testify/assert/assertions.go @@ -19,7 +19,9 @@ import ( "github.com/davecgh/go-spew/spew" "github.com/pmezard/go-difflib/difflib" - "gopkg.in/yaml.v3" + + // Wrapper around gopkg.in/yaml.v3 + "github.com/stretchr/testify/assert/yaml" ) //go:generate sh -c "cd ../_codegen && go build && cd - && ../_codegen/_codegen -output-package=assert -template=assertion_format.go.tmpl" @@ -45,6 +47,10 @@ type BoolAssertionFunc func(TestingT, bool, ...interface{}) bool // for table driven tests. type ErrorAssertionFunc func(TestingT, error, ...interface{}) bool +// PanicAssertionFunc is a common function prototype when validating a panic value. Can be useful +// for table driven tests. +type PanicAssertionFunc = func(t TestingT, f PanicTestFunc, msgAndArgs ...interface{}) bool + // Comparison is a custom function that returns true on success and false on failure type Comparison func() (success bool) @@ -496,7 +502,13 @@ func Same(t TestingT, expected, actual interface{}, msgAndArgs ...interface{}) b h.Helper() } - if !samePointers(expected, actual) { + same, ok := samePointers(expected, actual) + if !ok { + return Fail(t, "Both arguments must be pointers", msgAndArgs...) + } + + if !same { + // both are pointers but not the same type & pointing to the same address return Fail(t, fmt.Sprintf("Not same: \n"+ "expected: %p %#v\n"+ "actual : %p %#v", expected, expected, actual, actual), msgAndArgs...) @@ -516,7 +528,13 @@ func NotSame(t TestingT, expected, actual interface{}, msgAndArgs ...interface{} h.Helper() } - if samePointers(expected, actual) { + same, ok := samePointers(expected, actual) + if !ok { + //fails when the arguments are not pointers + return !(Fail(t, "Both arguments must be pointers", msgAndArgs...)) + } + + if same { return Fail(t, fmt.Sprintf( "Expected and actual point to the same object: %p %#v", expected, expected), msgAndArgs...) @@ -524,21 +542,23 @@ func NotSame(t TestingT, expected, actual interface{}, msgAndArgs ...interface{} return true } -// samePointers compares two generic interface objects and returns whether -// they point to the same object -func samePointers(first, second interface{}) bool { +// samePointers checks if two generic interface objects are pointers of the same +// type pointing to the same object. It returns two values: same indicating if +// they are the same type and point to the same object, and ok indicating that +// both inputs are pointers. +func samePointers(first, second interface{}) (same bool, ok bool) { firstPtr, secondPtr := reflect.ValueOf(first), reflect.ValueOf(second) if firstPtr.Kind() != reflect.Ptr || secondPtr.Kind() != reflect.Ptr { - return false + return false, false //not both are pointers } firstType, secondType := reflect.TypeOf(first), reflect.TypeOf(second) if firstType != secondType { - return false + return false, true // both are pointers, but of different types } // compare pointer addresses - return first == second + return first == second, true } // formatUnequalValues takes two values of arbitrary types and returns string @@ -572,8 +592,8 @@ func truncatingFormat(data interface{}) string { return value } -// EqualValues asserts that two objects are equal or convertible to the same types -// and equal. +// EqualValues asserts that two objects are equal or convertible to the larger +// type and equal. // // assert.EqualValues(t, uint32(123), int32(123)) func EqualValues(t TestingT, expected, actual interface{}, msgAndArgs ...interface{}) bool { @@ -615,21 +635,6 @@ func EqualExportedValues(t TestingT, expected, actual interface{}, msgAndArgs .. return Fail(t, fmt.Sprintf("Types expected to match exactly\n\t%v != %v", aType, bType), msgAndArgs...) } - if aType.Kind() == reflect.Ptr { - aType = aType.Elem() - } - if bType.Kind() == reflect.Ptr { - bType = bType.Elem() - } - - if aType.Kind() != reflect.Struct { - return Fail(t, fmt.Sprintf("Types expected to both be struct or pointer to struct \n\t%v != %v", aType.Kind(), reflect.Struct), msgAndArgs...) - } - - if bType.Kind() != reflect.Struct { - return Fail(t, fmt.Sprintf("Types expected to both be struct or pointer to struct \n\t%v != %v", bType.Kind(), reflect.Struct), msgAndArgs...) - } - expected = copyExportedFields(expected) actual = copyExportedFields(actual) @@ -1170,6 +1175,39 @@ func formatListDiff(listA, listB interface{}, extraA, extraB []interface{}) stri return msg.String() } +// NotElementsMatch asserts that the specified listA(array, slice...) is NOT equal to specified +// listB(array, slice...) ignoring the order of the elements. If there are duplicate elements, +// the number of appearances of each of them in both lists should not match. +// This is an inverse of ElementsMatch. +// +// assert.NotElementsMatch(t, [1, 1, 2, 3], [1, 1, 2, 3]) -> false +// +// assert.NotElementsMatch(t, [1, 1, 2, 3], [1, 2, 3]) -> true +// +// assert.NotElementsMatch(t, [1, 2, 3], [1, 2, 4]) -> true +func NotElementsMatch(t TestingT, listA, listB interface{}, msgAndArgs ...interface{}) (ok bool) { + if h, ok := t.(tHelper); ok { + h.Helper() + } + if isEmpty(listA) && isEmpty(listB) { + return Fail(t, "listA and listB contain the same elements", msgAndArgs) + } + + if !isList(t, listA, msgAndArgs...) { + return Fail(t, "listA is not a list type", msgAndArgs...) + } + if !isList(t, listB, msgAndArgs...) { + return Fail(t, "listB is not a list type", msgAndArgs...) + } + + extraA, extraB := diffLists(listA, listB) + if len(extraA) == 0 && len(extraB) == 0 { + return Fail(t, "listA and listB contain the same elements", msgAndArgs) + } + + return true +} + // Condition uses a Comparison to assert a complex condition. func Condition(t TestingT, comp Comparison, msgAndArgs ...interface{}) bool { if h, ok := t.(tHelper); ok { @@ -1488,6 +1526,9 @@ func InEpsilon(t TestingT, expected, actual interface{}, epsilon float64, msgAnd if err != nil { return Fail(t, err.Error(), msgAndArgs...) } + if math.IsNaN(actualEpsilon) { + return Fail(t, "relative error is NaN", msgAndArgs...) + } if actualEpsilon > epsilon { return Fail(t, fmt.Sprintf("Relative error is too high: %#v (expected)\n"+ " < %#v (actual)", epsilon, actualEpsilon), msgAndArgs...) @@ -1611,7 +1652,6 @@ func ErrorContains(t TestingT, theError error, contains string, msgAndArgs ...in // matchRegexp return true if a specified regexp matches a string. func matchRegexp(rx interface{}, str interface{}) bool { - var r *regexp.Regexp if rr, ok := rx.(*regexp.Regexp); ok { r = rr @@ -1619,7 +1659,14 @@ func matchRegexp(rx interface{}, str interface{}) bool { r = regexp.MustCompile(fmt.Sprint(rx)) } - return (r.FindStringIndex(fmt.Sprint(str)) != nil) + switch v := str.(type) { + case []byte: + return r.Match(v) + case string: + return r.MatchString(v) + default: + return r.MatchString(fmt.Sprint(v)) + } } @@ -1872,7 +1919,7 @@ var spewConfigStringerEnabled = spew.ConfigState{ MaxDepth: 10, } -type tHelper interface { +type tHelper = interface { Helper() } @@ -1911,6 +1958,9 @@ func Eventually(t TestingT, condition func() bool, waitFor time.Duration, tick t // CollectT implements the TestingT interface and collects all errors. type CollectT struct { + // A slice of errors. Non-nil slice denotes a failure. + // If it's non-nil but len(c.errors) == 0, this is also a failure + // obtained by direct c.FailNow() call. errors []error } @@ -1919,9 +1969,10 @@ func (c *CollectT) Errorf(format string, args ...interface{}) { c.errors = append(c.errors, fmt.Errorf(format, args...)) } -// FailNow panics. -func (*CollectT) FailNow() { - panic("Assertion failed") +// FailNow stops execution by calling runtime.Goexit. +func (c *CollectT) FailNow() { + c.fail() + runtime.Goexit() } // Deprecated: That was a method for internal usage that should not have been published. Now just panics. @@ -1934,6 +1985,16 @@ func (*CollectT) Copy(TestingT) { panic("Copy() is deprecated") } +func (c *CollectT) fail() { + if !c.failed() { + c.errors = []error{} // Make it non-nil to mark a failure. + } +} + +func (c *CollectT) failed() bool { + return c.errors != nil +} + // EventuallyWithT asserts that given condition will be met in waitFor time, // periodically checking target function each tick. In contrast to Eventually, // it supplies a CollectT to the condition function, so that the condition @@ -1951,14 +2012,14 @@ func (*CollectT) Copy(TestingT) { // assert.EventuallyWithT(t, func(c *assert.CollectT) { // // add assertions as needed; any assertion failure will fail the current tick // assert.True(c, externalValue, "expected 'externalValue' to be true") -// }, 1*time.Second, 10*time.Second, "external state has not changed to 'true'; still false") +// }, 10*time.Second, 1*time.Second, "external state has not changed to 'true'; still false") func EventuallyWithT(t TestingT, condition func(collect *CollectT), waitFor time.Duration, tick time.Duration, msgAndArgs ...interface{}) bool { if h, ok := t.(tHelper); ok { h.Helper() } var lastFinishedTickErrs []error - ch := make(chan []error, 1) + ch := make(chan *CollectT, 1) timer := time.NewTimer(waitFor) defer timer.Stop() @@ -1978,16 +2039,16 @@ func EventuallyWithT(t TestingT, condition func(collect *CollectT), waitFor time go func() { collect := new(CollectT) defer func() { - ch <- collect.errors + ch <- collect }() condition(collect) }() - case errs := <-ch: - if len(errs) == 0 { + case collect := <-ch: + if !collect.failed() { return true } // Keep the errors from the last ended condition, so that they can be copied to t if timeout is reached. - lastFinishedTickErrs = errs + lastFinishedTickErrs = collect.errors tick = ticker.C } } @@ -2049,7 +2110,7 @@ func ErrorIs(t TestingT, err, target error, msgAndArgs ...interface{}) bool { ), msgAndArgs...) } -// NotErrorIs asserts that at none of the errors in err's chain matches target. +// NotErrorIs asserts that none of the errors in err's chain matches target. // This is a wrapper for errors.Is. func NotErrorIs(t TestingT, err, target error, msgAndArgs ...interface{}) bool { if h, ok := t.(tHelper); ok { @@ -2090,6 +2151,24 @@ func ErrorAs(t TestingT, err error, target interface{}, msgAndArgs ...interface{ ), msgAndArgs...) } +// NotErrorAs asserts that none of the errors in err's chain matches target, +// but if so, sets target to that error value. +func NotErrorAs(t TestingT, err error, target interface{}, msgAndArgs ...interface{}) bool { + if h, ok := t.(tHelper); ok { + h.Helper() + } + if !errors.As(err, target) { + return true + } + + chain := buildErrorChainString(err) + + return Fail(t, fmt.Sprintf("Target error should not be in err chain:\n"+ + "found: %q\n"+ + "in chain: %s", target, chain, + ), msgAndArgs...) +} + func buildErrorChainString(err error) string { if err == nil { return "" diff --git a/vendor/github.com/stretchr/testify/assert/yaml/yaml_custom.go b/vendor/github.com/stretchr/testify/assert/yaml/yaml_custom.go new file mode 100644 index 0000000..baa0cc7 --- /dev/null +++ b/vendor/github.com/stretchr/testify/assert/yaml/yaml_custom.go @@ -0,0 +1,25 @@ +//go:build testify_yaml_custom && !testify_yaml_fail && !testify_yaml_default +// +build testify_yaml_custom,!testify_yaml_fail,!testify_yaml_default + +// Package yaml is an implementation of YAML functions that calls a pluggable implementation. +// +// This implementation is selected with the testify_yaml_custom build tag. +// +// go test -tags testify_yaml_custom +// +// This implementation can be used at build time to replace the default implementation +// to avoid linking with [gopkg.in/yaml.v3]. +// +// In your test package: +// +// import assertYaml "github.com/stretchr/testify/assert/yaml" +// +// func init() { +// assertYaml.Unmarshal = func (in []byte, out interface{}) error { +// // ... +// return nil +// } +// } +package yaml + +var Unmarshal func(in []byte, out interface{}) error diff --git a/vendor/github.com/stretchr/testify/assert/yaml/yaml_default.go b/vendor/github.com/stretchr/testify/assert/yaml/yaml_default.go new file mode 100644 index 0000000..b83c6cf --- /dev/null +++ b/vendor/github.com/stretchr/testify/assert/yaml/yaml_default.go @@ -0,0 +1,37 @@ +//go:build !testify_yaml_fail && !testify_yaml_custom +// +build !testify_yaml_fail,!testify_yaml_custom + +// Package yaml is just an indirection to handle YAML deserialization. +// +// This package is just an indirection that allows the builder to override the +// indirection with an alternative implementation of this package that uses +// another implementation of YAML deserialization. This allows to not either not +// use YAML deserialization at all, or to use another implementation than +// [gopkg.in/yaml.v3] (for example for license compatibility reasons, see [PR #1120]). +// +// Alternative implementations are selected using build tags: +// +// - testify_yaml_fail: [Unmarshal] always fails with an error +// - testify_yaml_custom: [Unmarshal] is a variable. Caller must initialize it +// before calling any of [github.com/stretchr/testify/assert.YAMLEq] or +// [github.com/stretchr/testify/assert.YAMLEqf]. +// +// Usage: +// +// go test -tags testify_yaml_fail +// +// You can check with "go list" which implementation is linked: +// +// go list -f '{{.Imports}}' github.com/stretchr/testify/assert/yaml +// go list -tags testify_yaml_fail -f '{{.Imports}}' github.com/stretchr/testify/assert/yaml +// go list -tags testify_yaml_custom -f '{{.Imports}}' github.com/stretchr/testify/assert/yaml +// +// [PR #1120]: https://github.com/stretchr/testify/pull/1120 +package yaml + +import goyaml "gopkg.in/yaml.v3" + +// Unmarshal is just a wrapper of [gopkg.in/yaml.v3.Unmarshal]. +func Unmarshal(in []byte, out interface{}) error { + return goyaml.Unmarshal(in, out) +} diff --git a/vendor/github.com/stretchr/testify/assert/yaml/yaml_fail.go b/vendor/github.com/stretchr/testify/assert/yaml/yaml_fail.go new file mode 100644 index 0000000..e78f7df --- /dev/null +++ b/vendor/github.com/stretchr/testify/assert/yaml/yaml_fail.go @@ -0,0 +1,18 @@ +//go:build testify_yaml_fail && !testify_yaml_custom && !testify_yaml_default +// +build testify_yaml_fail,!testify_yaml_custom,!testify_yaml_default + +// Package yaml is an implementation of YAML functions that always fail. +// +// This implementation can be used at build time to replace the default implementation +// to avoid linking with [gopkg.in/yaml.v3]: +// +// go test -tags testify_yaml_fail +package yaml + +import "errors" + +var errNotImplemented = errors.New("YAML functions are not available (see https://pkg.go.dev/github.com/stretchr/testify/assert/yaml)") + +func Unmarshal([]byte, interface{}) error { + return errNotImplemented +} diff --git a/vendor/golang.org/x/crypto/bcrypt/bcrypt.go b/vendor/golang.org/x/crypto/bcrypt/bcrypt.go index dc93118..3e7f8df 100644 --- a/vendor/golang.org/x/crypto/bcrypt/bcrypt.go +++ b/vendor/golang.org/x/crypto/bcrypt/bcrypt.go @@ -50,7 +50,7 @@ func (ih InvalidHashPrefixError) Error() string { type InvalidCostError int func (ic InvalidCostError) Error() string { - return fmt.Sprintf("crypto/bcrypt: cost %d is outside allowed range (%d,%d)", int(ic), MinCost, MaxCost) + return fmt.Sprintf("crypto/bcrypt: cost %d is outside allowed inclusive range %d..%d", int(ic), MinCost, MaxCost) } const ( diff --git a/vendor/golang.org/x/text/language/parse.go b/vendor/golang.org/x/text/language/parse.go index 4d57222..053336e 100644 --- a/vendor/golang.org/x/text/language/parse.go +++ b/vendor/golang.org/x/text/language/parse.go @@ -59,7 +59,7 @@ func (c CanonType) Parse(s string) (t Tag, err error) { if changed { tt.RemakeString() } - return makeTag(tt), err + return makeTag(tt), nil } // Compose creates a Tag from individual parts, which may be of type Tag, Base, diff --git a/vendor/modules.txt b/vendor/modules.txt index f46cdd1..b052128 100644 --- a/vendor/modules.txt +++ b/vendor/modules.txt @@ -2,18 +2,24 @@ ## explicit; go 1.20 filippo.io/edwards25519 filippo.io/edwards25519/field -# github.com/davecgh/go-spew v1.1.1 +# github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc ## explicit github.com/davecgh/go-spew/spew # github.com/doublehops/go-common v0.0.0-20230910011642-8556bd635e3f ## explicit; go 1.21 github.com/doublehops/go-common/str -# github.com/doublehops/go-migration v0.0.3 -## explicit; go 1.16 -github.com/doublehops/go-migration/helpers # github.com/go-sql-driver/mysql v1.8.1 ## explicit; go 1.18 github.com/go-sql-driver/mysql +# github.com/golang-migrate/migrate/v4 v4.19.1 +## explicit; go 1.24.0 +github.com/golang-migrate/migrate/v4 +github.com/golang-migrate/migrate/v4/database +github.com/golang-migrate/migrate/v4/database/mysql +github.com/golang-migrate/migrate/v4/internal/url +github.com/golang-migrate/migrate/v4/source +github.com/golang-migrate/migrate/v4/source/file +github.com/golang-migrate/migrate/v4/source/iofs # github.com/jinzhu/copier v0.4.0 ## explicit; go 1.13 github.com/jinzhu/copier @@ -30,18 +36,19 @@ github.com/julienschmidt/httprouter # github.com/mythrnr/httprouter-group v0.9.1 ## explicit; go 1.20 github.com/mythrnr/httprouter-group -# github.com/pmezard/go-difflib v1.0.0 +# github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 ## explicit github.com/pmezard/go-difflib/difflib -# github.com/stretchr/testify v1.9.0 +# github.com/stretchr/testify v1.10.0 ## explicit; go 1.17 github.com/stretchr/testify/assert -# golang.org/x/crypto v0.27.0 -## explicit; go 1.20 +github.com/stretchr/testify/assert/yaml +# golang.org/x/crypto v0.45.0 +## explicit; go 1.24.0 golang.org/x/crypto/bcrypt golang.org/x/crypto/blowfish -# golang.org/x/text v0.18.0 -## explicit; go 1.18 +# golang.org/x/text v0.31.0 +## explicit; go 1.24.0 golang.org/x/text/cases golang.org/x/text/internal golang.org/x/text/internal/language