Bug fix/cleanup by Rhaqim · Pull Request #13 · Rhaqim/buckt

Rhaqim · 2026-04-09T17:38:27Z

Bug Fixes

Folder traversal logic fixed
Max file size initailisation
Dependency updates
Example updates
Migration logic implemented
Content-type auto-detected from bytes
MoveFolder/RenameFolder updates child paths
Soft-delete mangles name/path
Cloud backend now has ping

Copilot

Pull request overview

This PR is a broad “bug fix/cleanup” update across the core Buckt library, backends, repositories, and the web client, aiming to improve safety (path traversal + error wrapping), correctness (soft-delete restore + descendant path updates), and operability (migration backend + backend ping), while refreshing dependencies and examples.

Changes:

Adds safer error-wrapping helpers, max file size enforcement, and content-type detection from bytes.
Implements/extends soft-delete + restore flows and adds paginated list repository methods.
Introduces/expands backend migration support (dual-write + background migration) and updates the web UI/handlers accordingly.

Reviewed changes

Copilot reviewed 51 out of 58 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
pkg/response/response.go	Adds `WrapErrorSafe` and cleans up error response formatting.
pkg/response/response_test.go	Updates tests and adds coverage for `WrapErrorSafe`.
pkg/logger/logger.go	Fixes/clarifies stdout vs discard selection logic comment/flow.
pkg/fileutil/fileutil.go	Adds multipart file processing helper (reads to memory).
internal/utils/file.go	Adds exec path validation for preview generation commands.
internal/service/folder.go	Attempts restore of soft-deleted folders on unique constraint violations.
internal/service/file.go	Adds max file size enforcement, content-type detection, restore-on-unique, hash change, cache decoding tweaks.
internal/service/file_test.go	Updates constructor usage for new max-file-size parameter.
internal/repository/folder.go	Adds folder restore + pagination, updates move/rename to also update descendant paths, implements name/path mangling on delete.
internal/repository/file.go	Adds file restore + pagination, implements name/path mangling on delete.
internal/model/user.go	Removes password field from `UserModel`.
internal/model/migration.go	Adds explicit `TableName()` for migration table.
internal/model/file.go	Adds `UserID` field to file model.
internal/model/db.go	Adds a `Pagination` helper type and methods.
internal/mocks/fs_old.go	Removes old filesystem mock file.
internal/mocks/folder_repo.go	Updates folder repo mock for new methods.
internal/mocks/file_repo.go	Updates file repo mock for new pagination method.
internal/domain/repository.go	Extends repository interfaces with pagination + restore methods.
internal/database/database.go	Updates `Migrate` signature and optionally migrates `MigrationModel`.
internal/database/database_test.go	Updates migration test for new `Migrate(true)` signature.
internal/backend/migrate.go	Implements migration backend behavior, background migration, and progress tracking.
internal/backend/localfs.go	Adds path traversal detection by constraining resolved paths under media dir.
go.work	Bumps workspace Go version.
go.sum	Updates dependency sums.
go.mod	Bumps Go version and updates core dependencies.
example/Makefile	Adds Cloudflare target.
example/go.mod	Bumps example module Go version and buckt dependency version.
example/cloud/migration/main.go	Adds backend ping before running migration example.
example/cloud/gcp/main.go	Adds backend ping before running GCP example.
example/cloud/cloudflare/main.go	Adds backend ping before running Cloudflare example.
example/cloud/azure/main.go	Adds backend ping before running Azure example.
example/cloud/aws/main.go	Adds backend ping before running AWS example.
cloud/gcp/go.sum	Updates GCP module dependency sums.
cloud/gcp/go.mod	Bumps Go version and updates GCP module dependencies.
cloud/gcp/backend.go	Adds `Ping()` and improves upload failure handling.
cloud/azure/go.sum	Updates Azure module dependency sums.
cloud/azure/go.mod	Bumps Go version and updates Azure module dependencies.
cloud/azure/backend.go	Adds `Ping()`, makes `Stat` nil-safe, improves Move polling + timeout.
cloud/aws/go.sum	Updates AWS module dependency sums.
cloud/aws/go.mod	Updates AWS SDK dependencies.
cloud/aws/backend.go	Adds `Ping()` and makes Move delete synchronous/best-effort.
client/web/templates/folders.html	Redesigns folder cards, adds context menu + drag/drop handlers.
client/web/templates/files.html	Redesigns file cards, adds previews and context menu.
client/web/templates/dashboard.html	Major UX refresh, modals, context menus, drag/drop move via forms.
client/web/templates/base.html	Updates layout/branding and adds new styles for cards/drag/drop.
client/web/router/router.go	Switches rename/move endpoints to POST routes.
client/web/middleware/middleware.go	Minor control-flow cleanup in middleware.
client/web/go.sum	Updates web client module sums.
client/web/go.mod	Bumps Go version and updates web module dependencies.
client/web/buckt_templates.go	Adds `formatSize` template helper and registers it.
client/web/app/web.go	Improves view model, HTMX delete responses, implements MoveFile, adds safe Content-Disposition helper, switches to pkg/fileutil.
client/web/app/api.go	Implements GetSubFolders/GetDescendants, adds safe Content-Disposition helper, switches to pkg/fileutil.
buckt.go	Wires max file size into services, updates migrate call, improves reader upload path, improves backend resolution.
buckt_test.go	Updates backend resolution tests to use PlaceholderBackend and new resolver.
buckt_conf.go	Adds max file size config + `WithMaxFileSize`, introduces PlaceholderBackend for local backend selection.
.gitignore	Adds ignores for sqlite/logs/media/bin and improves OS file ignores.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 57 out of 64 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (2)

client/web/app/web.go:218

Upload handlers read the entire multipart file into memory via fileutil.ProcessFile(file) before any size enforcement. Even though FileService enforces maxFileSize, the server can still OOM on large uploads. Use ProcessFileWithLimit (and return 413 on overflow), ideally using the configured max size (or at least buckt.DefaultMaxFileSize) so the request is bounded before buffering.

	files := form.File["files"]

	// Loop through each file
	for _, file := range files {
		// Save each file (this example just prints)
		fileName, fileByte, err := fileutil.ProcessFile(file)
		if err != nil {
			c.AbortWithStatusJSON(500, response.WrapError("failed to process file", err))
			return
		}

		_, err = svc.client.UploadFile(user_id, folderID, fileName, file.Header.Get("Content-Type"), fileByte)
		if err != nil {

client/web/app/api.go:246

This API endpoint buffers the entire uploaded file into memory with fileutil.ProcessFile(file) and does not apply any max-size limit. This can cause excessive memory usage / OOM before downstream size checks run. Prefer ProcessFileWithLimit (return 413 when exceeded) and wire it to the same max file size configuration used by the Buckt client.

// UploadFile implements domain.APIService.
func (svc *APIService) UploadFile(c *gin.Context) {
	// get the user_id from the context
	user_id := c.GetString("owner_id")

	file, err := c.FormFile("file")
	if err != nil {
		c.AbortWithStatusJSON(400, response.Error("file is required", err.Error()))
		return
	}

	// get the parent_id from the request
	parentID := c.PostForm("parent_id")
	if parentID == "" {
		c.AbortWithStatusJSON(400, response.Error("parent_id is required", ""))
		return
	}

	// Read file from request
	fileName, fileByte, err := fileutil.ProcessFile(file)
	if err != nil {
		c.AbortWithStatusJSON(500, response.WrapError("failed to process file", err))
		return
	}

	fileID, err := svc.client.UploadFile(user_id, parentID, fileName, file.Header.Get("Content-Type"), fileByte)
	if err != nil {
		c.AbortWithStatusJSON(500, response.WrapError("failed to create file", err))
		return
	}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 57 out of 64 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 58 out of 65 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (2)

client/web/app/web.go:214

UploadFile reads each multipart file fully into memory via fileutil.ProcessFile (no limit), before buckt enforces MaxFileSize. This allows very large uploads to cause high memory usage / OOM even if they’ll later be rejected. Prefer streaming to UploadFileFromReader (using file.Open()), or call ProcessFileWithLimit with the configured max size so the request is bounded before allocation.

	files := form.File["files"]

	// Loop through each file
	for _, file := range files {
		// Save each file (this example just prints)
		fileName, fileByte, err := fileutil.ProcessFile(file)
		if err != nil {
			c.AbortWithStatusJSON(500, response.WrapError("failed to process file", err))
			return
		}

		_, err = svc.client.UploadFile(user_id, folderID, fileName, file.Header.Get("Content-Type"), fileByte)
		if err != nil {

client/web/app/api.go:241

This API upload path also reads the entire multipart file into memory (fileutil.ProcessFile) with no size limit, which can defeat MaxFileSize protections and enable memory exhaustion on large uploads. Prefer streaming to UploadFileFromReader or using ProcessFileWithLimit with a max size before calling UploadFile.

// UploadFile implements domain.APIService.
func (svc *APIService) UploadFile(c *gin.Context) {
	// get the user_id from the context
	user_id := c.GetString("owner_id")

	file, err := c.FormFile("file")
	if err != nil {
		c.AbortWithStatusJSON(400, response.Error("file is required", err.Error()))
		return
	}

	// get the parent_id from the request
	parentID := c.PostForm("parent_id")
	if parentID == "" {
		c.AbortWithStatusJSON(400, response.Error("parent_id is required", ""))
		return
	}

	// Read file from request
	fileName, fileByte, err := fileutil.ProcessFile(file)
	if err != nil {
		c.AbortWithStatusJSON(500, response.WrapError("failed to process file", err))
		return
	}

	fileID, err := svc.client.UploadFile(user_id, parentID, fileName, file.Header.Get("Content-Type"), fileByte)
	if err != nil {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 58 out of 65 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

client/web/app/web.go:218

UploadFile reads each multipart upload fully into memory via fileutil.ProcessFile (io.ReadAll) and then calls client.UploadFile with a []byte. With the new MaxFileSize enforcement, this still allows a large upload to allocate unbounded memory before the size check triggers, which is a DoS risk. Consider streaming to buckt via UploadFileFromReader(…)/UploadFileFromReaderContext (so the core size limiting logic can apply during reads) or using ProcessFileWithLimit with the configured max size and rejecting before reading the whole body.

// UploadFile implements domain.WebService.
func (svc *WebService) UploadFile(c *gin.Context) {
	user_id := c.GetString("owner_id")

	folderID := c.PostForm("folder_id")
	if folderID == "" {
		c.AbortWithStatusJSON(400, gin.H{"error": "folder_id is required"})
		return
	}

	// Handle file upload
	form, err := c.MultipartForm()
	if err != nil {
		c.AbortWithStatusJSON(400, gin.H{"error": "Invalid form data"})
		return
	}

	files := form.File["files"]

	// Loop through each file
	for _, file := range files {
		// Save each file (this example just prints)
		fileName, fileByte, err := fileutil.ProcessFile(file)
		if err != nil {
			c.AbortWithStatusJSON(500, response.WrapError("failed to process file", err))
			return
		}

		_, err = svc.client.UploadFile(user_id, folderID, fileName, file.Header.Get("Content-Type"), fileByte)
		if err != nil {
			c.AbortWithStatusJSON(500, response.WrapError("failed to create file", err))
			return
		}

client/web/app/api.go:244

API UploadFile fully buffers the uploaded multipart file into memory via fileutil.ProcessFile (io.ReadAll) before calling client.UploadFile. This bypasses the new MaxFileSize protections until after the allocation/read completes and can allow memory exhaustion on large requests. Prefer streaming into buckt with UploadFileFromReader(…)/UploadFileFromReaderContext, or enforce an early bound with ProcessFileWithLimit using the same max size limit configured for the client.

// UploadFile implements domain.APIService.
func (svc *APIService) UploadFile(c *gin.Context) {
	// get the user_id from the context
	user_id := c.GetString("owner_id")

	file, err := c.FormFile("file")
	if err != nil {
		c.AbortWithStatusJSON(400, response.Error("file is required", err.Error()))
		return
	}

	// get the parent_id from the request
	parentID := c.PostForm("parent_id")
	if parentID == "" {
		c.AbortWithStatusJSON(400, response.Error("parent_id is required", ""))
		return
	}

	// Read file from request
	fileName, fileByte, err := fileutil.ProcessFile(file)
	if err != nil {
		c.AbortWithStatusJSON(500, response.WrapError("failed to process file", err))
		return
	}

	fileID, err := svc.client.UploadFile(user_id, parentID, fileName, file.Header.Get("Content-Type"), fileByte)
	if err != nil {
		c.AbortWithStatusJSON(500, response.WrapError("failed to create file", err))
		return
	}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 58 out of 65 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

client/web/app/api.go:238

fileutil.ProcessFile buffers the entire upload in memory without a size cap. Even with the new max file size enforcement in Buckt, this handler can allocate arbitrarily large payloads before the limit check happens, which is a DoS risk. Consider switching to ProcessFileWithLimit and/or streaming uploads (UploadFileFromReader) with an io.LimitReader at the HTTP layer.

	// Read file from request
	fileName, fileByte, err := fileutil.ProcessFile(file)
	if err != nil {
		c.AbortWithStatusJSON(500, response.WrapError("failed to process file", err))
		return
	}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 58 out of 65 changed files in this pull request and generated 8 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 59 out of 66 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 59 out of 66 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 59 out of 66 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 59 out of 66 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+func (db *DB) cleanupLegacySoftDeletes() error {
+	hasFileCol := db.Migrator().HasColumn(&model.FileModel{}, "deleted_at")
+	hasFolderCol := db.Migrator().HasColumn(&model.FolderModel{}, "deleted_at")
+


+		// Refuse oversized batches before doing any work — protects against
+		// runaway operations on large folder trees that would tie up the DB
+		// transaction and storage backend with thousands of sequential moves.
+		if f.maxTrashBatchSize > 0 && len(fileMoves) > f.maxTrashBatchSize {
+			return fmt.Errorf("folder contains %d files which exceeds the max trash batch size of %d; delete subtrees first", len(fileMoves), f.maxTrashBatchSize)
+		}


 	// Create the file
-	err = f.repo.Create(ctx, file)
-	if err != nil {
-		if err.Error() == "UNIQUE constraint failed: file_models.name, file_models.parent_id" {
-			file, err = f.repo.RestoreFile(ctx, file.ParentID, file.Name)
-			if err != nil {
-				return "", f.logger.WrapError("failed to restore file", err)
-			}
-		} else {
-			return "", f.logger.WrapError("failed to create file", err)
-		}
-	} else {
-		// Write the file to the file system
-		if err := f.fileBackend.Put(ctx, file.Path, file_data); err != nil {
-			return "", err
-		}
+	if err = f.repo.Create(ctx, file); err != nil {
+		return "", f.logger.WrapError("failed to create file", err)
+	}
+
+	// Write the file to the backend
+	if err := f.fileBackend.Put(ctx, file.Path, file_data); err != nil {
+		return "", err
 	}


Rhaqim added 10 commits April 9, 2026 15:52

claude bug fixes, implementation for migration and cleanup

2ef981a

fix issue with resolving the backend, cleaned up the cloud backends

354ea4e

ui changes

cfaab3f

fix toggle menu on media and folders

1828fe1

delete and restore folder

0147b11

deps update

6adff2f

deps update

1731855

cloud deps update

3ab2873

compatibility check

584b7f5

tidy cloud, update examples

76fb573

Rhaqim requested a review from Copilot April 9, 2026 17:38

Copilot started reviewing on behalf of Rhaqim April 9, 2026 17:39 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

Rhaqim added 2 commits April 10, 2026 12:25

improvments

8cc1743

for web added the delete permamnently

5d745ca

Rhaqim requested a review from Copilot April 10, 2026 11:35

Copilot started reviewing on behalf of Rhaqim April 10, 2026 11:35 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Rhaqim and others added 4 commits April 10, 2026 12:48

with backend and with migration

60076e7

use trash constant

42446b7

Potential fix for pull request finding

5ea76cd

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

resolve conversations

421282d

Rhaqim requested a review from Copilot April 10, 2026 12:05

Copilot started reviewing on behalf of Rhaqim April 10, 2026 12:06 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Comment thread buckt.go

Comment thread internal/repository/file.go Outdated

Comment thread internal/repository/folder.go

Comment thread internal/repository/folder.go Outdated

Rhaqim added 3 commits April 10, 2026 13:16

readme update

a6d1020

clean check for if folder exists

cc31625

backfill postgres compatibility issues

dd94284

Rhaqim requested a review from Copilot April 10, 2026 12:36

Copilot started reviewing on behalf of Rhaqim April 10, 2026 12:36 View session

Copilot AI reviewed Apr 11, 2026

View reviewed changes

Comment thread buckt.go Outdated

Comment thread pkg/response/response.go Outdated

Comment thread internal/mocks/folder_repo.go

Comment thread internal/service/file.go Outdated

Comment thread internal/service/folder.go Outdated

Comment thread internal/repository/file.go Outdated

added more tests

58dd8fa

Rhaqim requested a review from Copilot April 11, 2026 11:47

Copilot started reviewing on behalf of Rhaqim April 11, 2026 11:47 View session

Copilot AI reviewed Apr 11, 2026

View reviewed changes

Comment thread internal/service/file.go

Comment thread internal/service/file.go Outdated

Comment thread internal/service/folder.go Outdated

commiting to the filespace changes along with the database changes

86f53c9

Rhaqim requested a review from Copilot April 13, 2026 02:48

Copilot started reviewing on behalf of Rhaqim April 13, 2026 02:49 View session

Copilot AI reviewed Apr 13, 2026

View reviewed changes

Comment thread internal/repository/file.go Outdated

Comment thread internal/database/database.go Outdated

Comment thread client/web/app/web.go

Comment thread internal/mocks/folder_repo.go Outdated

Comment thread internal/mocks/file_repo.go Outdated

resolve conversations

a3aeb33

Rhaqim requested a review from Copilot April 13, 2026 03:02

Copilot started reviewing on behalf of Rhaqim April 13, 2026 03:03 View session

Copilot AI reviewed Apr 13, 2026

View reviewed changes

Rhaqim added 2 commits April 13, 2026 12:33

resolve conversations

e1d6011

flatname space flag

0a53b75

Rhaqim requested a review from Copilot April 13, 2026 11:42

Copilot started reviewing on behalf of Rhaqim April 13, 2026 11:43 View session

Copilot AI reviewed Apr 13, 2026

View reviewed changes

Comment thread internal/repository/file.go Outdated

Comment thread internal/backend/migrate.go

Comment thread internal/service/folder.go

resolve conversations

d952064

Rhaqim requested a review from Copilot April 13, 2026 21:55

Copilot started reviewing on behalf of Rhaqim April 13, 2026 21:55 View session

Copilot AI reviewed Apr 13, 2026

View reviewed changes

Comment thread internal/service/folder.go

Comment thread internal/repository/file.go Outdated

Comment thread internal/repository/file.go Outdated

Comment thread internal/backend/migrate.go

go fmt

fa86b15

Rhaqim requested a review from Copilot April 14, 2026 12:27

Copilot started reviewing on behalf of Rhaqim April 14, 2026 12:27 View session

Copilot AI reviewed Apr 14, 2026

View reviewed changes

Comment thread internal/repository/file.go Outdated

Comment thread internal/backend/migrate.go

Comment thread internal/service/folder.go

resolve conversations

fa48b78

Rhaqim requested a review from Copilot April 14, 2026 12:35

Copilot started reviewing on behalf of Rhaqim April 14, 2026 12:36 View session

Copilot AI reviewed Apr 14, 2026

View reviewed changes

Conversation

Rhaqim commented Apr 9, 2026

Bug Fixes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!