Skip to content

feat: save file metadata as sidecar file#210

Open
mz289 wants to merge 1 commit into
krau:mainfrom
mz289:feat/save-metadata
Open

feat: save file metadata as sidecar file#210
mz289 wants to merge 1 commit into
krau:mainfrom
mz289:feat/save-metadata

Conversation

@mz289
Copy link
Copy Markdown

@mz289 mz289 commented May 6, 2026

No description provided.

@krau
Copy link
Copy Markdown
Owner

krau commented May 6, 2026

hi, thank you for your contribution. Could you explain the purpose of implementing this feature? I also noticed that this feature requires changes to the existing storage interface definition, so I need some time to consider this idea and review the code.

@mz289
Copy link
Copy Markdown
Author

mz289 commented May 6, 2026

Added a configuration option to generate a .meta.json sidecar metadata file when saving files. If only the file is stored, it is difficult to trace its origin later, so the metadata is saved to identify the source and the associated message.

The storage interface has been modified to return the final file path. Since the storage layer includes deduplication logic for duplicate names (e.g., file_1.ext, file_2.ext), the path provided by the caller may not be the actual saved path.

This is my first time contributing to a Go project, and I’m not sure whether this approach aligns with common practices. Any feedback or suggestions would be appreciated.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces optional .meta.json sidecar files to persist Telegram message-derived metadata alongside saved files, and updates the storage layer to return the final saved path so callers can correctly attach sidecars (especially when storages auto-rename on collisions).

Changes:

  • Add save_metadata config flag and a new pkg/metadata module to build and serialize Telegram message metadata into .meta.json sidecars.
  • Change storage.Storage.Save to return (actualPath string, err error) and update all storage implementations/call sites accordingly.
  • Extend transfer logic to copy an existing metadata sidecar when transferring files between storages.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
storage/webdav/webdav.go Update Save to return actual saved path (supports collision renames).
storage/telegram/telegram.go Update Save signature and propagate actual path result.
storage/storage.go Storage interface Save now returns (string, error).
storage/s3/s3.go Return actual saved key (incl. rename) from Save.
storage/s3/s3_test.go Adjust tests for new Save return value; assert rename path.
storage/rclone/rclone.go Update Save signature to return chosen candidate path.
storage/minio/client.go Update Save signature and return actual saved key (incl. rename).
storage/minio/client_stub.go Update stub Save signature for no_minio builds.
storage/local/local.go Update Save to return actual saved path (incl. rename).
storage/alist/alist.go Update Save signature and return actual path (incl. rename).
pkg/metadata/metadata.go New metadata model + builder + JSON serializer + .meta.json suffix.
docs/content/zh/deployment/configuration/_index.md Document save_metadata option (ZH).
docs/content/en/deployment/configuration/_index.md Document save_metadata option (EN).
core/tasks/ytdlp/task_test.go Update mock storage Save signature for tests.
core/tasks/ytdlp/execute.go Update call sites for new Save return signature.
core/tasks/transfer/execute.go Capture actual path and attempt to transfer metadata sidecar.
core/tasks/tfile/tftask.go Pre-build metadata JSON (optional) and add metadata save helper.
core/tasks/tfile/stream.go Capture actual saved path in stream mode; save metadata sidecar.
core/tasks/tfile/execute.go Capture actual saved path; save metadata sidecar after upload.
core/tasks/telegraph/execute.go Update Save call sites for new signature.
core/tasks/parsed/execute.go Update Save call sites for new signature.
core/tasks/directlinks/execute.go Update Save call sites for new signature.
core/tasks/batchtfile/task.go Add per-element metadata JSON + save helper; update factory signature.
core/tasks/batchtfile/execute.go Capture actual saved path per element; save metadata sidecar.
core/tasks/aria2dl/task_test.go Update mock storage Save signature for tests.
core/tasks/aria2dl/execute.go Update call sites for new Save return signature.
config/viper.go Add SaveMetadata field and default config value.
config.example.toml Add save_metadata example setting.
common/utils/tgutil/chatinfo.go Add helper to derive chat title/username from ext/entities/API.
cmd/upload/cmd.go Update CLI upload path to ignore returned Save value.
client/bot/handlers/utils/shortcut/tftask.go Pass ctx into batch task element creation (for metadata build).
api/factory.go Pass factory ctx into batch task element creation (for metadata build).
Comments suppressed due to low confidence (1)

storage/telegram/telegram.go:146

  • In the split-upload branch, Save() returns the value from splitUpload() (currently just filename). That loses the original storagePath context (e.g., chat ID/dir segments) and makes actualPath inconsistent with the normal branch (which returns originalPath). This breaks sidecar metadata saving/transfer because the .meta.json path may be written to the wrong chat/path. Consider having the split-upload path return the same logical storage path as the non-split path (e.g., originalPath) while still propagating any upload error.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +159 to +165
reader, _, err := readable.OpenFile(ctx, metaSourcePath)
if err != nil {
log.FromContext(ctx).Debugf("Failed to open metadata file %s, skipping: %s", metaSourcePath, err)
return nil
}
defer reader.Close()
_, err = elem.TargetStorage.Save(ctx, reader, actualPath+metadata.MetaSuffix)
Comment on lines +19 to +26
if extCtx.Entities != nil {
if t, u, found := lookupEntities(extCtx.Entities, chatID); found {
return t, u
}
}

return fetchFromAPI(extCtx, peer, chatID)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants