Skip to content

feat: Complete v2.3.0 roadmap - S3 storage, dashboard enhancements, documentation #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
Copilot wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

feat: Complete v2.3.0 roadmap - S3 storage, dashboard enhancements, documentation #2

Copilot wants to merge 5 commits into from

Conversation

Copy link

Copilot AI commented Jan 23, 2026 •
edited
Loading

Implements v2.3.0 roadmap: completes incomplete S3 storage backend, enhances dashboard UX, and restructures documentation with configuration examples.

S3 Storage Backend

Completed the partial S3Storage implementation:

  • File serving: express() generates presigned URLs (1h TTL) via S3 client
  • Missing file detection: getMissingFiles() paginates bucket listings to identify sync gaps
  • Cleanup: recycleFiles() batch-deletes (1000 objects/request) files not in manifest
  • Async handling: All boto3 calls wrapped in asyncio.to_thread() to prevent event loop blocking

Factory integration in core/storages/__init__.py supports configuration:

storages:
  - type: s3
    endpoint: https://s3.amazonaws.com
    access_key_id: <key>
    secret_access_key: <secret>
    bucket: <name>
    signature_version: s3v4  # optional
    addressing_style: auto   # optional: auto/path/virtual

Compatible with AWS S3, MinIO, Alibaba Cloud OSS, and S3-compatible services.

Dashboard Improvements

  • Refresh button with loading state and animation
  • 10s polling interval (previously 100s) with automatic refresh on window focus
  • Error handling UI for failed API requests
  • Fixed GitHub repository URL in navigation
  • Comprehensive README documenting tech stack and development setup

Documentation Restructuring

Added to both Chinese and English READMEs:

  • Table of contents with anchor links
  • Core features overview (multi-storage, monitoring, async architecture, i18n)
  • Quick start guide covering pip and Docker deployment
  • Configuration reference with examples for local, AList, and S3 storage backends

Example S3 configuration includes endpoint patterns for major providers (AWS, MinIO, Alibaba).

Original prompt

This section details on the original issue you should resolve

<issue_title>feat: v2.3.0 Roadmap</issue_title>
<issue_description># v2.3.0 Roadmap

新增功能

  • 支持 S3 储存
  • 面板微调
  • 文档重构</issue_description>

Comments on the Issue (you are @copilot in this section)

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@SilianZ
Complete S3 storage implementation with express and recycleFiles methods
5fbdaaa 
Co-authored-by: SilianZ <113701655+SilianZ@users.noreply.github.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 23, 2026
View reviewed changes
core/storages/s3.py
except ClientError as e:
if e.response["Error"]["Code"] == "404":
return web.HTTPNotFound()
logger.terror("storage.error.s3.express", hash=hash, e=e)

Check warning

Code scanning / CodeQL

Information exposure through an exception Medium

Stack trace information
Loading
flows to this location and may be exposed to an external user.

Copilot Autofix

AI 2 days ago

In general, to fix information exposure via exceptions you should log detailed exception data on the server side and return only generic, non-sensitive error messages to clients. Avoid including str(e), stack traces, or other internal state directly in responses.

For this concrete case in core/storages/s3.py, the best fix is to replace web.HTTPError(text=str(e)) with generic error responses. We can keep the existing logging calls (logger.terror(...)) so that developers still have full diagnostic information, but change what is returned to the client:

  • For ClientError where the code is not "404", we can respond with a 502 or 500-level error with a generic message such as "Failed to access storage backend" or a localized equivalent.
  • For the generic Exception handler, respond with a generic 500 error (e.g., web.HTTPInternalServerError) and a minimal message.

We already import locale from core.i18n; following existing patterns in the project, we can use it to obtain a localized generic error message key like "storage.error.s3.express". The only lines that need modification are 145 and 148 (the two return web.HTTPError(text=str(e)) statements). No new imports or helper methods are required.

Suggested changeset 1
core/storages/s3.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/core/storages/s3.py b/core/storages/s3.py
--- a/core/storages/s3.py
+++ b/core/storages/s3.py
@@ -142,10 +142,10 @@
             if e.response["Error"]["Code"] == "404":
                 return web.HTTPNotFound()
             logger.terror("storage.error.s3.express", hash=hash, e=e)
-            return web.HTTPError(text=str(e))
+            return web.HTTPBadGateway(text=locale("storage.error.s3.express"))
         except Exception as e:
             logger.terror("storage.error.s3.express", hash=hash, e=e)
-            return web.HTTPError(text=str(e))
+            return web.HTTPInternalServerError(text=locale("storage.error.s3.express"))
 
     async def recycleFiles(self, files: FileList) -> None:
         """Clean up files in S3 that are not in the valid file list"""
EOF
@@ -142,10 +142,10 @@
if e.response["Error"]["Code"] == "404":
return web.HTTPNotFound()
logger.terror("storage.error.s3.express", hash=hash, e=e)
return web.HTTPError(text=str(e))
return web.HTTPBadGateway(text=locale("storage.error.s3.express"))
except Exception as e:
logger.terror("storage.error.s3.express", hash=hash, e=e)
return web.HTTPError(text=str(e))
return web.HTTPInternalServerError(text=locale("storage.error.s3.express"))

async def recycleFiles(self, files: FileList) -> None:
"""Clean up files in S3 that are not in the valid file list"""
Copilot is powered by AI and may make mistakes. Always verify output.
core/storages/s3.py
logger.terror("storage.error.s3.express", hash=hash, e=e)
return web.HTTPError(text=str(e))
except Exception as e:
logger.terror("storage.error.s3.express", hash=hash, e=e)

Check warning

Code scanning / CodeQL

Information exposure through an exception Medium

Stack trace information
Loading
flows to this location and may be exposed to an external user.

Copilot Autofix

AI 2 days ago

In general, to fix information exposure via exceptions, you should never send raw exception messages or stack traces back to the client. Instead, log detailed errors server-side and return a generic, user-safe error response (optionally with a stable error code that can be correlated in logs).

For this specific code, the best minimal fix is to keep the detailed logging (logger.terror("storage.error.s3.express", ...)) for diagnostics but replace web.HTTPError(text=str(e)) with a response that contains a generic message. We should do this for both the ClientError and generic Exception branches in express while keeping existing behavior (status codes, logging, counters) as intact as possible. Because we must not assume additional project structure, we can directly use a short, non-sensitive message like "Internal server error" in the HTTP error body and leave the status code to web.HTTPError’s default (500). No new imports are needed, since we are just changing the argument passed to web.HTTPError.

Concretely:

  • In core/storages/s3.py, locate the express method.
  • On line 145, change web.HTTPError(text=str(e)) to web.HTTPError(text="Internal server error").
  • On line 148, do the same: replace str(e) with the same generic message.
  • Keep the logger.terror(...) calls unchanged so that developers still see the real exception details in logs.

This preserves existing functionality (S3 presigned-URL creation, counter updates, logging, 404 behavior) while preventing sensitive information from being exposed to clients.

Suggested changeset 1
core/storages/s3.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/core/storages/s3.py b/core/storages/s3.py
--- a/core/storages/s3.py
+++ b/core/storages/s3.py
@@ -142,10 +142,10 @@
             if e.response["Error"]["Code"] == "404":
                 return web.HTTPNotFound()
             logger.terror("storage.error.s3.express", hash=hash, e=e)
-            return web.HTTPError(text=str(e))
+            return web.HTTPError(text="Internal server error")
         except Exception as e:
             logger.terror("storage.error.s3.express", hash=hash, e=e)
-            return web.HTTPError(text=str(e))
+            return web.HTTPError(text="Internal server error")
 
     async def recycleFiles(self, files: FileList) -> None:
         """Clean up files in S3 that are not in the valid file list"""
EOF
@@ -142,10 +142,10 @@
if e.response["Error"]["Code"] == "404":
return web.HTTPNotFound()
logger.terror("storage.error.s3.express", hash=hash, e=e)
return web.HTTPError(text=str(e))
return web.HTTPError(text="Internal server error")
except Exception as e:
logger.terror("storage.error.s3.express", hash=hash, e=e)
return web.HTTPError(text=str(e))
return web.HTTPError(text="Internal server error")

async def recycleFiles(self, files: FileList) -> None:
"""Clean up files in S3 that are not in the valid file list"""
Copilot is powered by AI and may make mistakes. Always verify output.
Copilot AI and others added 3 commits January 23, 2026 00:39
@SilianZ
Improve dashboard with refresh button, better polling, error handling…
5f0afb5 
… and updated README

Co-authored-by: SilianZ <113701655+SilianZ@users.noreply.github.com>
@SilianZ
Restructure and enhance documentation with detailed setup and configu…
032a52a 
…ration guides

Co-authored-by: SilianZ <113701655+SilianZ@users.noreply.github.com>
@SilianZ
Fix async/await issues in S3 storage and adjust dashboard polling int…
3ae64b4 
…erval

Co-authored-by: SilianZ <113701655+SilianZ@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for S3 storage in version 2.3.0 feat: Complete v2.3.0 roadmap - S3 storage, dashboard enhancements, documentation Jan 23, 2026
Copilot AI requested a review from SilianZ January 23, 2026 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SilianZ SilianZ Awaiting requested review from SilianZ

Assignees

@SilianZ SilianZ

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feat: v2.3.0 Roadmap

2 participants

@SilianZ