What's needed
Support S3 CopyObject (server-side copy via x-amz-copy-source) so clients can copy objects within the store without downloading and re-uploading.
Motivation
Reported by a Source Coop contributor trying to write an Icechunk Zarr store natively through the proxy. Every Icechunk commit server-side-copies the mutable repo pointer file (to archive the previous ref state) before atomically replacing it. Today the proxy hard-rejects this:
$ aws s3api copy-object --bucket <account> --key <product>/dest \
--copy-source <account>/<product>/src --endpoint-url https://data.source.coop
An error occurred (NotImplemented) when calling the CopyObject operation:
not implemented: server-side copy (x-amz-copy-source) is not supported
Without it, contributors must build the store locally and sync files up, which loses Icechunk's transactional commit path. dynamical.org reportedly uses Icechunk-native writes for their weather datasets on AWS Open Data.
Current behavior
parse_s3_request() rejects any PUT carrying x-amz-copy-source up front:
crates/core/src/api/request.rs:24-28 — returns ProxyError::NotImplemented for both CopyObject and UploadPartCopy (both arrive as PUT + x-amz-copy-source).- Error → S3 code
NotImplemented / HTTP 501: crates/core/src/error.rs:38,104,127.
- Existing test asserting the rejection:
copy_source_put_is_rejected_not_implemented at crates/core/src/api/request.rs:229.
Why it isn't a trivial header pass-through
The normal PUT write path forwards to the backend via a presigned URL and can only carry a whitelisted set of signed headers (crates/core/src/proxy.rs:802-833, whitelist at :818-827). x-amz-copy-source is not in that set, and an unsigned header can't ride a presigned URL — it must be part of the signature. So CopyObject needs the copy-source header included in the presign/signing for the destination, and the copy-source path (<src-account>/<src-product>/<key>) must map through the proxy's path-mapping/auth the same way a normal key does.
Scope / proposed approach
- Add a
CopyObject (and UploadPartCopy) variant to S3Operation (crates/core/src/types.rs:346) with the action()/bucket()/key() plumbing.
- Parse the copy-source header in
build_s3_operation() instead of blanket-rejecting; validate + rewrite the source path through path-mapping/authz (a caller must be allowed to read the source and write the destination).
- Dispatch: presign/sign the destination PUT with
x-amz-copy-source (+ optional x-amz-copy-source-if-match etc.) so the backend performs the copy.
- Update the
request.rs:229 test to assert success (or scoped rejection), and add copy round-trip coverage.
- Update
docs/ for the newly supported operation.
Pairs with conditional-write support (#120) — Icechunk needs both for native transactional commits.
What's needed
Support S3 CopyObject (server-side copy via
x-amz-copy-source) so clients can copy objects within the store without downloading and re-uploading.Motivation
Reported by a Source Coop contributor trying to write an Icechunk Zarr store natively through the proxy. Every Icechunk commit server-side-copies the mutable repo pointer file (to archive the previous ref state) before atomically replacing it. Today the proxy hard-rejects this:
Without it, contributors must build the store locally and sync files up, which loses Icechunk's transactional commit path.
dynamical.orgreportedly uses Icechunk-native writes for their weather datasets on AWS Open Data.Current behavior
parse_s3_request()rejects any PUT carryingx-amz-copy-sourceup front:crates/core/src/api/request.rs:24-28— returnsProxyError::NotImplementedfor both CopyObject and UploadPartCopy (both arrive asPUT+x-amz-copy-source).NotImplemented/ HTTP 501:crates/core/src/error.rs:38,104,127.copy_source_put_is_rejected_not_implementedatcrates/core/src/api/request.rs:229.Why it isn't a trivial header pass-through
The normal PUT write path forwards to the backend via a presigned URL and can only carry a whitelisted set of signed headers (
crates/core/src/proxy.rs:802-833, whitelist at:818-827).x-amz-copy-sourceis not in that set, and an unsigned header can't ride a presigned URL — it must be part of the signature. So CopyObject needs the copy-source header included in the presign/signing for the destination, and the copy-source path (<src-account>/<src-product>/<key>) must map through the proxy's path-mapping/auth the same way a normal key does.Scope / proposed approach
CopyObject(andUploadPartCopy) variant toS3Operation(crates/core/src/types.rs:346) with theaction()/bucket()/key()plumbing.build_s3_operation()instead of blanket-rejecting; validate + rewrite the source path through path-mapping/authz (a caller must be allowed to read the source and write the destination).x-amz-copy-source(+ optionalx-amz-copy-source-if-matchetc.) so the backend performs the copy.request.rs:229test to assert success (or scoped rejection), and add copy round-trip coverage.docs/for the newly supported operation.Pairs with conditional-write support (#120) — Icechunk needs both for native transactional commits.