feat: enforce conditional-write preconditions on PutObject (#120) - #122
Merged
Conversation
Add `if-match`/`if-none-match` to the PutObject signed-forward whitelist so the backend evaluates the precondition and returns 412 Precondition Failed on a stale/wrong ETag instead of silently clobbering. These are standard HTTP headers (not `x-amz-*`), so S3 applies them on a host-only presigned URL exactly like the GET/HEAD read paths already do. Native Zarr/Icechunk writers use conditional writes as the compare-and-swap that protects refs from concurrent commits. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Claude finished @alukach's task in 2m 1s —— View job ✅ No blocking issues — safe to merge. Reviewed the diff (
No correctness, security, or best-practice issues found. Nothing to flag under Simplify (ponytail) either — the diff is a minimal, well-scoped whitelist extension with no new abstractions or dependencies. 💰 Estimated review cost: $0.84 · 2m01s · 25 turns |
|
📖 Docs preview deployed to https://multistore-docs-pr-122.development-seed.workers.dev
|
|
🚀 Latest commit deployed to https://multistore-proxy-pr-122.development-seed.workers.dev
|
The initial fix only forwarded `if-match`/`if-none-match` on the presigned PutObject path. But AWS SDKs and the CLI send PutObject bodies as `aws-chunked` streaming uploads, which take the header-signed `build_streaming_forward` path instead — so an SDK-based Icechunk writer's precondition was still silently dropped, exactly the reported case. Add the two preconditions to `AWS_CHUNKED_FORWARD_HEADERS`. Unlike the presigned path these are signed (`sign_s3_request` signs the whole header map), so S3 evaluates them and returns 412 on a mismatch. They don't apply to `UploadPart`, but clients never send them there, so sharing the list is a no-op for parts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drive the full `handle_request` pipeline against a backend that emulates S3's compare-and-swap, asserting the client sees 412 on a failed precondition and 200 otherwise — on both the presigned and the aws-chunked streaming path. Regression guard for both forward whitelists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Exercise If-Match / If-None-Match on PutObject end-to-end through boto3 → proxy → MinIO. boto3 streams PutObject bodies as aws-chunked, so these hit the header-signed streaming path (the one SDKs use) against a real backend that enforces the precondition and returns 412 — the compare-and-swap Icechunk relies on. Covers the failing precondition (412, object unchanged), the create-if-absent case, and the successful If-Match swap. Verified end-to-end locally (scripts/integration-test.sh -k ConditionalWrites): 4 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Third and final write path: a large object written via multipart is materialized by CompleteMultipartUpload, whose forward whitelist (`execute_multipart`) dropped `if-match`/`if-none-match` — so a multipart write silently ignored the precondition, the same failure mode as the PutObject bug. Add both to the whitelist; the raw multipart path signs every header (`sign_s3_request`), so they are enforced (S3 and MinIO return 412 on a mismatch). They are a no-op on Create/UploadPart/Abort (clients only send them on completion). Tests: extend the mock CasBackend to enforce the precondition on `send_raw` and add CompleteMultipartUpload cases (Rust integration + regression guard), plus boto3 → proxy → MinIO end-to-end multipart cases. Verified end-to-end (scripts/integration-test.sh -k ConditionalWrites): 6 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two review follow-ups on the multipart conditional-write tests: - Hygiene: a failed CompleteMultipartUpload leaves the upload open, so repeated CI runs accumulated stray incomplete uploads. `_complete_multipart` now aborts the upload before propagating the error. - Lock in the "clients never send If-Match on UploadPart" assumption that makes sharing AWS_CHUNKED_FORWARD_HEADERS with the part path safe: botocore has no IfMatch member on UploadPart, but a hand-injected signed precondition must still ride through the proxy's sign-and-forward without breaking the upload (the backend ignores it). New test injects one and asserts the multipart upload completes intact. Verified end-to-end (scripts/integration-test.sh -k ConditionalWrites): 7 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #120.
What I'm changing
Native Zarr/Icechunk writers use conditional writes (
If-Match/If-None-Match) as the compare-and-swap that protects refs from concurrent commits. Today a write with a deliberately wrong ETag succeeds where S3 returns 412 Precondition Failed — the precondition is silently dropped and concurrent commits can clobber each other.The precondition was dropped on all three write paths that produce an object, each with its own forward whitelist:
PutObject, presigned path (build_forward) — plain bodies.if-match/if-none-matchwere absent. These are standard HTTP headers (notx-amz-*), so S3 honors them on a host-only presigned URL unsigned — the same way the GET/HEAD read paths already do.PutObject, aws-chunked streaming path (build_streaming_forward) — what AWS SDKs and the CLI send by default (STREAMING-UNSIGNED-PAYLOAD-TRAILER). This path re-signs the seed. The precondition was missing fromAWS_CHUNKED_FORWARD_HEADERS, so an SDK-based writer'sIf-Matchwas still silently dropped — the exact reported case. Fixing only the presigned path would have missed the most common real-world client.CompleteMultipartUpload(execute_multipart) — the request that materializes a multipart object. Its whitelist forwarded only entity + checksum headers, so a large multipart write silently ignored the precondition too (flagged in review). This raw path signs every header, so the fix is on the safe signed path.The
x-amz-copy-source-if-*copy preconditions remain out of scope — server-side copy is still rejected (#119), so there is nothing to hang them on yet.How I did it
crates/core/src/proxy.rs—"if-match"/"if-none-match"to thePutObjectsigned-forward set, mirroring GET/HEAD.AWS_CHUNKED_FORWARD_HEADERS(sign_s3_requestsigns the whole map, so S3 enforces them).execute_multipartwhitelist. No-op on Create/UploadPart/Abort (clients only send them on completion).Forward/raw response unchanged.crates/core/tests/conditional_writes.rs— Rust integration tests throughProxyGateway::handle_requestagainst a mock backend emulating S3's compare-and-swap on bothforward(presigned + streaming PutObject) andsend_raw(multipart completion). Regression guard for all three whitelists (each verified failing when its fix is reverted).tests/integration/test_integration.py(TestConditionalWrites) — real end-to-end tests: boto3 → proxy → MinIO. Covers PutObject (failing precondition/412 + object unchanged, create-if-absent, successfulIf-Matchswap) andCompleteMultipartUpload(412 onIf-None-Match: *and wrongIf-Match).crates/core/src/proxy.rs(unit test) —put_forward_preserves_conditional_headersasserts the presignedForwardcarries both headers.docs/reference/operations.md— documented the honored preconditions and 412 compare-and-swap semantics across all three write paths.Test plan
cargo test -p multistore— 128 lib + 10 Rust integration tests passcargo fmtcargo clippy --fix— cleancargo checkcargo check -p multistore-cf-workers --target wasm32-unknown-unknownscripts/integration-test.sh -k ConditionalWrites— 6 passed end-to-end (boto3 → wrangler dev → MinIO); confirms the proxy signs the precondition and MinIO enforces it (412PreconditionFailed) on both PutObject and CompleteMultipartUpload🤖 Generated with Claude Code