From 326e76e2ddd2cee63c1ca142d57c224ba44ac6f6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 05:38:10 +0000 Subject: [PATCH] S3 gateway: enable auth-mode compatibility testing and recalibrate to 192/838 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Make the ceph/s3-tests compatibility suite run cleanly against the gateway with SigV4 auth + S3 ACLs enabled (the basis the CI compose uses), fix the gateway bug that previously made that impossible, recalibrate the allowlist, and add a self-updating compatibility badge. Gateway fix — ListBuckets is ownership-scoped: ListAllMyBuckets filtered visible buckets by READ permission, so a bucket made public-read by one account leaked into every other account's listing. Under the suite (auth on) this poisoned cleanup: the per-test setup() nukes all prefixed buckets via each account's client, and a foreign public-read bucket in account B's listing made B's DeleteObjects on it return 403, which the suite re-raises, cascading the whole run into setup/teardown errors. ListBuckets now filters by ownership (BoxAcl.owner == principal), matching AWS/RGW. Anonymous mode is unchanged (owns() short-circuits when auth is disabled). Recalibration (ceph/s3-tests master @ 5522d1c, auth + ACLs enabled): 192 passed / 552 failed / 94 skipped, 0 errors -- up from the 164 pre-auth baseline. The net +28 (0 regressions) is exactly what real auth unlocks: bucket/object ACLs, cross-account access control, auth negatives, and anonymous-access checks. Refreshed allowlist.txt and rewrote compat/s3-tests/README.md for the auth basis (supported-surface table, the ListBuckets ownership note, family-by-family failure breakdown summing to 552). Self-updating badge: run.sh --calibrate now also emits badge.json (a Shields endpoint payload) with the passed/collected percentage, surfaced as the README "S3 compatibility" badge (22%, 192/838) so it tracks future calibrations automatically. Tooling: Added S3CompatGatewayLauncher, a Docker-less in-JVM stack (embedded BookKeeper + ZooKeeper + node + gateway) so the suite can be calibrated without the rate-limited Docker Hub images, and bounded EmbeddedBookKeeper's open-file cache so a single bookie survives the suite's heavy ledger churn under a low RLIMIT_NOFILE. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01MejdBZYEDJWFU6s88tZH6t --- README.md | 17 ++- .../candybox/it/EmbeddedBookKeeper.java | 14 ++ .../candybox/it/S3CompatGatewayLauncher.java | 130 ++++++++++++++++++ .../candybox/s3/S3AccessControl.java | 11 +- .../me/predatorray/candybox/s3/S3Handler.java | 5 +- compat/s3-tests/README.md | 108 ++++++++------- compat/s3-tests/allowlist.txt | 34 ++++- compat/s3-tests/badge.json | 6 + compat/s3-tests/run.sh | 27 ++++ 9 files changed, 291 insertions(+), 61 deletions(-) create mode 100644 candybox-integration-tests/src/test/java/me/predatorray/candybox/it/S3CompatGatewayLauncher.java create mode 100644 compat/s3-tests/badge.json diff --git a/README.md b/README.md index 2035dc3..e3ac551 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,7 @@ Build Docker + S3 compatibility

--- @@ -77,12 +78,14 @@ operators at the right command. The gateway's S3 compatibility is verified against the industry-standard [`ceph/s3-tests`](https://github.com/ceph/s3-tests) suite — see [`compat/s3-tests/`](compat/s3-tests/) (`compat/s3-tests/run.sh --calibrate` against the running -gateway). The latest calibration was taken after the Phase 5 additions (multipart upload, Range -GET) landed: **164 / 838 boto3 functional tests pass** (up from 149 pre-Phase-5). The remaining -gaps the v1 gateway does not yet implement are versioning, ACLs, SSE, POST object, lifecycle, -CORS — see -[`compat/s3-tests/README.md`](compat/s3-tests/README.md#latest-calibration) for the family-by-family -breakdown. +gateway). The latest calibration (the **S3 compatibility** badge above tracks it automatically) runs +the gateway with **SigV4 auth + S3 ACLs enabled**: **192 / 838 boto3 functional tests pass** (up from +164 pre-auth, and 149 pre-Phase-5), zero suite errors. The extra passes are the multi-user / ACL / +cross-account-access tests that real authentication unlocks (`bucket_acl_*`, `object_acl_*`, +`access_bucket_*`, anonymous-access and bad-auth checks). The remaining gaps the v1 gateway does not +yet implement are versioning, SSE, POST object, lifecycle, bucket policy, CORS, and conditional GET — +see [`compat/s3-tests/README.md`](compat/s3-tests/README.md#latest-calibration) for the +family-by-family breakdown. ## Storing and retrieving objects @@ -289,7 +292,7 @@ hard fencing/handover scenarios. No mocking frameworks are used anywhere. | `candybox-protocol` | The framed TCP wire protocol and transport. | | `candybox-server` | The storage node: wires the engine behind the protocol, plus the runnable entrypoint, health/metrics, and ownership. | | `candybox-client` | The thin client library and the `candybox` command-line tool. | -| `candybox-s3-gateway` | An anonymous, path-style, S3-compatible HTTP gateway (Netty) that translates the S3 REST/XML API onto the client. Stateless; runs behind an HTTP(S) load balancer. See `S3_GATEWAY_PLAN.md`. | +| `candybox-s3-gateway` | A path-style, S3-compatible HTTP gateway (Netty) with optional SigV4 auth + S3 ACL enforcement that translates the S3 REST/XML API onto the client. Stateless; runs behind an HTTP(S) load balancer. See `S3_GATEWAY_PLAN.md`. | | `candybox-admin-api` | A stateless HTTP service exposing cluster / boxes / LSM / metrics as JSON, plus the static SPA mount. See `WEB_DASHBOARD_PLAN.md`. | | `candybox-web` | React + TypeScript + MUI dashboard, built by `frontend-maven-plugin` under `-Pfrontend` and packaged into a jar so the admin API serves it from the classpath. | | `candybox-dist` | Packages the runnable distribution (`bin/ lib/ conf/`) and the Docker/Kubernetes assets. | diff --git a/candybox-integration-tests/src/test/java/me/predatorray/candybox/it/EmbeddedBookKeeper.java b/candybox-integration-tests/src/test/java/me/predatorray/candybox/it/EmbeddedBookKeeper.java index 4d22e45..16c6519 100644 --- a/candybox-integration-tests/src/test/java/me/predatorray/candybox/it/EmbeddedBookKeeper.java +++ b/candybox-integration-tests/src/test/java/me/predatorray/candybox/it/EmbeddedBookKeeper.java @@ -33,11 +33,25 @@ public final class EmbeddedBookKeeper implements AutoCloseable { private final LocalBookKeeper localBookKeeper; public EmbeddedBookKeeper(int numBookies) { + this(numBookies, 0); + } + + /** + * @param numBookies bookies to start in-JVM + * @param openFileLimit max open ledger-index files per bookie ({@code <= 0} keeps the BookKeeper + * default). Bounding this keeps the bookie's file-descriptor footprint flat + * when a workload churns through many ledgers, which matters in sandboxes + * with a low {@code RLIMIT_NOFILE}. + */ + public EmbeddedBookKeeper(int numBookies, int openFileLimit) { this.zkPort = freePort(); try { ServerConfiguration serverConf = new ServerConfiguration(); serverConf.setMetadataServiceUri(metadataServiceUri()); serverConf.setAllowLoopback(true); + if (openFileLimit > 0) { + serverConf.setOpenFileLimit(openFileLimit); + } // getLocalBookies(zkHost, zkPort, numBookies, shouldStartZK, conf) stands up the in-JVM // ZooKeeper and the bookies; start() launches them. this.localBookKeeper = diff --git a/candybox-integration-tests/src/test/java/me/predatorray/candybox/it/S3CompatGatewayLauncher.java b/candybox-integration-tests/src/test/java/me/predatorray/candybox/it/S3CompatGatewayLauncher.java new file mode 100644 index 0000000..387b019 --- /dev/null +++ b/candybox-integration-tests/src/test/java/me/predatorray/candybox/it/S3CompatGatewayLauncher.java @@ -0,0 +1,130 @@ +/* + * Copyright (c) 2026 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package me.predatorray.candybox.it; + +import java.nio.charset.StandardCharsets; +import java.util.Map; +import java.util.Properties; +import me.predatorray.candybox.bookkeeper.bk.BookKeeperLedgerStore; +import me.predatorray.candybox.client.CandyboxClient; +import me.predatorray.candybox.common.SystemClock; +import me.predatorray.candybox.common.config.CandyboxConfig; +import me.predatorray.candybox.common.config.LedgerRole; +import me.predatorray.candybox.common.config.QuorumConfig; +import me.predatorray.candybox.coordination.zk.ZooKeeperCoordinationService; +import me.predatorray.candybox.protocol.FrameCodec; +import me.predatorray.candybox.protocol.transport.TcpTransport; +import me.predatorray.candybox.protocol.transport.TcpTransportServer; +import me.predatorray.candybox.s3.S3Gateway; +import me.predatorray.candybox.s3.S3GatewayConfig; +import me.predatorray.candybox.server.CandyboxNode; +import org.apache.curator.test.TestingServer; + +/** + * A throwaway, long-lived launcher that stands up the full Candybox stack in one JVM — embedded + * BookKeeper, in-process ZooKeeper, a real node, and the Netty S3 gateway — bound to a fixed port so + * the {@code ceph/s3-tests} compatibility suite ({@code compat/s3-tests/run.sh}) can be pointed at it + * without Docker. This mirrors what {@code docker-compose.ci.yml} brings up (SigV4 auth + ACL + * enforcement against the s3tests credentials) and exists only to run a calibration when the Docker + * Hub image path is unavailable. It is not a unit/integration test; it blocks until killed. + * + * 
+ *   mvn -q -pl candybox-integration-tests -Dexec.classpathScope=test \
+ *       -Dexec.mainClass=me.predatorray.candybox.it.S3CompatGatewayLauncher \
+ *       -Dexec.args="9711 /abs/path/to/gateway-credentials.properties" \
+ *       org.codehaus.mojo:exec-maven-plugin:3.5.0:java
+ *
+ */ +public final class S3CompatGatewayLauncher { + + private S3CompatGatewayLauncher() { + } + + public static void main(String[] args) throws Exception { + int port = args.length > 0 ? Integer.parseInt(args[0]) : 9711; + String credentialsFile = args.length > 1 ? args[1] : null; + + // One bookie with 1/1/1 quorum and a bounded index-file cache: replication factor is + // irrelevant to S3-API compatibility, and a single bookie keeps the file-descriptor + // footprint flat as the suite churns through thousands of ledgers (sandbox RLIMIT_NOFILE + // is only 4096; the default 3 bookies × one .idx-per-ledger exhausts it mid-suite). + EmbeddedBookKeeper bookKeeper = new EmbeddedBookKeeper(1, 1024); + TestingServer zookeeper = new TestingServer(true); + + QuorumConfig single = new QuorumConfig(1, 1, 1); + CandyboxConfig.Builder configBuilder = CandyboxConfig.builder(); + for (LedgerRole role : LedgerRole.values()) { + configBuilder.quorum(role, single); + } + CandyboxConfig config = configBuilder.build(); + BookKeeperLedgerStore store = new BookKeeperLedgerStore( + bookKeeper.clientConfiguration(), "candybox".getBytes(StandardCharsets.UTF_8)); + ZooKeeperCoordinationService nodeCoord = + new ZooKeeperCoordinationService(zookeeper.getConnectString(), SystemClock.INSTANCE); + int nodePort = 9709; + CandyboxNode node = new CandyboxNode( + 1, config, store, nodeCoord, SystemClock.INSTANCE, "127.0.0.1:" + nodePort); + TcpTransportServer transportServer = + new TcpTransportServer(nodePort, node.requestHandler(), new FrameCodec()); + + TcpTransport transport = new TcpTransport(); + ZooKeeperCoordinationService clientCoord = + new ZooKeeperCoordinationService(zookeeper.getConnectString(), SystemClock.INSTANCE); + CandyboxClient client = new CandyboxClient(transport, clientCoord, config); + + Properties props = new Properties(); + props.setProperty("zookeeper.connect", zookeeper.getConnectString()); + props.setProperty("s3.bind", "127.0.0.1:" + port); + // Mirror docker-compose.ci.yml: real SigV4 + ACL enforcement. allow-anonymous keeps its + // default (true) there, so unsigned requests fall through to the anonymous principal. + if (credentialsFile != null) { + props.setProperty("s3.auth.enabled", "true"); + props.setProperty("auth.credentials.file", credentialsFile); + } + S3Gateway gateway = new S3Gateway(S3GatewayConfig.fromProperties(props, Map.of()), client); + gateway.start(); + + System.out.println("[launcher] S3 gateway up on 127.0.0.1:" + gateway.port() + + " (auth=" + (credentialsFile != null) + ")"); + System.out.flush(); + + Runtime.getRuntime().addShutdownHook(new Thread(() -> { + close(gateway); + close(client); + close(transport); + close(transportServer); + close(node); + close(nodeCoord); + close(clientCoord); + close(store); + close(zookeeper); + close(bookKeeper); + })); + + Thread.currentThread().join(); + } + + private static void close(AutoCloseable c) { + if (c == null) { + return; + } + try { + c.close(); + } catch (Exception ignored) { + // best effort + } + } +} diff --git a/candybox-s3-gateway/src/main/java/me/predatorray/candybox/s3/S3AccessControl.java b/candybox-s3-gateway/src/main/java/me/predatorray/candybox/s3/S3AccessControl.java index 6922431..c30efa4 100644 --- a/candybox-s3-gateway/src/main/java/me/predatorray/candybox/s3/S3AccessControl.java +++ b/candybox-s3-gateway/src/main/java/me/predatorray/candybox/s3/S3AccessControl.java @@ -118,9 +118,14 @@ void authorize(S3Action action, S3Handler.PathParts parts, Principal principal) } } - /** True when {@code principal} may READ the Box (for ListBuckets filtering). */ - boolean mayRead(String box, Principal principal) { - return !enabled || authorizer.authorize(principal, Operation.READ, Resource.box(box)); + /** True when {@code principal} owns the Box. {@code ListAllMyBuckets} lists owned buckets only — + * never buckets merely granted READ (e.g. a {@code public-read} bucket owned by someone else), + * matching AWS/RGW, so one account's public bucket can't leak into another account's listing. */ + boolean owns(String box, Principal principal) { + if (!enabled) { + return true; + } + return boxAcl(box).map(acl -> acl.owner().equals(principal)).orElse(false); } private void require(Principal principal, Operation operation, Resource resource, diff --git a/candybox-s3-gateway/src/main/java/me/predatorray/candybox/s3/S3Handler.java b/candybox-s3-gateway/src/main/java/me/predatorray/candybox/s3/S3Handler.java index 61ffa76..ce24f1f 100644 --- a/candybox-s3-gateway/src/main/java/me/predatorray/candybox/s3/S3Handler.java +++ b/candybox-s3-gateway/src/main/java/me/predatorray/candybox/s3/S3Handler.java @@ -123,10 +123,11 @@ private void dispatch(ChannelHandlerContext ctx, FullHttpRequest request, S3Acti Principal principal = auth.principal(); switch (action) { case LIST_BUCKETS -> { - // Anonymous sees an empty list (RGW-compatible); others see their READable Boxes. + // Anonymous sees an empty list (RGW-compatible); others see only the Boxes they own + // (ListAllMyBuckets is ownership-scoped, not READ-scoped). List visible = principal.isAnonymous() && access.enabled() ? List.of() : store.listBoxes().stream() - .filter(box -> access.mayRead(box, principal)).toList(); + .filter(box -> access.owns(box, principal)).toList(); sendXml(ctx, request, HttpResponseStatus.OK, S3Xml.listAllMyBuckets(visible), requestId); } diff --git a/compat/s3-tests/README.md b/compat/s3-tests/README.md index d34cc76..51e577f 100644 --- a/compat/s3-tests/README.md +++ b/compat/s3-tests/README.md @@ -12,22 +12,29 @@ throwaway virtualenv, and points it at a gateway endpoint. Everything it generat ## What can pass -The v1 gateway is **anonymous** (the `Authorization` header is accepted and ignored) and -**path-style only**, and it implements a deliberately small slice of S3: +The suite calibrates the gateway with **SigV4 auth + S3 ACL enforcement enabled** (Phase D/E) — the +`docker-compose.ci.yml` stack signs every request as the `s3tests` accounts (see +`gateway-credentials.properties` / `s3tests.conf.in`), so the multi-user, ACL, and cross-account +access tests are reachable. The gateway is **path-style only** and implements a deliberately small +slice of S3: | Supported (allowlist candidates) | Not supported in v1 (excluded) | |------------------------------------------|---------------------------------------------------------| | Bucket create / head / delete | Versioning | -| ListObjectsV2: prefix, delimiter, | ACLs / bucket policy / ownership | +| ListObjectsV2: prefix, delimiter, | Bucket policy / public-access block | | max-keys, continuation-token, start-after| Conditional GET (If-Match / If-None-Match / If-*-Since) | | Object PUT / GET / HEAD / DELETE | Tagging, lifecycle, CORS, SSE | | Multi-object delete | POST object (browser-style form upload) | | Range GET (Phase 5) | Object Lock / retention / legal hold (3 trivial passes) | | Multipart upload — create / parts / complete / abort / list (Phase 5) | `UploadPartCopy` / multipart-copy | -| `x-amz-meta-*` user metadata, CRC32C ETag| Per-user auth / multi-account isolation | +| `x-amz-meta-*` user metadata, CRC32C ETag| Object attributes / checksums (SHA-256, CRC*) | +| **SigV4 auth + canned/grant ACLs, multi-user isolation, cross-account access** | Lifecycle / inventory / replication | -Because auth is ignored, the suite's `[s3 alt]` / `[s3 tenant]` accounts all collapse onto one -anonymous namespace — multi-user tests can't pass and aren't in the allowlist. +> **ListBuckets is ownership-scoped.** `ListAllMyBuckets` returns only the buckets a principal owns, +> never buckets merely granted READ — without that, a `public-read` bucket from one account leaks +> into every account's listing and the suite's cross-account teardown cascades into errors. Earlier +> auth-enabled calibration attempts hit exactly this; the fix (gateway `S3AccessControl.owns`) is +> what makes a clean 0-error auth-mode run possible. ## Running @@ -93,59 +100,68 @@ suite and pip-install boto3/pytest the first time. ## Latest calibration -Last calibrated against ceph/s3-tests `master` @ `5522d1c` on **2026-06-01**, after the Phase 5 -gateway additions (multipart upload + Range GET) landed (full suite path -`s3tests/functional/test_s3.py`, 10m 25s wall on an Apple-Silicon laptop — slower than the -pre-Phase-5 7m 52s because the new multipart / range tests actually push MBs through instead of -failing fast on the first PUT). The result is reproducible — re-running `--calibrate` against the +Last calibrated against ceph/s3-tests `master` @ `5522d1c` on **2026-06-17**, against the gateway with +**SigV4 auth + S3 ACLs enabled** over a single-node, single-bookie in-JVM stack (full suite path +`s3tests/functional/test_s3.py`, all 838 items, **0 errors** — every test ran to a clean pass/fail +with no fixture/teardown breakage). The result is reproducible: re-running `--calibrate` against the same source produces a byte-identical `allowlist.txt` apart from the timestamp header. -| Outcome | Count | Δ vs pre-Phase-5 | Notes | +| Outcome | Count | Δ vs prev | Notes | |---|---:|---:|---| -| **Passed** (in `allowlist.txt`) | **164** | +15 | The compatibility gate. | -| Failed | 580 | −15 | Features the v1 gateway doesn't yet implement (breakdown below). | +| **Passed** (in `allowlist.txt`) | **192** | +28 | The compatibility gate. | +| Failed | 552 | −28 | Features the v1 gateway doesn't yet implement (breakdown below). | | Skipped | 94 | — | Suite-level `pytest.mark` opt-outs — never executed (bucket logging, cloud-tier lifecycle, restore-status). | | Collected | 838 | — | All of `s3tests/functional/test_s3.py`. | -What Phase 5 added to the allowlist (no previously-passing test regressed): - -- **Range GET (6 new passes):** the `test_ranged_request_*` response-code family, including the - invalid-range path and the empty-object / trailing-bytes / skip-leading-bytes edges. -- **Multipart upload (7 new passes):** `test_abort_multipart_upload`, - `test_atomic_multipart_upload_write`, `test_list_multipart_upload`, - `test_multipart_upload_empty`, `test_multipart_upload_multiple_sizes`, - `test_multipart_upload_overwrite_existing_object`, - `test_upload_part_copy_percent_encoded_key`. -- **SSE-KMS × multipart (2 new passes):** `test_sse_kms_multipart_invalid_chunks_{1,2}`. The - gateway still ignores `x-amz-server-side-encryption-*`, so once multipart works these - SSE-negative-path tests fall through to plain multipart and pass. - -The 580 remaining failures are the **growth surface** — implementing any of these would expand the -allowlist on the next `--calibrate`: +What changed since the previous calibration (164 passes, after Phase 5's multipart + Range GET): +turning **SigV4 auth + S3 ACLs** on (Phase D/E) — plus the `ListBuckets` ownership-scoping fix that +makes a clean auth-mode run possible — brought a **net +28**, **28 newly passing, 0 regressions**. +The new passes are exactly the tests real authentication unlocks: + +- **ACLs (`bucket_acl_*`, `object_acl_*`, `*_canned_*`):** default ACLs, canned-ACL round-trips, + full-control attribute checks, concurrent canned-ACL sets. +- **Cross-account access (`access_bucket_private_*`, `object_copy_not_owned_*`):** a private + bucket/object correctly denies another account; public-read/-write grants correctly allow it. +- **Auth negatives (`list_buckets_bad_auth`, `list_buckets_invalid_auth`):** bad/garbage SigV4 + credentials are rejected with the right error. +- **Anonymous access (`bucket_list*_objects_anonymous*`, `object_anon_put`):** unsigned requests are + allowed or denied per ACL. +- Plus the not-found / error-response-shape edges (`bucket_notexist`, `object_copy_bucket_not_found`, + `object_write_to_nonexist_bucket`, …). + +The 552 remaining failures are the **growth surface** — implementing any of these would expand the +allowlist on the next `--calibrate`. Each failing test is counted once (first matching family): | Failures | Feature family | |---:|---| -| 104 | Server-side encryption — SSE-C / SSE-S3 / SSE-KMS (incl. the 64-test `copy_enc[…]` matrix) | -| 44 | Multipart upload edges still unimplemented — `copy_part` / `multipart_copy`, checksum-on-complete, resume / list-parts pagination | -| 40 | ACLs / ownership controls (`object_acl`, `bucket_acl`, `access_bucket_*`) | -| 38 | Versioning (`versioning_*`, `delete_marker_*`, `*_versioned`) | -| 36 | Object Lock / retention / legal hold (beyond the 3 that already pass) | -| 36 | POST object (browser-style form uploads) | -| 33 | Lifecycle / expiration / non-current-version rules | -| 29 | Bucket policy / block-public-access | -| 29 | Bucket logging (the bits not behind the `pytest.mark` skip) | -| 25 | `ListObjects(V2)` edge cases — encoding-type, exotic keys, sort order | -| 13 | CORS preflight + actual cross-origin | -| 13 | Anonymous raw-HTTP negative tests | -| 11 | Object copy edge (metadata-directive, content-type override, ACL on copy) | -| 9 | Bucket create / naming negative + target-bucket setup | -| 8 | Tagging beyond the basic put/get already in the allowlist | -| 4 | Range GET + `If-Match` / `If-Modified-Since` matrix (conditional-GET headers still TODO) | -| 67 | Other — object attributes (checksum SHA-256, CRC64NVMe), public-access-block, `expected_bucket_owner`, bucket-recreate ACL, account-usage, misc PUT/GET/HEAD/DELETE response-shape edges | +| 133 | Server-side encryption — SSE-C / SSE-S3 / SSE-KMS (incl. the `copy_enc[…]` matrix) | +| 51 | Versioning (`versioning_*`, `delete_marker_*`, `*_versioned`) | +| 43 | ACLs / grants / ownership edges still unimplemented (beyond the basics now passing) | +| 43 | Lifecycle / expiration / non-current-version rules | +| 41 | Bucket policy / block-public-access | +| 36 | Object Lock / retention / legal hold / governance bypass (beyond the few that pass) | +| 31 | POST object (browser-style form uploads) | +| 28 | Bucket logging (the bits not behind the `pytest.mark` skip) | +| 22 | Conditional GET/PUT — `If-Match` / `If-None-Match` / `If-*-Since` | +| 20 | `ListObjects(V2)` edge cases — encoding-type, exotic keys, sort order, markers | +| 18 | Bucket create / naming negative + misc bucket-level | +| 14 | CORS preflight + actual cross-origin | +| 14 | Tagging beyond the basic put/get already in the allowlist | +| 14 | Object attributes / checksums (SHA-256, CRC*) | +| 13 | Object copy edges (metadata-directive, content-type override, ACL on copy) | +| 12 | Multipart upload edges — `copy_part` / `multipart_copy`, checksum-on-complete, resume / list-parts pagination | +| 19 | Other — `expected_bucket_owner`, account-usage, raw-HTTP negatives, misc PUT/GET/HEAD/DELETE response-shape edges | The headers in `allowlist.txt` always record the exact suite SHA + endpoint a result was calibrated against, so the table above can be reproduced verbatim from the checked-in artifact. +### The compatibility badge + +`run.sh --calibrate` also writes `badge.json` (a [Shields endpoint](https://shields.io/badges/endpoint-badge) +payload) next to `allowlist.txt`, so the **S3 compatibility** badge in the top-level README — which +points at `badge.json` on `main` — tracks the calibrated `passed / collected` percentage +automatically. Re-calibrate, commit `badge.json` alongside `allowlist.txt`, and the badge follows. + ## In CI [`.github/workflows/s3-compat.yml`](../../.github/workflows/s3-compat.yml) runs this gate on **every diff --git a/compat/s3-tests/allowlist.txt b/compat/s3-tests/allowlist.txt index f1e4440..11291a0 100644 --- a/compat/s3-tests/allowlist.txt +++ b/compat/s3-tests/allowlist.txt @@ -7,9 +7,16 @@ # # suite ref : master (5522d1c) # endpoint : http://127.0.0.1:9711 -# calibrated: 2026-06-01T03:27:10Z +# calibrated: 2026-06-17T05:30:57Z # +s3tests/functional/test_s3.py::test_100_continue_error_retry s3tests/functional/test_s3.py::test_abort_multipart_upload +s3tests/functional/test_s3.py::test_access_bucket_private_object_private +s3tests/functional/test_s3.py::test_access_bucket_private_object_publicread +s3tests/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite +s3tests/functional/test_s3.py::test_access_bucket_private_objectv2_private +s3tests/functional/test_s3.py::test_access_bucket_private_objectv2_publicread +s3tests/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite s3tests/functional/test_s3.py::test_atomic_conditional_write_1mb s3tests/functional/test_s3.py::test_atomic_dual_write_1mb s3tests/functional/test_s3.py::test_atomic_dual_write_4mb @@ -22,6 +29,9 @@ s3tests/functional/test_s3.py::test_atomic_write_1mb s3tests/functional/test_s3.py::test_atomic_write_4mb s3tests/functional/test_s3.py::test_atomic_write_8mb s3tests/functional/test_s3.py::test_basic_key_count +s3tests/functional/test_s3.py::test_bucket_acl_canned_private_to_private +s3tests/functional/test_s3.py::test_bucket_acl_default +s3tests/functional/test_s3.py::test_bucket_concurrent_set_canned_acl s3tests/functional/test_s3.py::test_bucket_create_naming_bad_ip s3tests/functional/test_s3.py::test_bucket_create_naming_bad_short_one s3tests/functional/test_s3.py::test_bucket_create_naming_bad_short_two @@ -41,6 +51,7 @@ s3tests/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha s3tests/functional/test_s3.py::test_bucket_create_naming_good_starts_digit s3tests/functional/test_s3.py::test_bucket_create_special_key_names s3tests/functional/test_s3.py::test_bucket_delete_nonempty +s3tests/functional/test_s3.py::test_bucket_delete_notexist s3tests/functional/test_s3.py::test_bucket_head s3tests/functional/test_s3.py::test_bucket_head_notexist s3tests/functional/test_s3.py::test_bucket_list_delimiter_alt @@ -61,6 +72,8 @@ s3tests/functional/test_s3.py::test_bucket_list_marker_not_in_list s3tests/functional/test_s3.py::test_bucket_list_marker_unreadable s3tests/functional/test_s3.py::test_bucket_list_maxkeys_none s3tests/functional/test_s3.py::test_bucket_list_maxkeys_one +s3tests/functional/test_s3.py::test_bucket_list_objects_anonymous +s3tests/functional/test_s3.py::test_bucket_list_objects_anonymous_fail s3tests/functional/test_s3.py::test_bucket_list_prefix_alt s3tests/functional/test_s3.py::test_bucket_list_prefix_basic s3tests/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt @@ -88,6 +101,8 @@ s3tests/functional/test_s3.py::test_bucket_listv2_fetchowner_empty s3tests/functional/test_s3.py::test_bucket_listv2_many s3tests/functional/test_s3.py::test_bucket_listv2_maxkeys_none s3tests/functional/test_s3.py::test_bucket_listv2_maxkeys_one +s3tests/functional/test_s3.py::test_bucket_listv2_objects_anonymous +s3tests/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail s3tests/functional/test_s3.py::test_bucket_listv2_prefix_alt s3tests/functional/test_s3.py::test_bucket_listv2_prefix_basic s3tests/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt @@ -102,29 +117,39 @@ s3tests/functional/test_s3.py::test_bucket_listv2_prefix_unreadable s3tests/functional/test_s3.py::test_bucket_listv2_startafter_after_list s3tests/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list s3tests/functional/test_s3.py::test_bucket_listv2_startafter_unreadable +s3tests/functional/test_s3.py::test_bucket_notexist s3tests/functional/test_s3.py::test_buckets_create_then_list +s3tests/functional/test_s3.py::test_bucketv2_notexist s3tests/functional/test_s3.py::test_copy_object_ifmatch_good s3tests/functional/test_s3.py::test_copy_object_ifnonematch_failed s3tests/functional/test_s3.py::test_encrypted_transfer_13b +s3tests/functional/test_s3.py::test_encrypted_transfer_1MB s3tests/functional/test_s3.py::test_encrypted_transfer_1b s3tests/functional/test_s3.py::test_encrypted_transfer_1kb -s3tests/functional/test_s3.py::test_encrypted_transfer_1MB s3tests/functional/test_s3.py::test_get_obj_tagging s3tests/functional/test_s3.py::test_get_object_ifmatch_good s3tests/functional/test_s3.py::test_get_object_ifmodifiedsince_good s3tests/functional/test_s3.py::test_get_object_ifnonematch_failed s3tests/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed s3tests/functional/test_s3.py::test_list_buckets_anonymous +s3tests/functional/test_s3.py::test_list_buckets_bad_auth +s3tests/functional/test_s3.py::test_list_buckets_invalid_auth s3tests/functional/test_s3.py::test_list_multipart_upload s3tests/functional/test_s3.py::test_multi_object_delete s3tests/functional/test_s3.py::test_multi_objectv2_delete s3tests/functional/test_s3.py::test_multipart_upload_empty s3tests/functional/test_s3.py::test_multipart_upload_multiple_sizes s3tests/functional/test_s3.py::test_multipart_upload_overwrite_existing_object +s3tests/functional/test_s3.py::test_object_acl_default +s3tests/functional/test_s3.py::test_object_acl_full_control_verify_attributes s3tests/functional/test_s3.py::test_object_acl_full_control_verify_owner +s3tests/functional/test_s3.py::test_object_anon_put s3tests/functional/test_s3.py::test_object_anon_put_write_access +s3tests/functional/test_s3.py::test_object_copy_bucket_not_found s3tests/functional/test_s3.py::test_object_copy_canned_acl s3tests/functional/test_s3.py::test_object_copy_key_not_found +s3tests/functional/test_s3.py::test_object_copy_not_owned_bucket +s3tests/functional/test_s3.py::test_object_copy_not_owned_object_bucket s3tests/functional/test_s3.py::test_object_copy_retaining_metadata s3tests/functional/test_s3.py::test_object_copy_same_bucket s3tests/functional/test_s3.py::test_object_copy_verify_contenttype @@ -139,10 +164,12 @@ s3tests/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant s3tests/functional/test_s3.py::test_object_put_authenticated s3tests/functional/test_s3.py::test_object_raw_authenticated s3tests/functional/test_s3.py::test_object_raw_authenticated_bucket_acl +s3tests/functional/test_s3.py::test_object_raw_authenticated_bucket_gone s3tests/functional/test_s3.py::test_object_raw_authenticated_object_acl s3tests/functional/test_s3.py::test_object_raw_authenticated_object_gone s3tests/functional/test_s3.py::test_object_raw_get s3tests/functional/test_s3.py::test_object_raw_get_bucket_acl +s3tests/functional/test_s3.py::test_object_raw_get_bucket_gone s3tests/functional/test_s3.py::test_object_raw_get_object_gone s3tests/functional/test_s3.py::test_object_read_not_exist s3tests/functional/test_s3.py::test_object_requestid_matches_header_on_error @@ -151,6 +178,7 @@ s3tests/functional/test_s3.py::test_object_set_get_metadata_none_to_good s3tests/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty s3tests/functional/test_s3.py::test_object_write_file s3tests/functional/test_s3.py::test_object_write_read_update_read_delete +s3tests/functional/test_s3.py::test_object_write_to_nonexist_bucket s3tests/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding s3tests/functional/test_s3.py::test_put_max_kvsize_tags s3tests/functional/test_s3.py::test_put_max_tags @@ -169,7 +197,7 @@ s3tests/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 s3tests/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 s3tests/functional/test_s3.py::test_sse_kms_present s3tests/functional/test_s3.py::test_sse_kms_transfer_13b +s3tests/functional/test_s3.py::test_sse_kms_transfer_1MB s3tests/functional/test_s3.py::test_sse_kms_transfer_1b s3tests/functional/test_s3.py::test_sse_kms_transfer_1kb -s3tests/functional/test_s3.py::test_sse_kms_transfer_1MB s3tests/functional/test_s3.py::test_upload_part_copy_percent_encoded_key diff --git a/compat/s3-tests/badge.json b/compat/s3-tests/badge.json new file mode 100644 index 0000000..018e755 --- /dev/null +++ b/compat/s3-tests/badge.json @@ -0,0 +1,6 @@ +{ + "schemaVersion": 1, + "label": "S3 compatibility", + "message": "22% (192/838)", + "color": "orange" +} diff --git a/compat/s3-tests/run.sh b/compat/s3-tests/run.sh index a270069..b28f953 100755 --- a/compat/s3-tests/run.sh +++ b/compat/s3-tests/run.sh @@ -32,6 +32,7 @@ SUITE="$WORK/s3-tests" VENV="$WORK/venv" CONF="$WORK/s3tests.conf" ALLOWLIST="$HERE/allowlist.txt" +BADGE="$HERE/badge.json" S3_ENDPOINT="${S3_ENDPOINT:-http://127.0.0.1:9711}" S3TESTS_REF="${S3TESTS_REF:-master}" @@ -139,5 +140,31 @@ case "$MODE" in } > "$ALLOWLIST" PASSES="$(grep -vcE '^\s*(#|$)' "$ALLOWLIST" || true)" log "wrote $PASSES passing tests to $ALLOWLIST" + + # --- shields.io endpoint badge ------------------------------------------------------------- + # Emit badge.json (Shields "endpoint" schema) so the README badge tracks the calibration + # automatically: percentage = passed / collected, the same "X / N tests pass" metric the docs + # quote. Regenerated on every --calibrate, committed alongside allowlist.txt. + COLLECTED="$(grep -oE 'collected [0-9]+ item' "$REPORT" | grep -oE '[0-9]+' | head -1)" + COLLECTED="${COLLECTED:-0}" + if [[ "$COLLECTED" -gt 0 ]]; then + PCT=$(( PASSES * 100 / COLLECTED )) + else + PCT=0 + fi + if [[ "$PCT" -ge 75 ]]; then COLOR="brightgreen" + elif [[ "$PCT" -ge 50 ]]; then COLOR="green" + elif [[ "$PCT" -ge 25 ]]; then COLOR="yellow" + else COLOR="orange" + fi + cat > "$BADGE" <