S3 gateway: auth-mode compatibility testing, recalibrate to 192/838, add badge

[predatorray]

What & why

Make the ceph/s3-tests compatibility suite run cleanly against the gateway with SigV4 auth + S3 ACLs enabled (the basis the CI compose uses), fix the gateway bug that previously made auth-mode calibration impossible, recalibrate the allowlist, and add a self-updating compatibility badge.

Previously the auth-enabled suite collapsed into 659 fixture errors — not a real compatibility signal. Root cause below.

Gateway fix — ListBuckets is ownership-scoped

ListAllMyBuckets filtered visible buckets by READ permission, so a public-read bucket owned by one account leaked into every account's listing. Under the suite (auth on) this poisoned cleanup: the per-test setup() nukes all prefixed buckets via each account's client, and a foreign public-read bucket showing up in account B's listing made B's DeleteObjects on it return 403, which the suite re-raises — cascading the whole run into setup/teardown errors.

Fix: ListBuckets now filters by ownership (BoxAcl.owner == principal), matching AWS/RGW where ListAllMyBuckets returns only buckets you own. Anonymous mode is unchanged (owns() short-circuits when auth is disabled).

Recalibration (ceph/s3-tests master @ 5522d1c, auth + ACLs enabled)

Outcome	Count	Δ vs baseline
Passed	192	+28
Failed	552	−28
Skipped	94	—
Errors	0	(was 659)
Collected	838	—

28 newly passing, 0 regressions over the 164 pre-auth baseline — exactly what real auth unlocks: bucket/object ACLs (bucket_acl_*, object_acl_*), cross-account access control (access_bucket_private_*), auth negatives (list_buckets_bad_auth), and anonymous-access checks.

Self-updating badge

run.sh --calibrate now also emits badge.json (a Shields endpoint payload) with the passed / collected percentage, surfaced as the README S3 compatibility badge (22%, 192/838) so it tracks future calibrations automatically.

Tooling

Added S3CompatGatewayLauncher, a Docker-less in-JVM stack (embedded BookKeeper + ZooKeeper + node + gateway) so the suite can be calibrated without the rate-limited Docker Hub images, and bounded EmbeddedBookKeeper's open-file cache so a single bookie survives the suite's heavy ledger churn under a low RLIMIT_NOFILE.

Verification

• Gateway unit tests pass.
• Calibration ran the full 838-test suite end-to-end with 0 errors (the cascade is gone).
• Re-running run.sh in gate mode against the auth gateway reproduces the 192-test allowlist green.

Notes / follow-ups

• The remaining 552 failures are the growth surface — SSE (133), versioning (51), lifecycle (43), bucket policy (41), Object Lock (36), POST object (31), … — see compat/s3-tests/README.md.
• CI (GitHub Actions) builds the image from source, so it is unaffected by the Docker Hub rate-limiting worked around here; it runs the same auth-enabled compose this calibration used.

🤖 Generated with Claude Code

---

Generated by Claude Code

[github-actions]

✅ S3 compatibility gate passed

• mode: gate
• result: 192 passed in 82.02s (0:01:22)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.66%. Comparing base (626e55c) to head (326e76e).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main      #23   +/-   ##
=========================================
  Coverage     83.66%   83.66%           
  Complexity      677      677           
=========================================
  Files           161      161           
  Lines          8874     8876    +2     
  Branches       1330     1330           
=========================================
+ Hits           7424     7426    +2     
  Misses          970      970           
  Partials        480      480           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.