Skip to content

Add the S3 production cache profile and GA contract - #107

Merged
forhappy merged 1 commit into
mainfrom
codex/s3-production-readiness
Aug 17, 2026
Merged

Add the S3 production cache profile and GA contract#107
forhappy merged 1 commit into
mainfrom
codex/s3-production-readiness

Conversation

@forhappy

Copy link
Copy Markdown
Contributor

Summary

  • add a cardinality-sized production cache profile backed by persistent Foyer storage
  • prewarm and byte-bound root/upper-node pinning, with graceful cache shutdown
  • expose cache admission, cold-start, provider-byte, prefetch, and amplification metrics
  • add optional OpenTelemetry instruments with bounded dimensions
  • create smaller metadata-only tree geometry for new production-profile repositories
  • resolve snapshot roots and deltas directly from journal-derived commit indexes for listing, diff, and merge planning
  • enable bounded predictive metadata prefetch
  • define the proposed GA format, upgrade/downgrade, recovery, alerting, and provider/cardinality support contract

Why

The existing client remained correct but could load substantially more commit and metadata-pack data than sparse reads, listings, diffs, and merges required. Its in-memory cache was not a complete production cold-start strategy, and the project did not have an explicit compatibility or supported-scale contract.

This change makes the production path explicit and measurable without overstating current qualification.

Impact

Applications can opt into ProductionCacheProfile to get persistent hybrid caching, controlled prewarming, accurate pinned-tier accounting, metadata geometry appropriate for range reads, and cardinality-derived bounds. Existing repositories preserve their create-once tree format.

Logical payloads remain complete immutable provider objects. This PR does not introduce payload packing, payload chunking, multipart ownership, or payload extents.

The 0.1.x crates remain preview. AWS and one-million-object production support still require the provider/cardinality release gates documented in GA-CONTRACT.md.

Validation

  • cargo fmt --all -- --check
  • strict Clippy for workspace/all targets/all features
  • strict Clippy for the client with no default features
  • cargo test --workspace --all-features
  • live pinned-RustFS required suite: 13/13 non-scale tests
  • live production-cache persist/reopen/prewarm/shutdown integration test
  • Rust 1.89 core MSRV check
  • locked clean-downstream checks for minimal and Foyer clients
  • dependency advisory/TLS policy check
  • independent protocol conformance verifier

The expensive AWS, 10K/20K performance, and million-object release gates remain explicit qualification work rather than being treated as unit-test evidence.

@forhappy
forhappy marked this pull request as ready for review August 17, 2026 04:09
@forhappy
forhappy merged commit 60bf572 into main Aug 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant