Skip to content

fix(module-registry): harden OCI publication and caching - #1130

Merged
Brad-Edwards merged 10 commits into
devfrom
GOV-913-oci-module-registry-reliability-pr
Aug 13, 2026
Merged

fix(module-registry): harden OCI publication and caching#1130
Brad-Edwards merged 10 commits into
devfrom
GOV-913-oci-module-registry-reliability-pr

Conversation

@doublewhy

@doublewhy doublewhy commented Aug 12, 2026

Copy link
Copy Markdown

Plain-language summary

  • Context: OpenRAE can package and consume reusable SDL modules as OCI artifacts.
  • Problem: Publication, extraction, and cache updates could become nondeterministic, stale, partial, or inconsistent under failures and concurrency.
  • Fix: Make bundles deterministic, bound and verify archive/cache contents, snapshot verified source bytes, and switch layouts atomically between immutable versions.

Issue mapping

Summary

  • normalize bundle paths and tar/gzip metadata so identical inputs produce identical OCI artifact bytes;
  • canonicalize the publishing root, require complete signing configuration, apply configured network limits consistently, and normalize malformed registry metadata to stable public errors;
  • bound the entire decoded archive before tar parsing and require the standard-library safe extraction filter;
  • derive an exact cache-tree inventory from the verified bundle, revalidate every cache hit, and compose from immutable in-memory source snapshots;
  • publish caches and OCI layouts through atomic pointer updates to immutable, repairable, bounded version stores;
  • centralize optional platform open flags without changing their values, keeping structural typing exclusions compatible with the forthcoming changed-code gate;
  • document the operator-visible behavior and reconstruct GOV-913 traceability for only this module-registry slice.

Reference-backend image-policy changes are intentionally excluded, so this PR does not close #1096.

Compatibility

  • No SDL field, schema, portable contract, OCI media type, hosted service, or experiment surface is added.
  • The successful CLI JSON schema is unchanged; layout_dir now identifies the selected complete immutable OCI layout.
  • OCI extraction on Python 3.11 requires 3.11.4 or newer because earlier patch releases lack the required safe extraction filter and fail closed.

Verification

Exact rebase base: 1a38538c26abbbca0adf6ec4c56d004cc9ce743a (dev). Candidate head: 54a311f1fc6344f5d8c21edc5330ed181450113f.

  • Exact-head focused registry/source-snapshot tests: 232 passed, 5 deselected.
  • Exact-head OCI integration tests: 5 passed, 232 deselected.
  • Exact-head combined focused + OCI integration coverage run: 237 passed.
  • Exact-head changed-code audit using the read-only checker from test(coverage): require 100% coverage for changed Python code #1139 commit ac98e2f0: changed executable lines 1,305/1,305, changed branch exits 376/376, with zero unmapped lines, forbidden pragmas, or failures. The checker is external because test(coverage): require 100% coverage for changed Python code #1139 is not yet in this PR's base.
  • Exact-head Ruff 0.15.9: formatting and lint pass for all changed Python files.
  • Exact-head repository and local policy suite: pass. Requirement governance was attempted, but the configured Ground Control endpoint was unavailable locally and reported skipped-unavailable.
  • Exact-head focused matrix on CPython 3.11.15, 3.12.13, 3.13.5, and 3.14.4 free-threaded: 232 passed, 5 deselected on each interpreter.
  • Native Windows Server 2022 portability was validated on the pre-rebase equivalent head (226 passed, 6 skipped, 5 deselected). The rebase preserves those OCI semantics and centralizes the same optional open-flag values; exact-head canonical hosted verification and the SonarCloud quality gate passed, with all required checks green.

Rebase notes

@doublewhy

Copy link
Copy Markdown
Author

CI follow-up (98a7ed7): the combined Sonar refactors preserved behavior but made _cache.py 626 lines, above the repository’s 500-line limit. I split cache-tree integrity into _cache_integrity.py; the resulting modules are 324 and 328 lines.

Verification after the split:

  • 217 focused unit tests passed; 5 OCI integration tests passed.
  • 1,235/1,235 changed executable lines and 366/366 changed branch exits were covered.
  • Ruff, diff hygiene, and the GOV-913 repository policy gate passed.

The PR remains draft while the fresh GitHub CI/Sonar run completes.

@doublewhy

Copy link
Copy Markdown
Author

Sonar follow-up (3394773): the analyzer’s stdlib model did not include Python 3.11.4’s backported TarFile.extractall(filter=...) contract. A narrow _DataFilterTarFile protocol now supplies that type information while the runtime keeps the mandatory filter="data" call and the existing fail-closed pre-3.11.4 behavior. No suppression or security-filter removal was used.

Post-fix verification: 217 focused unit tests + 5 OCI integration tests passed; Ruff and the GOV-913 policy gate passed; the changed-code coverage checker reports no uncovered changed line or branch.

@doublewhy
doublewhy marked this pull request as ready for review August 12, 2026 22:44
@doublewhy
doublewhy force-pushed the GOV-913-oci-module-registry-reliability-pr branch from ec10870 to 54a311f Compare August 13, 2026 03:08
@doublewhy
doublewhy marked this pull request as draft August 13, 2026 03:09
@Brad-Edwards
Brad-Edwards marked this pull request as ready for review August 13, 2026 03:09
@doublewhy
doublewhy marked this pull request as draft August 13, 2026 03:10
@Brad-Edwards
Brad-Edwards marked this pull request as ready for review August 13, 2026 03:20
@Brad-Edwards
Brad-Edwards merged commit 15e6118 into dev Aug 13, 2026
14 checks passed
@Brad-Edwards
Brad-Edwards deleted the GOV-913-oci-module-registry-reliability-pr branch August 13, 2026 03:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants