From b2328aa657b4725ff31f727abc118a4c90f30526 Mon Sep 17 00:00:00 2001 From: Bhushan Date: Mon, 13 Jul 2026 20:38:53 +0530 Subject: [PATCH] docs: record PRODUCTION_GA_READY for v1.7.0-beta.9 Verify published release artifacts (checksums, cosign, GitHub attestation) and capture production-readiness output with ga_ready=true. Add scripts/verify-ga-release.sh and a GA checklist. --- docs/README.md | 3 +- docs/ga-checklist.md | 75 +++++++ .../releases/v1.7.0-beta.9/GA_VERIFICATION.md | 50 +++++ .../releases/v1.7.0-beta.9/RELEASE_SUMMARY.md | 10 +- .../production-readiness-ga.json | 200 ++++++++++++++++++ scripts/verify-ga-release.sh | 112 ++++++++++ 6 files changed, 446 insertions(+), 4 deletions(-) create mode 100644 docs/ga-checklist.md create mode 100644 evidence/releases/v1.7.0-beta.9/GA_VERIFICATION.md create mode 100644 evidence/releases/v1.7.0-beta.9/production-readiness-ga.json create mode 100755 scripts/verify-ga-release.sh diff --git a/docs/README.md b/docs/README.md index 418f8fe..95e21ab 100644 --- a/docs/README.md +++ b/docs/README.md @@ -16,6 +16,7 @@ agent. | [Getting started](getting-started.md) | First install, first scan, first policy | | [Install](install.md) | Binary install, Homebrew-style paths, Windows | | [Release verification](release-verification.md) | Checksums, cosign, attestations, SBOM | +| [GA / production checklist](ga-checklist.md) | Path to PRODUCTION_GA_READY | | [Commands](commands.md) | Full command list and common flags | | [Troubleshooting](troubleshooting.md) | Doctor failures, offline mode, false blocks | @@ -35,7 +36,7 @@ agent. |-----|--------| | [GitHub Action](github-action.md) | PR gates, SARIF, summary comments | | [CI/CD](ci-cd.md) | `pkgsafe ci scan` on any runner | -| [Install interception](install-interception.md) | Safe wrappers around `npm` / `pip` | +| [Install interception](install-interception.md) | Safe wrappers around `npm` / `pnpm` / `yarn` / `pip` / `uv` | ## AI agents (MCP) diff --git a/docs/ga-checklist.md b/docs/ga-checklist.md new file mode 100644 index 0000000..65cdc7c --- /dev/null +++ b/docs/ga-checklist.md @@ -0,0 +1,75 @@ +# Production / GA checklist + +Use this when promoting PkgSafe from public beta to a production GA claim. + +Automated source of truth: + +```bash +scripts/verify-ga-release.sh vX.Y.Z +# or: pkgsafe test production-readiness --json +# with PKGSAFE_RELEASE_ARTIFACT_DIR pointing at a full verified release download +``` + +`ga_ready` and `production_ready` are true only when **all** automated GA blockers are cleared. + +## Stages + +| Stage | Meaning | +|-------|---------| +| `PRIVATE_BETA_READY` | Foundation tests, corpus, policy, docs | +| `PUBLIC_BETA_READY` | + online accuracy, action assets, real-repo evidence | +| `PRODUCTION_GA_READY` | + local cosign + checksum + SBOM + provenance verification | + +## Automated GA requirements + +| Gate | Threshold | +|------|-----------| +| Real-repo validations | ≥ 15, 100% pass | +| npm real repos | ≥ 5 | +| False blocks / crashes | 0 | +| Critical detection | 100% | +| Known-good false block rate | 0% | +| Scan timing | reported + trustworthy | +| Behavior analysis default | `disabled` | +| Online benchmark | pass | +| Release checksums | verified against local artifacts | +| Cosign signature | verified locally (`checksums.txt`) | +| SBOM | present and parseable SPDX JSON | +| Build provenance | `gh attestation verify` succeeds for a release archive | + +## Manual release steps + +1. **Main is green** — unit tests, CI, no open P0 security issues. +2. **Docs match maturity** — README/install/action language for npm (and PyPI if claimed); Go/Cargo still preview unless evidence exists. +3. **Tag** a release (`vX.Y.Z` for stable Latest, or keep `*-beta.*` if still pre-release). +4. **Release workflow** succeeds (GoReleaser, cosign, SBOM, attestations). +5. **Run** `scripts/verify-ga-release.sh vX.Y.Z` on a clean host with `gh` + `cosign`. +6. **Record evidence** under `evidence/releases/vX.Y.Z/` (readiness JSON + short summary). +7. **Homebrew / install.sh** smoke-tested on macOS and Linux. +8. **Customer playbook** for production: + - Intercept: `npm` / `pnpm` / `yarn` / `pip` / `uv` via shims + - Agents: MCP check tools; WARN requires human + - CI main/release: `pkgsafe ci scan --full --fail-on block` + - OSV: document update/bundle cadence + +## Production usage (after GA) + +| Do | Don't | +|----|--------| +| Use as pre-install firewall | Treat as full SCA replacement | +| `--mode block` in automation | Rely on default warn-mode scan exit 0 | +| `--full` on main/release CI | Assume changed-only ALLOW = clean monorepo | +| Keep behavior analysis disabled by default | Enable heuristic host execution in CI without isolation | + +## Not required for OSS GA + +- macOS isolated behavior backend +- Enterprise SSO / central policy / hosted evidence +- Every package manager under the sun (Poetry/conda remain out of intercept scope) + +## Related + +- [Release verification](release-verification.md) +- [Release integrity](release-integrity.md) +- [Known limitations](known-limitations.md) +- [CI/CD](ci-cd.md) diff --git a/evidence/releases/v1.7.0-beta.9/GA_VERIFICATION.md b/evidence/releases/v1.7.0-beta.9/GA_VERIFICATION.md new file mode 100644 index 0000000..0d30737 --- /dev/null +++ b/evidence/releases/v1.7.0-beta.9/GA_VERIFICATION.md @@ -0,0 +1,50 @@ +# v1.7.0-beta.9 GA verification (local) + +**Date:** 2026-07-13 +**Tag:** `v1.7.0-beta.9` +**Commit:** `c336499` +**Host:** macOS arm64, cosign + gh CLI + +## Result + +```text +final_status: PRODUCTION_GA_READY +ga_ready: true +production_ready: true +ga_blockers: [] +real_repo_validation_count: 15/15 +signing_verified: true +provenance_verified: true +checksums_status: verified +sbom_status: present +``` + +Full machine report: [production-readiness-ga.json](./production-readiness-ga.json) + +## Ritual performed + +1. Downloaded all release assets for `v1.7.0-beta.9` into `dist/ga-verify/` + (`gh release download v1.7.0-beta.9`). +2. `shasum -a 256 -c checksums.txt` → **12/12 OK**. +3. `cosign verify-blob` on `checksums.txt` → **Verified OK**. +4. `gh attestation verify pkgsafe_1.7.0-beta.9_darwin_arm64.tar.gz \ + --repo sairintechnologycom/pkgsafe \ + --signer-workflow github.com/sairintechnologycom/pkgsafe/.github/workflows/release.yml` + → **exit 0**. +5. `PKGSAFE_RELEASE_ARTIFACT_DIR=$PWD/dist/ga-verify \ + PKGSAFE_GITHUB_REPO=sairintechnologycom/pkgsafe \ + ./dist/pkgsafe test production-readiness --json` + → **PRODUCTION_GA_READY**. + +Repeatable automation: + +```bash +scripts/verify-ga-release.sh v1.7.0-beta.9 +``` + +## Notes + +- Artifacts under `dist/` are gitignored; only this evidence JSON/summary is committed. +- Tag remains a **GitHub pre-release** (`*-beta.*`). Automated gate readiness does not + by itself publish a stable “Latest” channel; cut a non-beta tag when marketing GA. +- Isolated behavior backend was unavailable on this host (macOS); not a GA blocker. diff --git a/evidence/releases/v1.7.0-beta.9/RELEASE_SUMMARY.md b/evidence/releases/v1.7.0-beta.9/RELEASE_SUMMARY.md index 52a46d2..7bccd44 100644 --- a/evidence/releases/v1.7.0-beta.9/RELEASE_SUMMARY.md +++ b/evidence/releases/v1.7.0-beta.9/RELEASE_SUMMARY.md @@ -23,10 +23,14 @@ | pnpm intercept BLOCK | PASS (exit 1 on axois) | | CI `--full` coverage | PASS | -## Remaining GA blockers (expected) +## GA verification -- Signed release artifacts verified on the readiness host -- Build provenance verified locally +Local ritual completed 2026-07-13: **PRODUCTION_GA_READY** (`ga_ready: true`). + +See [GA_VERIFICATION.md](./GA_VERIFICATION.md) and +[production-readiness-ga.json](./production-readiness-ga.json). + +Note: GitHub release is still marked **pre-release** because the tag is `*-beta.*`. ## Install diff --git a/evidence/releases/v1.7.0-beta.9/production-readiness-ga.json b/evidence/releases/v1.7.0-beta.9/production-readiness-ga.json new file mode 100644 index 0000000..44ab7c4 --- /dev/null +++ b/evidence/releases/v1.7.0-beta.9/production-readiness-ga.json @@ -0,0 +1,200 @@ +{ + "generated_at": "2026-07-13T15:04:54Z", + "final_status": "PRODUCTION_GA_READY", + "current_stage": "PRODUCTION_GA_READY", + "recommendation": "PRODUCTION_GA_READY: all GA hardening gates verified.", + "pass": true, + "private_beta_ready": true, + "ga_ready": true, + "production_ready": true, + "online_benchmark_status": "pass", + "benchmark_candidate_eligible": true, + "benchmark_packages_configured": 25, + "benchmark_packages_executed": 25, + "benchmark_packages_skipped": 0, + "benchmark_coverage_ratio": 1, + "github_action_status": "valid", + "signed_release_status": "signed", + "signing_configured": true, + "signing_verified": true, + "sbom_status": "present", + "sbom_verified": true, + "checksums_status": "verified", + "checksums_verified": true, + "provenance_status": "verified", + "provenance_configured": true, + "provenance_verified": true, + "docs_status": "complete", + "real_repo_validation_count": 15, + "required_real_repo_validation_count": 15, + "repo_validation_pass_rate": 1, + "ecosystem_depth_status": "npm-ga-pypi-public-beta-go-cargo-preview", + "isolated_backend_status": "unavailable", + "isolated_backend_available": false, + "behavior_analysis_default_mode": "disabled", + "npm_repo_count": 12, + "pypi_repo_count": 4, + "go_repo_count": 0, + "cargo_repo_count": 0, + "false_block_count": 0, + "scanner_crash_count": 0, + "average_scan_duration_ms": 1, + "p95_scan_duration_ms": 2, + "average_scan_duration_us": 875, + "p95_scan_duration_us": 1348, + "scan_timing_trustworthy": true, + "scan_timing_floor_count": 8, + "critical_detection_rate": 1, + "known_good_false_block_rate": 0, + "private_beta_recommendation": true, + "gates": [ + { + "name": "rollout readiness", + "passed": true, + "blocking": true, + "duration_ms": 24300, + "summary": "PRIVATE_BETA_READY", + "details": [ + "GO for private beta rollout. Continue to keep lifecycle behavior analysis labelled best-effort." + ] + }, + { + "name": "benchmark validation", + "passed": true, + "blocking": true, + "duration_ms": 0, + "summary": "PRIVATE_BETA_ACCURACY_CANDIDATE", + "details": [ + "direct recall 100.00%", + "transitive recall 100.00%", + "source import recall 100.00%", + "packages configured=25 executed=25 skipped=0 coverage=100.00% candidate_eligible=true" + ] + }, + { + "name": "real repository evidence", + "passed": true, + "blocking": true, + "duration_ms": 0, + "summary": "real repository evidence threshold met", + "details": [ + "validated=15 required=15 passed=15 failed=0" + ] + }, + { + "name": "online benchmark", + "passed": true, + "blocking": false, + "duration_ms": 0, + "summary": "online benchmark: pass", + "details": [ + "mode=connected attempted=25 passed=25 failed=0 network_unavailable=0 registry_unavailable=0 package_not_found=0 scanner_failure=0 expectation_mismatch=0" + ] + }, + { + "name": "OSV cache", + "passed": true, + "blocking": true, + "duration_ms": 412, + "summary": "OSV database is initialized", + "details": [ + "records: 260056", + "path: /Users/bhushan/.pkgsafe/pkgsafe.db" + ] + }, + { + "name": "CI outputs", + "passed": true, + "blocking": true, + "duration_ms": 2, + "summary": "JSON, SARIF, and Markdown outputs generated", + "details": [ + "/var/folders/38/czrn5d6s5992dpmtjy_cn6rm0000gn/T/pkgsafe-ci-output-1648353453/results.json", + "/var/folders/38/czrn5d6s5992dpmtjy_cn6rm0000gn/T/pkgsafe-ci-output-1648353453/results.sarif", + "/var/folders/38/czrn5d6s5992dpmtjy_cn6rm0000gn/T/pkgsafe-ci-output-1648353453/summary.md" + ] + }, + { + "name": "documentation", + "passed": true, + "blocking": true, + "duration_ms": 0, + "summary": "production docs exist", + "details": [ + "README.md", + "SECURITY.md", + "docs/ci-cd.md", + "docs/github-action.md", + "docs/mcp-codex.md", + "docs/policy-guide.md", + "docs/private-registry.md", + "docs/known-limitations.md", + "docs/threat-model.md", + "docs/release-verification.md" + ] + }, + { + "name": "policy validation", + "passed": true, + "blocking": true, + "duration_ms": 1, + "summary": "default policy is valid", + "details": [ + "default-policy.yaml" + ] + }, + { + "name": "release artifacts", + "passed": true, + "blocking": false, + "duration_ms": 56, + "summary": "checksums and SBOM exist and checksums verify", + "details": [ + "/Users/bhushan/Documents/Projects/pkgsafe/dist/ga-verify/checksums.txt", + "/Users/bhushan/Documents/Projects/pkgsafe/dist/ga-verify/pkgsafe_1.7.0-beta.9_darwin_amd64.tar.gz.sbom.json" + ] + }, + { + "name": "github action", + "passed": true, + "blocking": false, + "duration_ms": 0, + "summary": "composite action, entrypoint, and example workflow present", + "details": [ + "action.yml", + "scripts/github-action-entrypoint.sh", + ".github/workflows/pkgsafe-action-example.yml" + ] + }, + { + "name": "signed release", + "passed": true, + "blocking": false, + "duration_ms": 1773, + "summary": "cosign signature verified for checksums.txt", + "details": [ + "Flag --certificate has been deprecated, please use --bundle with --trusted-root to provide the public certificate\nFlag --signature has been deprecated, please use --bundle to provide a signature\nVerified OK" + ] + }, + { + "name": "sbom", + "passed": true, + "blocking": false, + "duration_ms": 0, + "summary": "SBOM present", + "details": [ + "/Users/bhushan/Documents/Projects/pkgsafe/dist/ga-verify/pkgsafe_1.7.0-beta.9_darwin_amd64.tar.gz.sbom.json" + ] + }, + { + "name": "build provenance", + "passed": true, + "blocking": false, + "duration_ms": 14514, + "summary": "GitHub build provenance attestation verified", + "details": [ + "gh attestation verify succeeded" + ] + } + ] +} diff --git a/scripts/verify-ga-release.sh b/scripts/verify-ga-release.sh new file mode 100755 index 0000000..0fa417f --- /dev/null +++ b/scripts/verify-ga-release.sh @@ -0,0 +1,112 @@ +#!/usr/bin/env bash +# Verify a published PkgSafe release for PRODUCTION_GA_READY. +# +# Usage: +# scripts/verify-ga-release.sh [VERSION] +# scripts/verify-ga-release.sh v1.7.0-beta.9 +# +# Requires: gh, cosign, go (for make build), network access to GitHub releases. +# Optional env: +# PKGSAFE_GITHUB_REPO default: sairintechnologycom/pkgsafe +# PKGSAFE_ARTIFACT_DIR default: dist/ga-verify +# SKIP_BUILD=1 use existing ./dist/pkgsafe + +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$ROOT_DIR" + +VERSION_RAW="${1:-v1.7.0-beta.9}" +VERSION="${VERSION_RAW#v}" +TAG="v${VERSION}" +REPO="${PKGSAFE_GITHUB_REPO:-sairintechnologycom/pkgsafe}" +ART_DIR="${PKGSAFE_ARTIFACT_DIR:-$ROOT_DIR/dist/ga-verify}" + +need() { + command -v "$1" >/dev/null 2>&1 || { + echo "error: required tool not found: $1" >&2 + exit 2 + } +} + +need gh +need cosign +need shasum + +echo "==> Download release $TAG into $ART_DIR" +rm -rf "$ART_DIR" +mkdir -p "$ART_DIR" +gh release download "$TAG" --repo "$REPO" -D "$ART_DIR" + +echo "==> Verify checksums" +( + cd "$ART_DIR" + shasum -a 256 -c checksums.txt +) + +echo "==> Verify cosign signature on checksums.txt" +cosign verify-blob \ + --certificate "$ART_DIR/checksums.txt.pem" \ + --signature "$ART_DIR/checksums.txt.sig" \ + --certificate-identity-regexp 'https://github.com/.*/pkgsafe/.*' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + "$ART_DIR/checksums.txt" + +ARCHIVE="$(ls "$ART_DIR"/pkgsafe_*_darwin_arm64.tar.gz 2>/dev/null | head -1 || true)" +if [ -z "$ARCHIVE" ]; then + ARCHIVE="$(ls "$ART_DIR"/pkgsafe_*.tar.gz "$ART_DIR"/pkgsafe_*.zip 2>/dev/null | head -1 || true)" +fi +if [ -z "$ARCHIVE" ]; then + echo "error: no release archive found in $ART_DIR" >&2 + exit 1 +fi + +echo "==> Verify GitHub attestation for $(basename "$ARCHIVE")" +gh attestation verify "$ARCHIVE" \ + --repo "$REPO" \ + --signer-workflow "github.com/${REPO}/.github/workflows/release.yml" + +if [ "${SKIP_BUILD:-0}" != "1" ]; then + echo "==> Build local pkgsafe for readiness command" + make build +fi +if [ ! -x ./dist/pkgsafe ]; then + echo "error: ./dist/pkgsafe missing; run make build" >&2 + exit 1 +fi + +echo "==> Run production-readiness with verified artifact dir" +export PKGSAFE_RELEASE_ARTIFACT_DIR="$ART_DIR" +export PKGSAFE_GITHUB_REPO="$REPO" +OUT_JSON="${PKGSAFE_READINESS_JSON:-$ROOT_DIR/evidence/releases/${TAG}/production-readiness-ga.json}" +mkdir -p "$(dirname "$OUT_JSON")" + +./dist/pkgsafe test production-readiness --json | tee "$OUT_JSON.tmp" >/dev/null +# Keep only JSON object (command may print nothing else) +python3 - "$OUT_JSON.tmp" "$OUT_JSON" <<'PY' +import json, sys +raw = open(sys.argv[1]).read() +i = raw.find("{") +if i < 0: + raise SystemExit("no JSON object in production-readiness output") +data = json.loads(raw[i:]) +open(sys.argv[2], "w").write(json.dumps(data, indent=2) + "\n") +status = data.get("final_status") +ga = data.get("ga_ready") +blockers = data.get("ga_blockers") or [] +print(f"final_status={status}") +print(f"ga_ready={ga}") +print(f"production_ready={data.get('production_ready')}") +print(f"ga_blockers={blockers}") +print(f"signing_verified={data.get('signing_verified')}") +print(f"provenance_verified={data.get('provenance_verified')}") +print(f"real_repos={data.get('real_repo_validation_count')}/{data.get('required_real_repo_validation_count')}") +if status != "PRODUCTION_GA_READY" or not ga or blockers: + raise SystemExit(1) +PY +rm -f "$OUT_JSON.tmp" + +echo +echo "PRODUCTION_GA_READY verified for $TAG" +echo "Evidence JSON: $OUT_JSON" +echo "Artifact dir: $ART_DIR"