Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ agent.
| [Getting started](getting-started.md) | First install, first scan, first policy |
| [Install](install.md) | Binary install, Homebrew-style paths, Windows |
| [Release verification](release-verification.md) | Checksums, cosign, attestations, SBOM |
| [GA / production checklist](ga-checklist.md) | Path to PRODUCTION_GA_READY |
| [Commands](commands.md) | Full command list and common flags |
| [Troubleshooting](troubleshooting.md) | Doctor failures, offline mode, false blocks |

Expand All @@ -35,7 +36,7 @@ agent.
|-----|--------|
| [GitHub Action](github-action.md) | PR gates, SARIF, summary comments |
| [CI/CD](ci-cd.md) | `pkgsafe ci scan` on any runner |
| [Install interception](install-interception.md) | Safe wrappers around `npm` / `pip` |
| [Install interception](install-interception.md) | Safe wrappers around `npm` / `pnpm` / `yarn` / `pip` / `uv` |

## AI agents (MCP)

Expand Down
75 changes: 75 additions & 0 deletions docs/ga-checklist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Production / GA checklist

Use this when promoting PkgSafe from public beta to a production GA claim.

Automated source of truth:

```bash
scripts/verify-ga-release.sh vX.Y.Z
# or: pkgsafe test production-readiness --json
# with PKGSAFE_RELEASE_ARTIFACT_DIR pointing at a full verified release download
```

`ga_ready` and `production_ready` are true only when **all** automated GA blockers are cleared.

## Stages

| Stage | Meaning |
|-------|---------|
| `PRIVATE_BETA_READY` | Foundation tests, corpus, policy, docs |
| `PUBLIC_BETA_READY` | + online accuracy, action assets, real-repo evidence |
| `PRODUCTION_GA_READY` | + local cosign + checksum + SBOM + provenance verification |

## Automated GA requirements

| Gate | Threshold |
|------|-----------|
| Real-repo validations | ≥ 15, 100% pass |
| npm real repos | ≥ 5 |
| False blocks / crashes | 0 |
| Critical detection | 100% |
| Known-good false block rate | 0% |
| Scan timing | reported + trustworthy |
| Behavior analysis default | `disabled` |
| Online benchmark | pass |
| Release checksums | verified against local artifacts |
| Cosign signature | verified locally (`checksums.txt`) |
| SBOM | present and parseable SPDX JSON |
| Build provenance | `gh attestation verify` succeeds for a release archive |

## Manual release steps

1. **Main is green** — unit tests, CI, no open P0 security issues.
2. **Docs match maturity** — README/install/action language for npm (and PyPI if claimed); Go/Cargo still preview unless evidence exists.
3. **Tag** a release (`vX.Y.Z` for stable Latest, or keep `*-beta.*` if still pre-release).
4. **Release workflow** succeeds (GoReleaser, cosign, SBOM, attestations).
5. **Run** `scripts/verify-ga-release.sh vX.Y.Z` on a clean host with `gh` + `cosign`.
6. **Record evidence** under `evidence/releases/vX.Y.Z/` (readiness JSON + short summary).
7. **Homebrew / install.sh** smoke-tested on macOS and Linux.
8. **Customer playbook** for production:
- Intercept: `npm` / `pnpm` / `yarn` / `pip` / `uv` via shims
- Agents: MCP check tools; WARN requires human
- CI main/release: `pkgsafe ci scan --full --fail-on block`
- OSV: document update/bundle cadence

## Production usage (after GA)

| Do | Don't |
|----|--------|
| Use as pre-install firewall | Treat as full SCA replacement |
| `--mode block` in automation | Rely on default warn-mode scan exit 0 |
| `--full` on main/release CI | Assume changed-only ALLOW = clean monorepo |
| Keep behavior analysis disabled by default | Enable heuristic host execution in CI without isolation |

## Not required for OSS GA

- macOS isolated behavior backend
- Enterprise SSO / central policy / hosted evidence
- Every package manager under the sun (Poetry/conda remain out of intercept scope)

## Related

- [Release verification](release-verification.md)
- [Release integrity](release-integrity.md)
- [Known limitations](known-limitations.md)
- [CI/CD](ci-cd.md)
50 changes: 50 additions & 0 deletions evidence/releases/v1.7.0-beta.9/GA_VERIFICATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# v1.7.0-beta.9 GA verification (local)

**Date:** 2026-07-13
**Tag:** `v1.7.0-beta.9`
**Commit:** `c336499`
**Host:** macOS arm64, cosign + gh CLI

## Result

```text
final_status: PRODUCTION_GA_READY
ga_ready: true
production_ready: true
ga_blockers: []
real_repo_validation_count: 15/15
signing_verified: true
provenance_verified: true
checksums_status: verified
sbom_status: present
```

Full machine report: [production-readiness-ga.json](./production-readiness-ga.json)

## Ritual performed

1. Downloaded all release assets for `v1.7.0-beta.9` into `dist/ga-verify/`
(`gh release download v1.7.0-beta.9`).
2. `shasum -a 256 -c checksums.txt` → **12/12 OK**.
3. `cosign verify-blob` on `checksums.txt` → **Verified OK**.
4. `gh attestation verify pkgsafe_1.7.0-beta.9_darwin_arm64.tar.gz \
--repo sairintechnologycom/pkgsafe \
--signer-workflow github.com/sairintechnologycom/pkgsafe/.github/workflows/release.yml`
→ **exit 0**.
5. `PKGSAFE_RELEASE_ARTIFACT_DIR=$PWD/dist/ga-verify \
PKGSAFE_GITHUB_REPO=sairintechnologycom/pkgsafe \
./dist/pkgsafe test production-readiness --json`
→ **PRODUCTION_GA_READY**.

Repeatable automation:

```bash
scripts/verify-ga-release.sh v1.7.0-beta.9
```

## Notes

- Artifacts under `dist/` are gitignored; only this evidence JSON/summary is committed.
- Tag remains a **GitHub pre-release** (`*-beta.*`). Automated gate readiness does not
by itself publish a stable “Latest” channel; cut a non-beta tag when marketing GA.
- Isolated behavior backend was unavailable on this host (macOS); not a GA blocker.
10 changes: 7 additions & 3 deletions evidence/releases/v1.7.0-beta.9/RELEASE_SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,14 @@
| pnpm intercept BLOCK | PASS (exit 1 on axois) |
| CI `--full` coverage | PASS |

## Remaining GA blockers (expected)
## GA verification

- Signed release artifacts verified on the readiness host
- Build provenance verified locally
Local ritual completed 2026-07-13: **PRODUCTION_GA_READY** (`ga_ready: true`).

See [GA_VERIFICATION.md](./GA_VERIFICATION.md) and
[production-readiness-ga.json](./production-readiness-ga.json).

Note: GitHub release is still marked **pre-release** because the tag is `*-beta.*`.

## Install

Expand Down
200 changes: 200 additions & 0 deletions evidence/releases/v1.7.0-beta.9/production-readiness-ga.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
{
"generated_at": "2026-07-13T15:04:54Z",
"final_status": "PRODUCTION_GA_READY",
"current_stage": "PRODUCTION_GA_READY",
"recommendation": "PRODUCTION_GA_READY: all GA hardening gates verified.",
"pass": true,
"private_beta_ready": true,
"ga_ready": true,
"production_ready": true,
"online_benchmark_status": "pass",
"benchmark_candidate_eligible": true,
"benchmark_packages_configured": 25,
"benchmark_packages_executed": 25,
"benchmark_packages_skipped": 0,
"benchmark_coverage_ratio": 1,
"github_action_status": "valid",
"signed_release_status": "signed",
"signing_configured": true,
"signing_verified": true,
"sbom_status": "present",
"sbom_verified": true,
"checksums_status": "verified",
"checksums_verified": true,
"provenance_status": "verified",
"provenance_configured": true,
"provenance_verified": true,
"docs_status": "complete",
"real_repo_validation_count": 15,
"required_real_repo_validation_count": 15,
"repo_validation_pass_rate": 1,
"ecosystem_depth_status": "npm-ga-pypi-public-beta-go-cargo-preview",
"isolated_backend_status": "unavailable",
"isolated_backend_available": false,
"behavior_analysis_default_mode": "disabled",
"npm_repo_count": 12,
"pypi_repo_count": 4,
"go_repo_count": 0,
"cargo_repo_count": 0,
"false_block_count": 0,
"scanner_crash_count": 0,
"average_scan_duration_ms": 1,
"p95_scan_duration_ms": 2,
"average_scan_duration_us": 875,
"p95_scan_duration_us": 1348,
"scan_timing_trustworthy": true,
"scan_timing_floor_count": 8,
"critical_detection_rate": 1,
"known_good_false_block_rate": 0,
"private_beta_recommendation": true,
"gates": [
{
"name": "rollout readiness",
"passed": true,
"blocking": true,
"duration_ms": 24300,
"summary": "PRIVATE_BETA_READY",
"details": [
"GO for private beta rollout. Continue to keep lifecycle behavior analysis labelled best-effort."
]
},
{
"name": "benchmark validation",
"passed": true,
"blocking": true,
"duration_ms": 0,
"summary": "PRIVATE_BETA_ACCURACY_CANDIDATE",
"details": [
"direct recall 100.00%",
"transitive recall 100.00%",
"source import recall 100.00%",
"packages configured=25 executed=25 skipped=0 coverage=100.00% candidate_eligible=true"
]
},
{
"name": "real repository evidence",
"passed": true,
"blocking": true,
"duration_ms": 0,
"summary": "real repository evidence threshold met",
"details": [
"validated=15 required=15 passed=15 failed=0"
]
},
{
"name": "online benchmark",
"passed": true,
"blocking": false,
"duration_ms": 0,
"summary": "online benchmark: pass",
"details": [
"mode=connected attempted=25 passed=25 failed=0 network_unavailable=0 registry_unavailable=0 package_not_found=0 scanner_failure=0 expectation_mismatch=0"
]
},
{
"name": "OSV cache",
"passed": true,
"blocking": true,
"duration_ms": 412,
"summary": "OSV database is initialized",
"details": [
"records: 260056",
"path: /Users/bhushan/.pkgsafe/pkgsafe.db"
]
},
{
"name": "CI outputs",
"passed": true,
"blocking": true,
"duration_ms": 2,
"summary": "JSON, SARIF, and Markdown outputs generated",
"details": [
"/var/folders/38/czrn5d6s5992dpmtjy_cn6rm0000gn/T/pkgsafe-ci-output-1648353453/results.json",
"/var/folders/38/czrn5d6s5992dpmtjy_cn6rm0000gn/T/pkgsafe-ci-output-1648353453/results.sarif",
"/var/folders/38/czrn5d6s5992dpmtjy_cn6rm0000gn/T/pkgsafe-ci-output-1648353453/summary.md"
]
},
{
"name": "documentation",
"passed": true,
"blocking": true,
"duration_ms": 0,
"summary": "production docs exist",
"details": [
"README.md",
"SECURITY.md",
"docs/ci-cd.md",
"docs/github-action.md",
"docs/mcp-codex.md",
"docs/policy-guide.md",
"docs/private-registry.md",
"docs/known-limitations.md",
"docs/threat-model.md",
"docs/release-verification.md"
]
},
{
"name": "policy validation",
"passed": true,
"blocking": true,
"duration_ms": 1,
"summary": "default policy is valid",
"details": [
"default-policy.yaml"
]
},
{
"name": "release artifacts",
"passed": true,
"blocking": false,
"duration_ms": 56,
"summary": "checksums and SBOM exist and checksums verify",
"details": [
"/Users/bhushan/Documents/Projects/pkgsafe/dist/ga-verify/checksums.txt",
"/Users/bhushan/Documents/Projects/pkgsafe/dist/ga-verify/pkgsafe_1.7.0-beta.9_darwin_amd64.tar.gz.sbom.json"
]
},
{
"name": "github action",
"passed": true,
"blocking": false,
"duration_ms": 0,
"summary": "composite action, entrypoint, and example workflow present",
"details": [
"action.yml",
"scripts/github-action-entrypoint.sh",
".github/workflows/pkgsafe-action-example.yml"
]
},
{
"name": "signed release",
"passed": true,
"blocking": false,
"duration_ms": 1773,
"summary": "cosign signature verified for checksums.txt",
"details": [
"Flag --certificate has been deprecated, please use --bundle with --trusted-root to provide the public certificate\nFlag --signature has been deprecated, please use --bundle to provide a signature\nVerified OK"
]
},
{
"name": "sbom",
"passed": true,
"blocking": false,
"duration_ms": 0,
"summary": "SBOM present",
"details": [
"/Users/bhushan/Documents/Projects/pkgsafe/dist/ga-verify/pkgsafe_1.7.0-beta.9_darwin_amd64.tar.gz.sbom.json"
]
},
{
"name": "build provenance",
"passed": true,
"blocking": false,
"duration_ms": 14514,
"summary": "GitHub build provenance attestation verified",
"details": [
"gh attestation verify succeeded"
]
}
]
}
Loading
Loading