From 15545d78d022ae034d98062b67316468cf996649 Mon Sep 17 00:00:00 2001 From: Ben Limmer Date: Tue, 26 May 2026 14:02:13 -0600 Subject: [PATCH 1/2] docs: rework the README and fix the analyze.sh output comment Restructure the README around a one-line install, GitHub auth setup, what the tool reads from GitHub vs. what it reports, opening the report in a browser, and a tight telemetry/opt-out section. Document a read-only fine-grained token alongside gh auth login. Also fix the stale "current directory" claim in analyze.sh: the binary writes its report to a temp dir and prints the path. --- README.md | 99 +++++++++++++++++++++------------------------- scripts/analyze.sh | 4 +- 2 files changed, 48 insertions(+), 55 deletions(-) diff --git a/README.md b/README.md index cf2fa0a..24af50e 100644 --- a/README.md +++ b/README.md @@ -1,96 +1,89 @@ -# patchwave-analysis +# [PatchWave](https://patchwave.ai) Analysis -A diagnostic CLI that measures Dependabot toil and CVE exposure across a GitHub org. It writes a self-contained HTML report. - -## What it tells you - -Given an org or user, the report covers: - -- **Org overview** — repo count, public/private split, language mix, branch-protection coverage, active human committers -- **Dependabot coverage** — percent of repos with config, security-updates status, ecosystems, Node package manager split -- **PR backlog** — open vs merged vs closed in window, age buckets, bump-type split, time-to-merge p50/p90, CI status mix, dev-only-dep share -- **Stalled-PR signals** — repos at Dependabot's PR cap, repos with config but no recent PRs, reverts in window -- **People** — top human mergers and reviewers -- **Toil cost** — annualized engineer-time cost with browser-adjustable assumptions -- **CVE exposure** — open Dependabot security alerts by severity, oldest open Critical/High -- **Automation upside** — projected savings for common auto-merge rates +PatchWave Analysis is a free diagnostic CLI that measures Dependabot toil and CVE exposure across a GitHub org. It reads from the GitHub API and writes a self-contained HTML report you can use on its own, no PatchWave account needed. ## Run it -You need a GitHub token with `repo` and `read:org` scopes (add `security_events` for CVE metrics). The CLI resolves it from `GITHUB_TOKEN`, then `GH_TOKEN`, then `gh auth token` — so if you're signed in with the `gh` CLI there's nothing to set. - -### One-off run (recommended) - ```sh bash -c "$(curl -fsSL https://patchwave.ai/analyze.sh)" ``` -This downloads the signed binary for your platform from the latest release, verifies its checksum, runs the interactive session, and cleans up after itself — nothing is installed. The report is written to your current directory. Pin a specific release with `PW_VERSION`: +This grabs the latest signed binary for your platform, verifies its checksum, runs the interactive session, then deletes the binary. -```sh -PW_VERSION=v0.1.0 bash -c "$(curl -fsSL https://patchwave.ai/analyze.sh)" -``` +### Or grab the binary yourself -### Download the binary yourself - -Grab the archive for your platform from the [latest release](https://github.com/contextbridge/patchwave-analysis/releases/latest), then: +Download the archive for your platform from the [latest release](https://github.com/contextbridge/patchwave-analysis/releases/latest), then unpack and run it: ```sh tar -xzf patchwave-analysis_darwin_arm64.tar.gz ./patchwave-analysis ``` -### From source +## Setup + +Easiest path: run `gh auth login` (via [GitHub CLI](https://cli.github.com)) and you're done. The CLI also reads `GITHUB_TOKEN` and `GH_TOKEN`, so you can pass a token directly instead. + +Prefer a custom token? Create a [fine-grained token](https://github.com/settings/personal-access-tokens/new) instead and set these repository permissions to read: + +- **Contents:** commit history and the `dependabot.yml` config +- **Pull requests:** the Dependabot PR backlog +- **Administration:** branch-protection and ruleset coverage +- **Dependabot alerts:** the CVE numbers + +Then export it and run: ```sh -git clone https://github.com/contextbridge/patchwave-analysis -cd patchwave-analysis -just install -just run +export GITHUB_TOKEN=github_pat_... +bash -c "$(curl -fsSL https://patchwave.ai/analyze.sh)" ``` -## Usage +Your org has to allow fine-grained tokens for this to reach its repos. + +Whichever you pick, the CLI only reads from the API. It never writes. -```text -patchwave-analysis [] +## What it tells you -If is omitted, you are prompted for it. +The report covers: - --help show this help -``` +- **Dependabot coverage:** which repos have config, whether security updates are on, and for which ecosystems +- **PR backlog:** open vs. merged vs. closed, age buckets, time-to-merge, CI statuses +- **Stalled signals:** repos sitting at Dependabot's PR cap, or configured but quiet +- **CVE exposure:** open security alerts by severity, plus the oldest unpatched Critical/High +- **Toil cost:** annualized engineer-time, with assumptions you can adjust right in the browser +- **Automation upside:** projected savings with [PatchWave](https://patchwave.ai) -The CLI takes a single optional argument — the org or user to scan. There are no other flags; the time window (90 days) and output location are fixed. +## What it reads from GitHub -## Output +Everything comes from `api.github.com` over a fixed 90-day window. For the org and its repos (archived repos and forks are skipped), it reads: -Each run writes a single file into a fresh temporary directory and prints the full path when the scan finishes: +- The repo list, visibility, and language breakdown +- Dependabot PRs in the window, including state, timing, reviews, and CI status +- Open Dependabot security alerts (needs the `security_events` scope) +- Each repo's `.github/dependabot.yml` +- Branch-protection and ruleset settings on the default branch +- Commit authors in the window, to count active humans -- **`patchwave-report.html`** — the self-contained browser report. Open it locally; it embeds the rolled-up data that drives every metric. This is the artifact to send back when you want a deeper look from contextbridge. +All calls are read only. It writes nothing back to GitHub and pulls no file contents beyond the Dependabot config. -The report is not uploaded unless you choose to share it. It does not include tokens, secrets, or repository file contents. +## Output -## What it does not do +When the scan finishes, the CLI writes `patchwave-report.html` to a fresh temporary directory and offers to open it in your default browser. -- It does not upload the report or any GitHub data unless you choose to share it. It reads from `api.github.com`. Filesystem writes are limited to the `patchwave-report.html` file in a temporary directory and a one-time anonymous-id file (see Telemetry & privacy). -- It does not keep a Markdown compatibility report. -- It does not auto-update. +The report is one self-contained file with every metric baked in. It carries no tokens or source code, just the rolled-up numbers. ## Telemetry & privacy -Official binaries send product analytics and crash diagnostics so we can improve the tool. Builds from source do not include telemetry keys. +We send anonymous usage events and crash reports so we can improve the tool. Org names, repo names, tokens, report contents, and your hostname are never sent. Builds from source send nothing. -Disable telemetry with any of: +To turn it off, set any of these: - `DO_NOT_TRACK=1` - `CONTEXTBRIDGE_TELEMETRY_DISABLED=1` -- `CI=1` - -When disabled, no anonymous-id file is created, no analytics events are sent, and Sentry is never initialized. ## Contributing -Development setup, testing, and release workflow live in [`CONTRIBUTING.md`](./CONTRIBUTING.md). +Development setup, testing, and the release workflow live in [`CONTRIBUTING.md`](./CONTRIBUTING.md). ## License -MIT. +MIT diff --git a/scripts/analyze.sh b/scripts/analyze.sh index 5044219..31379ed 100644 --- a/scripts/analyze.sh +++ b/scripts/analyze.sh @@ -7,8 +7,8 @@ # # Downloads the patchwave-analysis binary for your platform from the latest # GitHub release into a temp dir, verifies its checksum, runs it, then deletes -# it. The binary runs an interactive session and writes its report to your -# current directory. Nothing is installed. +# it. The binary runs an interactive session and writes its report into a +# temporary directory, printing the path when it finishes. Nothing is installed. # # Use the `bash -c "$(curl ...)"` form rather than `curl ... | bash`: the # command-substitution form leaves your terminal on stdin so the prompts work. From ba9715fef0e6005399daa2fd6a81b27f9c1c4e02 Mon Sep 17 00:00:00 2001 From: Ben Limmer Date: Tue, 26 May 2026 14:02:15 -0600 Subject: [PATCH 2/2] docs: offer a read-only fine-grained token in the CLI walkthrough The token prompt only described the classic `repo` path, which is read+write. Add the read-only fine-grained alternative (Contents, Pull requests, Administration, Dependabot alerts) to both the gh and PAT instructions, and note that the CLI only ever reads. --- src/interactive/tokenWalkthrough.test.ts | 6 ++++++ src/interactive/tokenWalkthrough.ts | 13 +++++++++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/src/interactive/tokenWalkthrough.test.ts b/src/interactive/tokenWalkthrough.test.ts index 88f47d0..47704c2 100644 --- a/src/interactive/tokenWalkthrough.test.ts +++ b/src/interactive/tokenWalkthrough.test.ts @@ -43,6 +43,9 @@ describe('interactiveResolveToken', () => { expect(note?.message).toContain('gh auth login --scopes'); expect(note?.message).toContain('repo,read:org,security_events'); expect(note?.message).not.toContain('github.com/settings/tokens'); + // also offers the read-only fine-grained alternative + expect(note?.message).toContain('fine-grained'); + expect(note?.message).toContain('Dependabot alerts'); }); test('shows PAT instructions matching the GitHub UI when gh is not installed', async () => { @@ -64,6 +67,9 @@ describe('interactiveResolveToken', () => { expect(note?.message).toContain('[x] read:org'); expect(note?.message).toContain('Generate token'); expect(note?.message).toContain('export GITHUB_TOKEN=ghp_'); + // also offers the read-only fine-grained alternative + expect(note?.message).toContain('settings/personal-access-tokens/new'); + expect(note?.message).toContain('Dependabot alerts'); }); test('user declines the retry prompt: returns cancelled', async () => { diff --git a/src/interactive/tokenWalkthrough.ts b/src/interactive/tokenWalkthrough.ts index 44d4b35..be8dc66 100644 --- a/src/interactive/tokenWalkthrough.ts +++ b/src/interactive/tokenWalkthrough.ts @@ -61,19 +61,23 @@ function defaultHasGhCli(): boolean { function ghInstructions(): string { return [ - "We need a GitHub token to read your org's Dependabot data.", + "We need a GitHub token to read your org's Dependabot data. The CLI only reads; it never writes.", '', 'I see the gh CLI installed. In another terminal, run:', '', ' gh auth login --scopes "repo,read:org,security_events"', '', 'Pick GitHub.com → HTTPS → "Login with a web browser" and follow the prompts.', + '', + 'Want least privilege? A read-only fine-grained token works too. Grant these', + 'repository permissions, all Read-only: Contents, Pull requests, Administration,', + 'and Dependabot alerts, then export GITHUB_TOKEN.', ].join('\n'); } function patInstructions(): string { return [ - "We need a GitHub token to read your org's Dependabot data.", + "We need a GitHub token to read your org's Dependabot data. The CLI only reads; it never writes.", '', '1. Open https://github.com/settings/tokens/new', " (You'll be asked to confirm your password.)", @@ -97,6 +101,11 @@ function patInstructions(): string { ' Copy it, then in this terminal run:', '', ' export GITHUB_TOKEN=ghp_...', + '', + 'Prefer least privilege? A read-only fine-grained token works too, at', + 'https://github.com/settings/personal-access-tokens/new — grant these repository', + 'permissions, all Read-only: Contents, Pull requests, Administration, and', + 'Dependabot alerts. Your org must allow fine-grained tokens.', ].join('\n'); }