Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions .agents/skills/td-supply-chain-audit/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,12 @@ The compromise-date is the date the package is suspected to have been compromise

- `gh` CLI installed and authenticated (`gh auth status` must succeed)
- `python3` available (3.10+)
- Network access to GitHub API and PyPI/npm registries
- Network access to GitHub API, PyPI/npm registries, and GitHub releases (for Scorecard CLI auto-bootstrap)
- GitHub token available to `gh` / `GH_TOKEN` (Scorecard CLI uses it for API rate limits)
- `playwright` Python package with Chromium (for PDF export): `pip install playwright && playwright install chromium`

Do **not** ask the user to install Scorecard manually. `collect.py` auto-downloads the pinned `scorecard` binary into `.supply-chain-audit/bin/` when missing (**macOS, Linux, and Windows** — amd64/arm64), then scores every target repo that has no published OpenSSF API result. The CLI fallback runs the full Scorecard suite **except** `Vulnerabilities` (OSV dependency scan) — that check can hang for 15+ minutes on larger repos, and dependency CVEs are already covered by the audit's separate OSV.dev inventory pass.

## Instructions

### Step 1: Validate inputs
Expand Down Expand Up @@ -113,10 +116,13 @@ This will:
- Fetch commits, PRs, check suites, and dependency diffs for all target repos
- Fetch all individual commits and review timelines within each merged PR
- Fetch branch protection rules and rulesets for each repo
- Detect OpenSSF Scorecard workflow presence and fetch published scores from the Scorecard API
- Auto-bootstrap the Scorecard CLI if needed, then score every repo with no API result (all target environments/repos; skip only with `--skip-scorecard-cli`)
- When commit/PR data is cached but Scorecard scores are still missing, automatically re-run Scorecard collection for those repos
- Store results as JSON in the cache directory
- Write a `manifest.json` for reproducibility

The script is idempotent: if cache files already exist for the same time frame, they are reused without re-fetching.
The script is idempotent: if cache files already exist for the same time frame, they are reused without re-fetching (Scorecard is refreshed automatically when scores are still unavailable).

Monitor progress output. The script prints per-repo status. If rate-limited, it will back off automatically.

Expand All @@ -127,7 +133,7 @@ python3 .agents/skills/td-supply-chain-audit/scripts/analyze.py \
--cache-dir ".supply-chain-audit/cache"
```

This detects (13 passes):
This detects (14 passes):
- Unsigned commits
- GitHub-web-signed commits (signer is GitHub, not a personal key)
- Orphan commits (no associated PR)
Expand All @@ -141,6 +147,7 @@ This detects (13 passes):
- Bot-only approvals (PRs merged without any human review)
- Self-approved PRs (author approved their own code with no independent review)
- Known vulnerabilities (all current packages scanned against OSV.dev)
- OpenSSF Scorecard gaps (unpublished results, low published API aggregate score, weak critical checks; missing workflows are table-only, not findings)

Output: `findings.json` in the cache directory.

Expand All @@ -165,12 +172,13 @@ After analysis completes, **you** (the agent) must read the findings and write a
2. If you need more detail on specific findings, read the full: `.supply-chain-audit/cache/<hash>/findings.json`
3. Read the protection rules: `.supply-chain-audit/cache/<hash>/protection/*.json`
4. Read the renovate configs: `.supply-chain-audit/cache/<hash>/renovate/*.json`
5. Reason about the most impactful actions the team should take based on:
5. Read the Scorecard data: `.supply-chain-audit/cache/<hash>/scorecard/*.json`
6. Reason about the most impactful actions the team should take based on:
- Severity and count of findings by category
- Patterns across repos (e.g., many repos missing the same protection)
- Quick wins vs. systemic improvements
- What would prevent the *worst* findings from recurring
5. Write `.supply-chain-audit/cache/<hash>/recommendations.json` as a JSON array of objects:
7. Write `.supply-chain-audit/cache/<hash>/recommendations.json` as a JSON array of objects:

```json
[
Expand Down Expand Up @@ -206,6 +214,7 @@ This produces a standalone HTML file (no CDN dependencies) with:
- Timeline visualization (SVG)
- Commit integrity table (sortable, filterable)
- Dependency changes table with release dates
- OpenSSF Scorecard workflow and score table
- Suspicious patterns grouped by category
- Security recommendations (from step 5)
- Package focus section (if Phase 2 data exists)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -585,3 +585,67 @@ package inventory:
4. Assess whether the vulnerable code path is actually exercised
5. For critical/high: open an issue or PR to update immediately
6. For medium/low: schedule update via normal renovate cycle

---

## 14. OpenSSF Scorecard

**Category:** `scorecard`
**Default Risk:** High (published API score &lt; 5), Medium (API score &lt; 7 or weak
checks), Low/Info (workflow hygiene / CLI-only snapshot). Missing workflows are
shown in the Scorecard table only (not anomaly findings).

### What it detects

Gaps in OpenSSF Scorecard adoption and published posture for each audited repo:

1. **Incomplete workflow** — present but missing `publish_results`, schedule
trigger, or SARIF/code-scanning upload (or wrong action)
2. **Unpublished results** — workflow exists but OpenSSF API has no score yet
3. **Low aggregate score** — published API score below 7 (medium) or 5 (high);
not applied to CLI snapshots (CLI omits `Vulnerabilities`)
4. **Weak critical checks** — Token-Permissions, Dangerous-Workflow,
Branch-Protection, Code-Review, Maintained, Pinned-Dependencies, or
Security-Policy scoring below 5
5. **Missing workflow** — tracked in the report table only (no finding spam)

### Why it matters

Scorecard is the ecosystem-standard continuous signal for supply-chain
hygiene. Without the workflow (as introduced for abbenay in
[PR #57](https://github.com/redhat-developer/abbenay/pull/57)), scores drift,
badges stay stale, and code-scanning never receives SARIF findings. Weak
checks such as token permissions or unpinned actions are common entry points
for workflow compromise.

### Data sources

- GitHub Contents API: `.github/workflows/*scorecard*`
- OpenSSF Scorecard API:
`https://api.securityscorecards.dev/projects/github.com/{org}/{repo}`
- Local Scorecard CLI fallback (`scorecard --repo=github.com/{org}/{repo}
--format=json --checks=...`) when the API returns no published score.
`collect.py` auto-downloads a pinned CLI into `.supply-chain-audit/bin/` on
macOS, Linux, and Windows (amd64/arm64) and uses `gh`/env GitHub tokens for
rate limits. CLI runs omit the `Vulnerabilities` check (slow OSV walk;
covered by the audit's separate OSV.dev pass) and are labeled `source=cli`
with `cli_checks_excluded: ["Vulnerabilities"]`. CLI scores do not publish
to OpenSSF.

### False positive scenarios

- Brand-new repos where Scorecard has not completed a default-branch run yet
(INFO: unpublished results)
- Checks scored `-1` (not applicable) — ignored by the analyzer
- Private repos that cannot publish results without a PAT
- CLI scores may differ slightly from a later published API score (timing /
Scorecard version); treat CLI as an audit-time snapshot until workflows publish

### Investigation steps

1. Confirm whether `.github/workflows/scorecard.yml` exists on the default branch
2. Verify triggers include `push` to default branch, weekly `schedule`, and
optionally `branch_protection_rule`
3. Ensure `publish_results: true` and SARIF upload to code scanning
4. Open the OpenSSF API URL / badge and remediate failing critical checks
5. Re-run the workflow on `main` after fixes and re-audit
Loading
Loading