From 60548d5b8dbbae024ccdd1b6fe77964e445d3f42 Mon Sep 17 00:00:00 2001 From: Roberto Iskandarani Date: Fri, 7 Aug 2026 13:53:10 -0300 Subject: [PATCH 1/2] ci: pin the conformance catalog by SHA instead of cloning its default branch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI cloned github.com/AuthPlane/conformance at its default branch, so any change to the catalog reached this repo immediately — a case added there could turn an unrelated PR red here, with nothing in this repo having changed. Pinning decouples them: a catalog change arrives only when this repo deliberately bumps the pin together with the coverage for it. The revision is single-sourced in a tracked .conformance-catalog-ref, guarded by a ^[0-9a-f]{40}$ shape check so a branch or tag name cannot silently un-pin CI, and read by both ci.yml and release.yml through one script rather than a copy-pasted block — the pin was single-sourced before, the logic reading it was not. A weekly conformance-catalog-drift workflow clones the unpinned tip and runs the alignment assertion, so new cases surface as an early warning instead of a surprise. Its report step distinguishes real drift from a harness failure by matching the two messages that mean drift rather than the marker prefix, which is also emitted when the catalog cannot be read at all. The alignment step declares shell: bash so the pipeline gets pipefail — without it the step takes tee's exit status and reports success unconditionally. The catalog now lands in $RUNNER_TEMP rather than the workspace, so it stays out of `go list ./...`, coverage globs, and `git add -A` in the release commit. No SDK source changes. Verified locally: the fetch script runs end-to-end against the pinned revision, and the conformance suite passes against the catalog it checks out — the repo already aligns with this revision, so the pin adopts no new cases. --- .conformance-catalog-ref | 1 + .github/scripts/fetch-conformance-catalog.sh | 53 +++++++++ .github/workflows/ci.yml | 6 +- .../workflows/conformance-catalog-drift.yml | 103 ++++++++++++++++++ .github/workflows/release.yml | 9 +- 5 files changed, 162 insertions(+), 10 deletions(-) create mode 100644 .conformance-catalog-ref create mode 100755 .github/scripts/fetch-conformance-catalog.sh create mode 100644 .github/workflows/conformance-catalog-drift.yml diff --git a/.conformance-catalog-ref b/.conformance-catalog-ref new file mode 100644 index 0000000..efa9db0 --- /dev/null +++ b/.conformance-catalog-ref @@ -0,0 +1 @@ +b4c758a7dac698d7fcacd32dafcd4bb2f5dbddaf diff --git a/.github/scripts/fetch-conformance-catalog.sh b/.github/scripts/fetch-conformance-catalog.sh new file mode 100755 index 0000000..5652a5b --- /dev/null +++ b/.github/scripts/fetch-conformance-catalog.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +# +# Fetch the conformance catalog at the revision this repo pins. +# +# The catalog lives in github.com/AuthPlane/conformance and is updated +# independently of this repo, so cloning its default branch would let a catalog +# change turn an unrelated PR red here. The ref is pinned instead, single-sourced +# from the tracked .conformance-catalog-ref at the repo root — bump it there when +# adopting new catalog cases, together with the coverage for them, so a catalog +# change can never break CI on its own. +# +# This script exists because the read/guard/fetch sequence is needed by more than +# one workflow (ci.yml and release.yml). Keeping it inline in both meant the +# guard could be tightened in one and not the other; the pin was single-sourced +# but the logic reading it was not. +# +# Clones into $RUNNER_TEMP — outside $GITHUB_WORKSPACE — so the catalog stays out +# of the working tree: it must never trip `go list ./...` or a coverage glob, and +# `git add -A` in the release commit must never stage it as a gitlink. +# +# Requires: GITHUB_WORKSPACE, RUNNER_TEMP. + +set -euo pipefail + +: "${GITHUB_WORKSPACE:?GITHUB_WORKSPACE must be set}" +: "${RUNNER_TEMP:?RUNNER_TEMP must be set}" + +REF_FILE="$GITHUB_WORKSPACE/.conformance-catalog-ref" +DEST="$RUNNER_TEMP/conformance" +CATALOG_REPO="https://github.com/AuthPlane/conformance.git" + +if [[ ! -f "$REF_FILE" ]]; then + echo "::error::$REF_FILE is missing; the conformance catalog revision is unpinned" + exit 1 +fi + +CONFORMANCE_CATALOG_REF="$(tr -d '[:space:]' < "$REF_FILE")" + +# Guard against un-pinning: the ref must be a full commit SHA, not a branch or +# tag name, either of which would silently track a moving target. +if ! grep -Eq '^[0-9a-f]{40}$' <<< "$CONFORMANCE_CATALOG_REF"; then + echo "::error::.conformance-catalog-ref must be a 40-hex commit SHA, got '$CONFORMANCE_CATALOG_REF'" + exit 1 +fi + +git init -q "$DEST" +if ! git -C "$DEST" fetch --depth=1 "$CATALOG_REPO" "$CONFORMANCE_CATALOG_REF"; then + echo "::error::Pinned conformance catalog ref $CONFORMANCE_CATALOG_REF is unreachable" + exit 1 +fi +git -C "$DEST" checkout -q FETCH_HEAD + +echo "Conformance catalog checked out at $CONFORMANCE_CATALOG_REF" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e5d1adf..42dd3dd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -38,10 +38,8 @@ jobs: # conformance suite). - name: Clone shared conformance catalog (out of tree) if: matrix.module == 'core' - run: | - git -c advice.detachedHead=false clone --depth=1 \ - https://github.com/AuthPlane/conformance.git \ - "$RUNNER_TEMP/conformance" + shell: bash + run: .github/scripts/fetch-conformance-catalog.sh - name: Setup Go uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0 diff --git a/.github/workflows/conformance-catalog-drift.yml b/.github/workflows/conformance-catalog-drift.yml new file mode 100644 index 0000000..700135a --- /dev/null +++ b/.github/workflows/conformance-catalog-drift.yml @@ -0,0 +1,103 @@ +name: Conformance Catalog Drift + +# The main CI (ci.yml) and release (release.yml) workflows pin the shared +# conformance catalog to a fixed SHA (.conformance-catalog-ref) so a catalog +# change can never break PR CI on its own. The trade-off is that new catalog +# cases stay invisible until someone bumps the pin. This job closes that gap: +# on a weekly schedule it runs the SDK's catalog-alignment check against the +# LATEST (unpinned) default branch of the catalog and FAILS the job on any +# drift, so the scheduled run goes red and GitHub notifies maintainers (the +# same convention as security.yml). This workflow has no pull_request trigger, +# so a failure here can never block a PR. +# +# When this job fails on drift, adopt the new cases in core/conformancetests/ +# and bump .conformance-catalog-ref to the new catalog SHA in the same change. + +on: + schedule: + # Mondays at 06:00 UTC. + - cron: "0 6 * * 1" + workflow_dispatch: + +# Least-privilege default: this workflow only reads the repo. +permissions: + contents: read + +jobs: + drift: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + + # Clone the catalog's DEFAULT branch (latest, unpinned) — deliberately + # NOT the pinned .conformance-catalog-ref — so newly added cases show up. + # Cloned to $RUNNER_TEMP, outside $GITHUB_WORKSPACE, so it stays out of + # the working tree. Source: github.com/AuthPlane/conformance. + - name: Clone latest conformance catalog (out of tree) + run: | + git clone --depth=1 https://github.com/AuthPlane/conformance.git \ + "$RUNNER_TEMP/conformance" + + - name: Setup Go + uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0 + with: + go-version-file: "core/go.mod" + check-latest: true + cache-dependency-path: "core/go.sum" + + # The alignment check runs in TestMain, AFTER m.Run(), so the full suite + # must execute for every Case() registration to fire — the same command + # ci.yml runs, just pointed at the latest catalog. TestMain then fails the + # suite if the latest catalog holds a case ID with no matching Case() + # registration, which fails this step and the job — a red scheduled run is + # the signal GitHub notifies on. This workflow has no pull_request + # trigger, so the failure never blocks a PR. + - name: Run catalog-alignment check against latest catalog + id: align + working-directory: core + env: + CONFORMANCE_CATALOG_PATH: ${{ runner.temp }}/conformance/oauth-sdk-conformance-catalog.yaml + # `shell: bash` is load-bearing, not decoration. The default shell for a + # `run:` step on Linux is `bash -e {0}`, which does NOT set pipefail, so + # the pipeline below would exit with tee's status — always 0 — and this + # step would report success no matter what `go test` did. `shell: bash` + # is what adds `-o pipefail`. + shell: bash + run: go test ./conformancetests/ -v 2>&1 | tee "$RUNNER_TEMP/align.log" + + - name: Report drift + if: always() + run: | + if [ "${{ steps.align.outcome }}" = "success" ]; then + echo "Conformance catalog alignment: no drift against the latest catalog." >> "$GITHUB_STEP_SUMMARY" + elif ! grep -qE "has no conformance test|registers unknown case" "$RUNNER_TEMP/align.log" 2>/dev/null; then + # Match the two messages that actually mean drift, not the bare + # "CATALOG ALIGNMENT:" prefix. verifyCatalogAlignment emits that + # prefix for three distinct outcomes (catalog_alignment_test.go): + # + # catalog case %q has no conformance test -> drift + # conformance test registers unknown case %q -> drift + # load catalog: read catalog: ... -> harness problem + # + # The third fires when the catalog clone is missing or the path is + # wrong. Grepping the prefix would classify that as drift, which is + # exactly the case this branch exists to separate out. + echo "::warning::The catalog-alignment step failed without a drift message. This is a build or harness problem — a compile error, a failed module download, an unreadable catalog clone, or an unrelated conformance assertion — not catalog drift. Read the step log before touching .conformance-catalog-ref." + { + echo "## Conformance alignment check failed for another reason" + echo "" + echo "The step failed, but its output carries neither drift message" + echo "(\`has no conformance test\` / \`registers unknown case\`)." + echo "That points at a build or harness problem rather than a catalog change." + } >> "$GITHUB_STEP_SUMMARY" + else + echo "::warning::Conformance catalog drift detected — the latest catalog has cases not yet covered by the SDK. Adopt them in core/conformancetests/ and bump .conformance-catalog-ref." + { + echo "## Conformance catalog drift detected" + echo "" + echo "The latest (unpinned) conformance catalog contains cases the SDK does not yet cover, or the alignment check otherwise failed." + echo "" + echo "**Next steps:** adopt the new cases in \`core/conformancetests/\` and bump \`.conformance-catalog-ref\` to the new catalog SHA in the same change." + } >> "$GITHUB_STEP_SUMMARY" + fi diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c9721bf..482af25 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -102,11 +102,8 @@ jobs: token: ${{ steps.app_token.outputs.token }} - name: Check out shared conformance catalog - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - with: - repository: AuthPlane/conformance - path: conformance - fetch-depth: 1 + shell: bash + run: .github/scripts/fetch-conformance-catalog.sh - name: Set up Go 1.25 uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0 @@ -226,7 +223,7 @@ jobs: - name: Run tests in all four modules env: - CONFORMANCE_CATALOG_PATH: ${{ github.workspace }}/conformance/oauth-sdk-conformance-catalog.yaml + CONFORMANCE_CATALOG_PATH: ${{ runner.temp }}/conformance/oauth-sdk-conformance-catalog.yaml run: | (cd core && go test ./...) (cd mcp && go test ./...) From 35d9f60a8869d3e043efef67a53345e7a750b9e1 Mon Sep 17 00:00:00 2001 From: Roberto Iskandarani Date: Fri, 7 Aug 2026 14:09:35 -0300 Subject: [PATCH 2/2] docs: document the pinned catalog for contributors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pin is only useful if a contributor can reproduce CI. CONTRIBUTING and the conformance-tests README now say where the ref lives, how to check the catalog out at it, and that adopting new cases means bumping the ref in the same change. Also drops a clause the pin change made false: the release bot token is no longer used to check out the conformance catalog — the fetch script clones it directly, since the repo is public and needs no credential. --- CONTRIBUTING.md | 13 +++++++++++++ RELEASE_GUIDE.md | 2 +- core/conformancetests/README.md | 5 +++++ 3 files changed, 19 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 348c85a..70095c1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -76,6 +76,19 @@ go install golang.org/x/vuln/cmd/govulncheck@latest (cd mcp && govulncheck ./...) ``` +**Conformance catalog:** + +The `core` conformance suite maps to the shared [conformance catalog](https://github.com/AuthPlane/conformance). CI pins the catalog to the SHA tracked in [`.conformance-catalog-ref`](.conformance-catalog-ref) at the repo root, so a catalog change can never break CI on its own. To reproduce CI locally, check out that same ref: + +```bash +git clone https://github.com/AuthPlane/conformance.git /path/to/catalog +git -C /path/to/catalog checkout "$(cat .conformance-catalog-ref)" +export CONFORMANCE_CATALOG_PATH=/path/to/catalog/oauth-sdk-conformance-catalog.yaml +(cd core && go test ./conformancetests/ -v) +``` + +A weekly `conformance-catalog-drift` workflow runs the alignment check against the latest catalog and fails when new cases need adopting. When adopting them, update `core/conformancetests/` and bump `.conformance-catalog-ref` in the same change. See [`core/conformancetests/README.md`](core/conformancetests/README.md) for details. + ## Pull Request Guidelines - Branch off `main`. Release branches (`release/v*`, `hotfix/v*`) are managed by the release flow — see [RELEASE_POLICY.md](RELEASE_POLICY.md). diff --git a/RELEASE_GUIDE.md b/RELEASE_GUIDE.md index 58aabfa..5fed83f 100644 --- a/RELEASE_GUIDE.md +++ b/RELEASE_GUIDE.md @@ -5,7 +5,7 @@ How to ship a new version of the Go SDK (`core`, `http`, `mcp`). All three modul ## Prerequisites - You are a maintainer on `AuthPlane/go-sdk`. -- **`RELEASE_BOT_APP_ID`** and **`RELEASE_BOT_PRIVATE_KEY`** are set as organization secrets scoped to this repo. The Release Bot GitHub App mints a short-lived token used to push the four annotated tags and check out the conformance catalog. (`ci.yml` does not need these — the conformance repo is public.) `release.yml` fails fast with a clear error if either secret is missing — the workflow will not silently proceed. +- **`RELEASE_BOT_APP_ID`** and **`RELEASE_BOT_PRIVATE_KEY`** are set as organization secrets scoped to this repo. The Release Bot GitHub App mints a short-lived token used to push the four annotated tags. (`ci.yml` does not need these — the conformance repo is public.) `release.yml` fails fast with a clear error if either secret is missing — the workflow will not silently proceed. - `CHANGELOG.md` on `main` has a populated `## [Unreleased]` section. There is no registry to configure. `proxy.golang.org` polls public tags and begins serving the new module versions within seconds of the atomic push — **the tag push is the publish**. diff --git a/core/conformancetests/README.md b/core/conformancetests/README.md index 61fc42e..fbef6b9 100644 --- a/core/conformancetests/README.md +++ b/core/conformancetests/README.md @@ -85,12 +85,17 @@ Set `CONFORMANCE_CATALOG_PATH` (or `AUTHPLANE_CONFORMANCE_CATALOG`) to the absol # Clone the catalog repo anywhere git clone git@github.com:AuthPlane/conformance.git /path/to/catalog +# Check out the same pinned ref CI uses (single-sourced at the repo root) +git -C /path/to/catalog checkout "$(cat /path/to/go-sdk/.conformance-catalog-ref)" + # Point the harness at it export CONFORMANCE_CATALOG_PATH=/path/to/catalog/oauth-sdk-conformance-catalog.yaml ``` This is useful in CI or when the catalog is not a sibling directory. +CI pins the catalog to the SHA tracked in [`.conformance-catalog-ref`](../../.conformance-catalog-ref) at the repo root, so a catalog change can never break CI on its own. Check out that same ref locally (as above) to match CI exactly. A weekly `conformance-catalog-drift` workflow runs the alignment check against the latest catalog and fails when new cases need adopting; adopt them here and bump `.conformance-catalog-ref` in the same change. + ## Running ```bash