ci: pin the conformance catalog by SHA instead of cloning its default branch - #25
ci: pin the conformance catalog by SHA instead of cloning its default branch#25RobertoIskandarani wants to merge 2 commits into
Conversation
… branch
CI cloned github.com/AuthPlane/conformance at its default branch, so any change
to the catalog reached this repo immediately — a case added there could turn an
unrelated PR red here, with nothing in this repo having changed. Pinning
decouples them: a catalog change arrives only when this repo deliberately bumps
the pin together with the coverage for it.
The revision is single-sourced in a tracked .conformance-catalog-ref, guarded by
a ^[0-9a-f]{40}$ shape check so a branch or tag name cannot silently un-pin CI,
and read by both ci.yml and release.yml through one script rather than a
copy-pasted block — the pin was single-sourced before, the logic reading it was
not.
A weekly conformance-catalog-drift workflow clones the unpinned tip and runs the
alignment assertion, so new cases surface as an early warning instead of a
surprise. Its report step distinguishes real drift from a harness failure by
matching the two messages that mean drift rather than the marker prefix, which is
also emitted when the catalog cannot be read at all. The alignment step declares
shell: bash so the pipeline gets pipefail — without it the step takes tee's exit
status and reports success unconditionally.
The catalog now lands in $RUNNER_TEMP rather than the workspace, so it stays out
of `go list ./...`, coverage globs, and `git add -A` in the release commit.
No SDK source changes. Verified locally: the fetch script runs end-to-end against
the pinned revision, and the conformance suite passes against the catalog it
checks out — the repo already aligns with this revision, so the pin adopts no new
cases.
The pin is only useful if a contributor can reproduce CI. CONTRIBUTING and the conformance-tests README now say where the ref lives, how to check the catalog out at it, and that adopting new cases means bumping the ref in the same change. Also drops a clause the pin change made false: the release bot token is no longer used to check out the conformance catalog — the fetch script clones it directly, since the repo is public and needs no credential.
|
Added A pin is only useful if a contributor can reproduce CI. It also drops one clause the pin change made false. Post-create verification re-run: 2 commits, |
What
CI clones
github.com/AuthPlane/conformanceat its default branch, so any changeto the catalog reaches this repo immediately. A case added there can turn an
unrelated PR red here with nothing in this repo having changed.
This pins the revision instead. A catalog change arrives only when this repo
deliberately bumps the pin together with the coverage for it.
.conformance-catalog-ref— a tracked file holding a 40-hex commit SHA,guarded by a
^[0-9a-f]{40}$shape check before the fetch so a branch or tagname cannot silently un-pin CI.
.github/scripts/fetch-conformance-catalog.sh— the read/guard/fetchsequence, called by both
ci.ymlandrelease.yml. The ref was already goingto be single-sourced; the ~10 lines reading it would have been copy-pasted into
two workflows and could tighten in one and not the other.
.github/workflows/conformance-catalog-drift.yml— a weekly job that clonesthe unpinned tip and runs the alignment assertion, so new cases surface as an
early warning rather than a surprise at bump time.
The catalog now lands in
$RUNNER_TEMPrather than the workspace, so it stays outof
go list ./..., coverage globs, andgit add -Ain the release commit.Two details worth calling out
shell: bashon the alignment step is load-bearing. The default shell for arun:step on Linux isbash -e {0}, which does not setpipefail. The steppipes
go testthroughtee, so without it the pipeline takestee's exitstatus — always 0 — and the drift job would report success no matter what the test
did.
The drift report matches drift messages, not the marker prefix.
verifyCatalogAlignmentemitsCATALOG ALIGNMENT:for three outcomes: two thatmean drift (
has no conformance test,registers unknown case) and one that doesnot (
load catalog: read catalog: …, when the clone is missing or the path iswrong). Grepping the prefix would classify a harness failure as catalog drift,
which is exactly the case the branch exists to separate out.
Scope
No SDK source is touched — five files, all CI. The issuer-identity and PRM
derivation work is deliberately not in here; it lands separately so the mechanical
change and the semantic one can be reviewed on their own terms.
Verification
The fetch script was run end-to-end locally: it fetches and checks out
b4c758a. The conformance suite passes against the catalog it checks out — thisrepo already aligns with that revision, so the pin adopts no new cases and
requires no test changes.
shellcheckclean;go build,vet,testandgofmtgreen across all four modules.