Skip to content

ci: survive PowerShell Gallery outages, and stop the version-map gate failing on new REST versions - #54

Merged
juemerson-at-purestorage merged 4 commits into
dmann000:mainfrom
juemerson-at-purestorage:ci/workflow-hardening
Jul 25, 2026
Merged

ci: survive PowerShell Gallery outages, and stop the version-map gate failing on new REST versions#54
juemerson-at-purestorage merged 4 commits into
dmann000:mainfrom
juemerson-at-purestorage:ci/workflow-hardening

Conversation

@juemerson-at-purestorage

Copy link
Copy Markdown
Collaborator

Problem

Two unrelated ways CI fails for reasons that have nothing to do with the code under test.

1. The PowerShell Gallery is a hard dependency. Run
30149566997
died at Install test dependencies with No match was found ... module name 'Pester' — a
transient gallery blip. The run on the identical SHA passed the same
windows-latest, pwsh job 47 seconds later. Pester never installed, so zero tests ran
on that job. The existing retry helper (3 × 15s ≈ 50s) was outlasted by the outage.

2. Versions were unpinned, and that already changed CI silently. MinimumVersion = '5.0'
resolves to latest, so CI had already moved from Pester 5 to Pester 6 with nobody
deciding to — confirmed from the logs, all four jobs were running Pester v6.0.1. A 6.x point
release could break the suite with no repo change at all.

Separately, the capability-map workflow had two self-inflicted failure modes: its installs
at the Run tests step had no retry whatsoever, and its version-map gate fails on exactly
the runs that have something new to report (below).

Changes

Cache + pin the test-only modules

New local composite action .github/actions/install-test-modules, replacing four
copy-pasted install blocks (two in cross-platform-tests.yml, two in
update-api-capability-map.yml).

  • Pinned Pester 6.0.1 / Posh-SSH 3.2.7 — exactly what CI already resolved to, so the
    pin itself is a no-behaviour-change commit. Versions are action inputs, declared in one
    place, and appear in the cache key so a bump invalidates automatically.
  • Save-Module into a workspace-relative .psmodules/ rather than -Scope CurrentUser.
    This is the enabling trick: CurrentUser resolves to a different directory on each OS and
    differs again between PowerShell 7 (Documents\PowerShell\Modules) and Windows PowerShell
    5.1 (Documents\WindowsPowerShell\Modules). A fixed workspace path gives one identical,
    cacheable location for every job.
  • One pinned version serves both editions — Pester 6.0.1's manifest declares
    PowerShellVersion = '5.1', so no per-edition split and one cache entry per OS.
  • Gallery is touched only on a cache miss, with 5 attempts and exponential backoff
    (15/30/60/120s) on that path. Caching alone cannot reach "never": GitHub evicts entries
    untouched for 7 days and caps a repo at 10 GB, so cold runs recur by design and the retry
    stays as the fallback.

Referenced by local path (./.github/actions/...), not a cross-repo @ref, so every fork
stays self-contained and inherits this by ordinary merge with no token or trust implications.

Bump actions off the Node 20 runtime

actions/cache v4 → v6 (the new action plus the two pre-existing cache/restore /
cache/save pins) and peter-evans/create-pull-request v6 → v8.

The last capability-map run annotated precisely this:

Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run
on Node.js 24: actions/cache/restore@v4, actions/cache/save@v4

Notes on version choice, since it isn't uniform across actions:

  • There is no actions/cache@v7 — v6 is the latest major, even though actions/checkout
    is on v7. v5 is the release that moved to node24; v6 adds an ESM migration. Verified v6 keeps
    the restore/save sub-actions and the cache-hit output, both of which are depended on here.
  • create-pull-request v6 and v7 are both node20; only v8 is node24, so the two-major
    jump is required rather than optional. Safe for this usage: v7's breaking changes were
    renaming the git-token input to branch-token and removing the deprecated
    PULL_REQUEST_NUMBER output env var, and this step uses neither.

Stop the version-map gate self-sabotaging

Data/PfbVersionMap.json is refreshed by a manual run of tools/Update-PfbVersionMap.ps1
against an Everpure-internal SSOT endpoint that GitHub-hosted CI cannot reach. The SSOT table
also publishes a REST↔Purity//FB pairing only some time after a REST version ships.

So the old assertion — every capability-map version must have a pairing — was guaranteed to
fail whenever a new REST version appeared, which is the workflow's trigger condition. It
passed only when there was nothing new. Worse, the job died at Run tests, before
Check for changesSummarize new versionsOpen pull request, so the failure swallowed
the "new REST version detected" signal the workflow exists to emit
and discarded correctly-built
artifacts.

This is not hypothetical — it is what happened on
30115582771: every build
step succeeded with 2.28 included, then the gate failed and nothing was reported.

The gate now distinguishes lag at the head from a hole in the middle:

  • Missing pairing newer than anything mapped → Write-Warning plus a ::warning::
    annotation so the outstanding manual step surfaces in the run summary. Passes.
  • Missing pairing at or below the newest mapped version → still a hard failure. That is a
    data defect, not upstream lag, and it is the invariant the test was written to protect.

Factored into Split-PfbVersionMapGap so the semantics are unit-testable without real Data/
files. Watch out: REST versions sort lexically wrong'2.9' > '2.27' as strings — so a
naive sort picks 2.9 as the high-water mark and misclassifies everything newer as a hole. The
comparison collapses to an integer, and the tests pin exactly that.

REST 2.28 → Purity//FB 4.8.4

Force-refreshed all 29 published specs (tools/Update-PfbApiSpecs.ps1 -Force, including the new
fb2.28.json) and re-ran the version-map generator. It discovered 2.28 unaided from
tools/specs/ and reached the SSOT successfully, but the SSOT document still has no 2.28 row
at all
— the string appears nowhere in it while 2.27 does, and the parser is healthy (32 rows,
1.8 → 2.27, newest pairing 2.27 = 4.8.3). The generator correctly left the file untouched, so
this one row is entered by hand.

Verification

All figures from real CI runs on the fork, not local simulation.

Job Before Cache miss Cache hit
windows PS 5.1 305s 158s 132s
windows pwsh 143s 163s 125s
ubuntu 38s 41s 28s
macos 36s 37s 32s

PS 5.1 is 57% faster end-to-end. Note it improved even on a cache missSave-Module -RequiredVersion skips the version-resolution and publisher-check work that made
Install-Module slow on 5.1. Its remaining 132s is almost entirely the suite itself (118.6s),
so install went from roughly 180s to ~13s.

  • Miss path (30169601354
    attempt 1): all four jobs Cache not found → saved → green.
  • Hit path (same run, attempt 2): all four Cache hitCache restored successfully
    gallery step skipped entirely → green, and the post step correctly declined to re-save.
  • Shared Windows cache confirmed in CI: both Windows jobs hit the same key, and the pwsh job
    imported cleanly from a cache written by the 5.1 job.
  • v6 bump (30169939134):
    four for four, zero Node 20 warnings, and v6 successfully restored caches written by v4 —
    the bump did not orphan existing entries.
  • Final (30171181366):
    four for four on 016bfbb.

Test counts confirm the 6 new tests run everywhere, including 5.1:

Platform Before After Skipped
ubuntu / macos / windows-pwsh 406 passed 412 (+6) 6 (unchanged)
windows PS 5.1 339 passed 345 (+6) 73 (unchanged)

The 73 skips on 5.1 are pre-existing — identical on the upstream baseline run — and are
pwsh-only test files (anything gated on PS 7.0), not a regression from this change.

Gate tests: the coverage file goes from 2 tests to 8, passing on both editions.
Mutation-verified: reverting the numeric comparison to a naive lexical sort fails 4 of the
6
new tests. The 2 survivors are explainable rather than gaps — one has no missing versions at
all, and in the other 2.28 still lands above even a wrongly-chosen 2.9 high-water mark.

Install-Module call sites remaining anywhere under .github/: zero.

Live-array testing: not applicable, deliberately

No runtime surface — this touches .github/, one test file, and one data row. No Public/,
Private/, or module manifest changes, so nothing reaches a FlashBlade. The equivalent
verification is the real CI runs above. Calling the exemption out rather than skipping it silently.

Known limitations, stated rather than buried

  • update-api-capability-map.yml has not run end-to-end. Its trigger is main +
    Public/**, so none of its three changes have executed. Its Open pull request step is also
    gated on github.repository == 'dmann000/fb-powershell', so a fork dispatch cannot exercise
    create-pull-request@v8 either — that one only gets proven on upstream after merge. Accepted
    knowingly.
  • The 2.28 row will never be re-verified automatically. Update-PfbVersionMap.ps1 only looks
    up versions not already present, so adding 2.28 by hand means it now short-circuits with
    "No versions need looking up" and will never query the SSOT for 2.28 again. If the published
    pairing differs from 4.8.4, this entry stays silently wrong. Re-check (or delete the row and
    re-run) once the SSOT document catches up.
  • The version map now has 29 entries while the committed capability map has 28 (through
    2.27). Intentional: rebuilding the capability map is a large Data/ change belonging to the
    automated refresh PR, not this one. Nothing asserts the reverse direction, so the state is
    consistent.
  • Failed to save: Unable to reserve cache with key ... is expected when both Windows jobs
    miss and race to save. The loser logs it and still succeeds. Documented in the action so nobody
    "fixes" it by adding the edition to the key and doubling the cache footprint to store identical
    files.

Not included

Stubbing New-SSHSession et al. in Tests/Get-PfbApiTokenViaSsh.Tests.ps1 would drop the
Posh-SSH dependency from CI entirely — Posh-SSH is installed solely so Pester can resolve those
cmdlets for mocking. Real improvement, but it's a test-design change and this PR is CI plumbing.

🤖 Generated with Claude Code

juemerson-at-purestorage and others added 4 commits July 25, 2026 11:21
Run 30149566997 failed at "Install test dependencies" with "No match was found
... module name 'Pester'" while the run on the identical SHA passed 47 seconds
later -- a transient PSGallery blip that cost a whole job's worth of test
coverage. PSGallery is now a fast-path-only dependency, not a hard one.

- New composite action .github/actions/install-test-modules wraps
  actions/cache@v4 around Save-Module into a workspace-relative .psmodules/,
  so the gallery is only touched on a cache miss.
- Pin Pester 6.0.1 and Posh-SSH 3.2.7 (exactly what CI resolves to today, so
  no behaviour change) as action inputs -- one place to bump. The previous
  MinimumVersion = '5.0' is how CI silently moved onto Pester 6 with nobody
  deciding to.
- 5 attempts with 15/30/60/120s backoff on the miss path, replacing the old
  3 x 15s (~50s) window that gallery blips routinely outlast.
- Save-Module into a fixed workspace path instead of -Scope CurrentUser: that
  scope resolves to a different directory per OS and per edition, whereas one
  workspace path yields one identical, cacheable location for every job.
  PSModulePath is exported via $GITHUB_ENV using [IO.Path]::PathSeparator.
- Collapses four copy-pasted install blocks into one, and gives
  update-api-capability-map.yml retry + pinning, which it had neither of.

Local dual-edition smoke test: Pester 6.0.1 saved via -RequiredVersion into a
non-standard directory imports and runs a Tests/ subset (39 assertions, 0
failures) under both pwsh 7.6.4 and Windows PowerShell 5.1 -- including the
5.1 host reading the directory saved by pwsh, which is what the shared
per-OS cache key relies on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
actions/cache@v4 runs on Node 20 and emits a deprecation warning on every
step. v5 moved to node24; v6 adds the ESM migration and is the latest major.
There is no actions/cache@v7 -- only actions/checkout goes that high -- so v6
is the correct target here.

Covers the new composite action and the two pre-existing cache/restore and
cache/save pins in update-api-capability-map.yml, which had the same warning.
restore/save sub-actions and the cache-hit output are unchanged in v6.

Also replaces a guessed log string in the action's comment with the one
actually observed on run 30169601354 ("Failed to save: Unable to reserve
cache with key ...") and records that both Windows jobs sharing one key is
deliberate and CI-verified.
…-sabotaging

Two changes to the capability-map workflow's failure modes.

peter-evans/create-pull-request v6 -> v8: v6 and v7 both run on node20 and
emit a deprecation warning; v8 is the node24 release. Safe despite skipping a
major -- v7's breaking changes were renaming the `git-token` input to
`branch-token` and removing the deprecated PULL_REQUEST_NUMBER output env var,
and this step uses neither.

The version-map coverage gate now distinguishes lag at the head from a hole in
the middle. Data/PfbVersionMap.json is refreshed by a manual run against an
internal SSOT endpoint CI cannot reach, and the SSOT table publishes a
REST<->Purity//FB pairing only some time AFTER a REST version ships -- verified
2026-07-25, when REST 2.28 was live and already in the capability map while the
SSOT document contained no 2.28 row at all. The old assertion therefore failed
on exactly the runs that had something new to report, and because the job died
before `Open pull request` it swallowed the "new REST version detected" signal
the workflow exists to emit.

Head-lag now warns (plus a ::warning:: annotation so the outstanding manual
step surfaces in the run summary) and passes. A missing pairing at or below the
newest mapped version is still a hard failure -- that is a data defect, not
upstream lag.

The classification is factored into Split-PfbVersionMapGap so the semantics are
unit-testable without real Data/ files, with 6 new tests. Note REST versions
sort lexically wrong ('2.9' > '2.27' as strings); the comparison collapses to an
integer, and reverting it to a naive sort fails 4 of the 6 new tests.
…e SSOT)

Force-refreshed all 29 published specs (tools/Update-PfbApiSpecs.ps1 -Force,
including the new fb2.28.json) and re-ran tools/Update-PfbVersionMap.ps1. The
script discovered 2.28 on its own from tools/specs/ and reached the SSOT
successfully, but the SSOT reference document still has no 2.28 row at all --
the string does not appear anywhere in it, while 2.27 does, and the parser is
healthy (32 rows, 1.8 through 2.27, newest pairing 2.27 = Purity//FB 4.8.3).
So the generator correctly left the file untouched and this pairing is entered
manually instead.

CAVEAT, deliberately recorded: because Update-PfbVersionMap.ps1 only looks up
versions NOT already present in the map, adding 2.28 by hand means the script
will never query the SSOT for 2.28 again -- it now short-circuits with "No
versions need looking up". If the SSOT eventually publishes a pairing other
than 4.8.4, this entry will stay silently wrong. Re-verify (or delete the entry
and re-run) once the SSOT document catches up.

tools/specs/ itself stays uncommitted per .gitignore.
@juemerson-at-purestorage
juemerson-at-purestorage merged commit bdf9d67 into dmann000:main Jul 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant