Skip to content

Commit b5fd08b

Browse files
MariusStorhaugMariusStorhaug
authored
Merge branch 'main' into dependabot/github_actions/PSModule/Process-PSModule/dot-github/workflows/workflow.yml-5.5.0
2 parents 3b3f44c + 89c975a commit b5fd08b

20 files changed

Lines changed: 654 additions & 290 deletions

‎.github/instructions/tests.instructions.md‎

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
---
2+
description: "Use when writing, editing, or reviewing Pester test files under the tests/ folder. Covers per-test-file repository setup, self-contained test lifecycle, auth case iteration, naming conventions, and skip patterns for the GitHub module integration tests."
3+
applyTo: "tests/**"
4+
---
5+
# Integration Test Conventions
6+
7+
## Test infrastructure accounts
8+
9+
### User
10+
11+
Login: `psmodule-user`
12+
Owner of:
13+
14+
- [psmodule-user](https://github.com/psmodule-user) (standalone org)
15+
- [psmodule-test-org2](https://github.com/orgs/psmodule-test-org2) (standalone org)
16+
17+
Secrets:
18+
19+
- `TEST_USER_PAT` → `psmodule-user` (user)
20+
- `TEST_USER_USER_FG_PAT` → `psmodule-user` (user)
21+
- `TEST_USER_ORG_FG_PAT` → `psmodule-test-org2` (org)
22+
23+
### APP_ENT — PSModule Enterprise App
24+
25+
Homed in `MSX`. ClientID: `Iv23lieHcDQDwVV3alK1`.
26+
Installed on [psmodule-test-org3](https://github.com/orgs/psmodule-test-org3) (enterprise org) with all permissions and push events.
27+
28+
Secrets: `TEST_APP_ENT_CLIENT_ID`, `TEST_APP_ENT_PRIVATE_KEY`
29+
30+
### APP_ORG — PSModule Organization App
31+
32+
Homed in `PSModule`. ClientID: `Iv23liYDnEbKlS9IVzHf`.
33+
Installed on [psmodule-test-org](https://github.com/orgs/psmodule-test-org) (standalone org) with all permissions and push events.
34+
35+
Secrets: `TEST_APP_ORG_CLIENT_ID`, `TEST_APP_ORG_PRIVATE_KEY`
36+
37+
## Auth cases
38+
39+
`AuthCases.ps1` defines 7 auth cases. Each test file iterates over all cases, skipping those
40+
that don't apply (e.g., `repository` and `enterprise` owner types skip repo-dependent tests).
41+
42+
| # | AuthType | TokenType | Owner | OwnerType |
43+
|---|----------|---------------|--------------------|--------------|
44+
| 1 | PAT | USER_FG_PAT | psmodule-user | user |
45+
| 2 | PAT | ORG_FG_PAT | psmodule-test-org2 | organization |
46+
| 3 | PAT | PAT | psmodule-user | user |
47+
| 4 | IAT | GITHUB_TOKEN | PSModule | repository |
48+
| 5 | App | APP_ORG | psmodule-test-org | organization |
49+
| 6 | App | APP_ENT | psmodule-test-org3 | organization |
50+
| 7 | App | APP_ENT | msx | enterprise |
51+
52+
Cases 4 (`repository`) and 7 (`enterprise`) skip repo creation. Cases 1 and 3 share the same user owner
53+
(`psmodule-user`) but have different `$TokenType` values, so repo names are unique.
54+
55+
## Setup and teardown
56+
57+
Test infrastructure is provisioned once per workflow run using `BeforeAll.ps1` and torn down using `AfterAll.ps1`.
58+
For generic guidance on setup/teardown scripts, see the
59+
[Process-PSModule documentation](https://github.com/PSModule/Process-PSModule#setup-and-teardown-scripts).
60+
61+
Each test file gets its own repository, scoped by test name: `{TestName}-{OS}-{TokenType}-{RunID}`. This
62+
eliminates cross-file resource collisions when test files run in parallel across OSes and in sequence
63+
across auth contexts.
64+
65+
### `BeforeAll.ps1` — global setup
66+
67+
Runs once before all parallel test files. For each auth case (except `GITHUB_TOKEN`):
68+
69+
1. Connects using the auth case credentials
70+
2. Removes any existing repositories for the deterministic names used by the run
71+
3. Provisions a per-test-file repository per OS using `Set-GitHubRepository`: `{TestName}-{OS}-{TokenType}-{GITHUB_RUN_ID}`
72+
- Includes `-AddReadme`, `-License 'mit'`, and `-Gitignore 'VisualStudio'` so tests have a default branch with content
73+
- For `user` owners: `Set-GitHubRepository -Name $repoName ...`
74+
- For `organization` owners: `Set-GitHubRepository -Organization $Owner -Name $repoName ...`
75+
4. For `organization` owners only, provisions extra repositories (`-2`, `-3` suffix) for
76+
test files that need companion repos (e.g., Secrets/Variables `SelectedRepository` tests)
77+
78+
`Set-GitHubRepository` is idempotent — if the repository already exists it updates it in place (issuing a
79+
PATCH), and if it does not exist it creates it. Because the same parameters are passed each time, the
80+
end-state is identical regardless of how many times the setup runs. The extra PATCH on the happy path is
81+
a deliberate trade-off for simplicity: one call handles both first-run and partial-rerun scenarios without
82+
branching logic.
83+
84+
### `AfterAll.ps1` — global teardown
85+
86+
Runs once after all parallel test files complete. For each auth case (except `GITHUB_TOKEN`):
87+
88+
1. Connects using the auth case credentials
89+
2. Removes the per-test-file repositories by their deterministic names
90+
91+
## Per-test-file repositories
92+
93+
Each test file that depends on a GitHub repository uses its own repository, scoped by test name:
94+
`{TestName}-{OS}-{TokenType}-{RunID}`. This prevents cross-file resource collisions — test files
95+
run in parallel across OSes and in sequence across auth contexts, so one test file must never
96+
create resources on another test file's repository.
97+
98+
Each test file must ensure its repository exists using `Set-GitHubRepository` in its per-context
99+
`BeforeAll`. `Set-GitHubRepository` is idempotent — if the repository already exists it updates it
100+
in place (PATCH), and if it does not exist it creates it. When the same parameters are passed each
101+
time the end-state is identical. This makes every test file self-sufficient regardless of whether
102+
the global `BeforeAll.ps1` already provisioned the repository.
103+
104+
**Do not** use `Get-GitHubRepository` with a throw guard — that breaks partial reruns.
105+
**Do not** use `New-GitHubRepository` — that fails if the repository already exists.
106+
107+
Primary repositories use `-AddReadme`, `-License 'mit'`, and `-Gitignore 'VisualStudio'` so that
108+
a default branch with content is available for tests that need commits (e.g., releases, tags).
109+
110+
Some auth cases (e.g., `repository`, `enterprise`) do not operate on a user- or org-owned repository.
111+
Skip provisioning for those owner types and set `$repo = $null` so that repo-dependent tests can
112+
be skipped cleanly:
113+
114+
```powershell
115+
$repoPrefix = "$testName-$os-$TokenType"
116+
$repoName = "$repoPrefix-$id"
117+
if ($OwnerType -in ('repository', 'enterprise')) {
118+
$repo = $null
119+
} else {
120+
$repoParams = @{
121+
Name = $repoName
122+
AddReadme = $true
123+
License = 'mit'
124+
Gitignore = 'VisualStudio'
125+
}
126+
$repo = switch ($OwnerType) {
127+
'user' { Set-GitHubRepository @repoParams }
128+
'organization' { Set-GitHubRepository @repoParams -Organization $Owner }
129+
}
130+
}
131+
```
132+
133+
For organization-scoped tests that need companion repositories (Secrets/Variables `SelectedRepository`),
134+
provision `-2` and `-3` variants the same way:
135+
136+
```powershell
137+
$repo2 = Set-GitHubRepository -Organization $Owner -Name "$repoName-2"
138+
$repo3 = Set-GitHubRepository -Organization $Owner -Name "$repoName-3"
139+
```
140+
141+
## Test file structure
142+
143+
```powershell
144+
BeforeAll {
145+
$testName = 'TestName'
146+
$os = $env:RUNNER_OS
147+
$id = $env:GITHUB_RUN_ID
148+
}
149+
150+
Describe 'TestName' {
151+
$authCases = . "$PSScriptRoot/Data/AuthCases.ps1"
152+
153+
Context 'As <Type> using <Case> on <Target>' -ForEach $authCases {
154+
BeforeAll {
155+
$context = Connect-GitHubAccount @connectParams -PassThru -Silent
156+
if ($AuthType -eq 'APP') {
157+
$context = Connect-GitHubApp @connectAppParams -PassThru -Default -Silent
158+
}
159+
160+
$repoPrefix = "$testName-$os-$TokenType"
161+
$repoName = "$repoPrefix-$id"
162+
if ($OwnerType -in ('repository', 'enterprise')) {
163+
$repo = $null
164+
} else {
165+
$repoParams = @{
166+
Name = $repoName
167+
AddReadme = $true
168+
License = 'mit'
169+
Gitignore = 'VisualStudio'
170+
}
171+
$repo = switch ($OwnerType) {
172+
'user' { Set-GitHubRepository @repoParams }
173+
'organization' { Set-GitHubRepository @repoParams -Organization $Owner }
174+
}
175+
}
176+
177+
# Clean up stale resources from prior runs (re-runs with same GITHUB_RUN_ID)
178+
# Example: remove leftover releases, environments, etc.
179+
}
180+
181+
AfterAll {
182+
# Remove all test-specific resources created during this context
183+
# (environments, releases, secrets, etc.)
184+
Get-GitHubContext -ListAvailable | Disconnect-GitHubAccount -Silent
185+
}
186+
187+
It 'Should do something' -Skip:($OwnerType -in ('repository', 'enterprise')) {
188+
# Test logic using $repo, $Owner, $repoName
189+
}
190+
}
191+
}
192+
```
193+
194+
## Naming conventions
195+
196+
| Resource | Pattern | Example |
197+
|------------|----------------------------------------------|---------------------------------------|
198+
| Repo | `{TestName}-{OS}-{TokenType}-{RunID}` | `Releases-Linux-USER_FG_PAT-1234` |
199+
| Extra repo | `{TestName}-{OS}-{TokenType}-{RunID}-{N}` | `Secrets-Linux-ORG_FG_PAT-1234-2` |
200+
| Secret | `{TestName}_{OS}_{TokenType}_{RunID}` | `Secrets_Linux_PAT_1234` |
201+
| Variable | `{TestName}_{OS}_{TokenType}_{RunID}` | `Variables_Linux_PAT_1234` |
202+
| Team | `{TestName}_{OS}_{TokenType}_{RunID}_{Name}` | `Teams_Linux_APP_ORG_1234_Pull` |
203+
| Env | `{TestName}-{OS}-{TokenType}-{RunID}` | `Secrets-Linux-PAT-1234` |
204+
205+
## Key rules
206+
207+
- `$id` must always be `$env:GITHUB_RUN_ID` — never `[guid]::NewGuid()` or `Get-Random`.
208+
- Skip repo-dependent tests with `-Skip:($OwnerType -in ('repository', 'enterprise'))`.
209+
- Disconnect all sessions in `AfterAll`: `Get-GitHubContext -ListAvailable | Disconnect-GitHubAccount -Silent`.
210+
- Each test file uses its own repository: `{TestName}-{OS}-{TokenType}-{RunID}`. No two test files share a repository.
211+
- Each test file is self-contained and responsible for its own setup and teardown:
212+
- **BeforeAll (per-context):** Ensure the repository exists via `Set-GitHubRepository`. Clean up stale test-specific resources from prior runs (re-runs with the same `GITHUB_RUN_ID`).
213+
- **AfterAll (per-context):** Remove all test-specific resources created during the run (environments, releases, secrets, variables, etc.).
214+
- Any individual test file or auth context can be re-run independently. Tests must not assume clean initial state — they must be idempotent.
215+
- Tests run in parallel across OSes (Linux, macOS, Windows) and in sequence across auth contexts (7 cases). Resource names must include enough dimensions to prevent collisions across all parallel and sequential axes.
216+
- `Repositories.Tests.ps1` is independent — it creates and deletes its own repos because it tests CRUD.

‎tests/Actions.Tests.ps1‎

Lines changed: 22 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,12 @@
1616
param()
1717

1818
BeforeAll {
19-
$testName = 'ActionsTests'
19+
$testName = 'Actions'
2020
$os = $env:RUNNER_OS
21-
$guid = [guid]::NewGuid().ToString()
21+
$id = $env:GITHUB_RUN_ID
22+
if (-not $id) {
23+
throw 'GITHUB_RUN_ID is required for Actions tests because it is used to build repository-scoped names for OIDC operations.'
24+
}
2225
}
2326

2427
Describe 'Actions' {
@@ -51,37 +54,29 @@ Describe 'Actions' {
5154
Write-Host ($context | Format-List | Out-String)
5255
}
5356
}
54-
$repoPrefix = "$testName-$os-$TokenType"
55-
$repoName = "$repoPrefix-$guid"
57+
$repoPrefix = "Test-$os-$TokenType"
58+
$repoName = "$repoPrefix-$id"
5659

57-
switch ($OwnerType) {
58-
'user' {
59-
Get-GitHubRepository | Where-Object { $_.Name -like "$repoPrefix*" } |
60-
Remove-GitHubRepository -Confirm:$false
61-
$repo = New-GitHubRepository -Name $repoName -Confirm:$false
62-
}
63-
'organization' {
64-
Get-GitHubRepository -Organization $Owner | Where-Object { $_.Name -like "$repoPrefix*" } |
65-
Remove-GitHubRepository -Confirm:$false
66-
$repo = New-GitHubRepository -Organization $Owner -Name $repoName -Confirm:$false
60+
LogGroup "Using Repository - [$repoName]" {
61+
if ($OwnerType -in ('repository', 'enterprise')) {
62+
$repo = $null
63+
} else {
64+
$repoParams = @{
65+
Name = $repoName
66+
AddReadme = $true
67+
License = 'mit'
68+
Gitignore = 'VisualStudio'
69+
}
70+
$repo = switch ($OwnerType) {
71+
'user' { Set-GitHubRepository @repoParams }
72+
'organization' { Set-GitHubRepository @repoParams -Organization $Owner }
73+
}
74+
Write-Host ($repo | Select-Object * | Out-String)
6775
}
6876
}
69-
LogGroup "Repository - [$repoName]" {
70-
Write-Host ($repo | Select-Object * | Out-String)
71-
}
7277
}
7378

7479
AfterAll {
75-
switch ($OwnerType) {
76-
'user' {
77-
Get-GitHubRepository | Where-Object { $_.Name -like "$repoPrefix*" } |
78-
Remove-GitHubRepository -Confirm:$false
79-
}
80-
'organization' {
81-
Get-GitHubRepository -Organization $Owner | Where-Object { $_.Name -like "$repoPrefix*" } |
82-
Remove-GitHubRepository -Confirm:$false
83-
}
84-
}
8580
Get-GitHubContext -ListAvailable | Disconnect-GitHubAccount -Silent
8681
Write-Host ('-' * 60)
8782
}

‎tests/AfterAll.ps1‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
[CmdletBinding()]
2+
param()
3+
4+
LogGroup 'AfterAll - Global Test Teardown' {
5+
$authCases = . "$PSScriptRoot/Data/AuthCases.ps1"
6+
7+
$id = $env:GITHUB_RUN_ID
8+
if (-not $id) {
9+
throw 'GITHUB_RUN_ID environment variable is not set. Refusing to clean up test repositories with an unscoped wildcard (would impact concurrent runs).'
10+
}
11+
if (-not $env:Settings) {
12+
throw 'Settings environment variable is not set. Process-PSModule must populate it with the test suite configuration.'
13+
}
14+
$prefix = 'Test'
15+
16+
# Derive the list of OS names from the Settings JSON provided by Process-PSModule.
17+
try {
18+
$settings = $env:Settings | ConvertFrom-Json
19+
} catch {
20+
throw "Settings environment variable contains invalid JSON. Expected TestSuites.Module.OSName to be present. $_"
21+
}
22+
23+
$osNames = @($settings.TestSuites.Module.OSName | Sort-Object -Unique)
24+
if (-not $osNames) {
25+
throw 'Settings JSON must include at least one non-empty TestSuites.Module.OSName value.'
26+
}
27+
$invalidOsNames = @($osNames | Where-Object { -not $_ -or -not $_.ToString().Trim() })
28+
if ($invalidOsNames.Count -gt 0) {
29+
throw 'Settings JSON contains one or more null or empty TestSuites.Module.OSName values.'
30+
}
31+
Write-Host "Cleaning up test repositories for OSes: $($osNames -join ', ')"
32+
33+
foreach ($authCase in $authCases) {
34+
$authCase.GetEnumerator() | ForEach-Object { Set-Variable -Name $_.Key -Value $_.Value }
35+
36+
if ($TokenType -eq 'GITHUB_TOKEN') {
37+
Write-Host "Skipping teardown for $AuthType-$TokenType (uses existing repository)"
38+
continue
39+
}
40+
41+
LogGroup "Teardown - $AuthType-$TokenType" {
42+
$context = Connect-GitHubAccount @connectParams -PassThru -Silent
43+
if ($AuthType -eq 'APP') {
44+
$context = Connect-GitHubApp @connectAppParams -PassThru -Default -Silent
45+
}
46+
Write-Host ($context | Format-List | Out-String)
47+
48+
foreach ($os in $osNames) {
49+
$repoPrefix = "$prefix-$os-$TokenType"
50+
$repoName = "$repoPrefix-$id"
51+
52+
LogGroup "Repository cleanup - $AuthType-$TokenType - $os" {
53+
# Use deterministic name lookups instead of listing all repos to reduce API calls.
54+
$cleanupRepoNames = @($repoName)
55+
if ($OwnerType -eq 'organization') {
56+
$cleanupRepoNames += "$repoName-2", "$repoName-3"
57+
}
58+
59+
foreach ($cleanupRepoName in $cleanupRepoNames) {
60+
switch ($OwnerType) {
61+
'user' {
62+
Get-GitHubRepository -Name $cleanupRepoName -ErrorAction SilentlyContinue |
63+
Remove-GitHubRepository -Confirm:$false
64+
}
65+
'organization' {
66+
Get-GitHubRepository -Owner $Owner -Name $cleanupRepoName -ErrorAction SilentlyContinue |
67+
Remove-GitHubRepository -Confirm:$false
68+
}
69+
}
70+
}
71+
}
72+
}
73+
74+
Get-GitHubContext -ListAvailable | Disconnect-GitHubAccount -Silent
75+
}
76+
}
77+
}

‎tests/Apps.Tests.ps1‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,12 @@
1919
[CmdletBinding()]
2020
param()
2121

22+
BeforeAll {
23+
$testName = 'Apps'
24+
$os = $env:RUNNER_OS
25+
$id = $env:GITHUB_RUN_ID
26+
}
27+
2228
Describe 'Apps' {
2329
$authCases = . "$PSScriptRoot/Data/AuthCases.ps1"
2430

‎tests/Artifacts.Tests.ps1‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,12 @@
1919
[CmdletBinding()]
2020
param()
2121

22+
BeforeAll {
23+
$testName = 'Artifacts'
24+
$os = $env:RUNNER_OS
25+
$id = $env:GITHUB_RUN_ID
26+
}
27+
2228
Describe 'Artifacts' {
2329
$authCases = . "$PSScriptRoot/Data/AuthCases.ps1"
2430

0 commit comments

Comments
 (0)