Skip to content

fix: enforce PSScriptAnalyzer in the build and unify the lint ruleset#64

Merged
tablackburn merged 1 commit into
mainfrom
ci/real-inbuild-scriptanalysis
May 29, 2026
Merged

fix: enforce PSScriptAnalyzer in the build and unify the lint ruleset#64
tablackburn merged 1 commit into
mainfrom
ci/real-inbuild-scriptanalysis

Conversation

@tablackburn
Copy link
Copy Markdown
Owner

@tablackburn tablackburn commented May 29, 2026

Summary

  • Make in-build PSScriptAnalyzer actually enforce. The Test task pulls PowerShellBuild's Analyze via -FromModule, and that task's Test-PSBuildScriptAnalysis filters results with $_Severity (an undefined variable) instead of $_.Severity — so its error/warning counts are always 0 and it never fails the build. This bug is in every released PowerShellBuild version (through 0.8.0, the latest) and on main, so no version bump fixes it.
  • Replace it with a real Analyze task. -FromModule dot-sources PowerShellBuild's psakeFile (registering its Analyze), and psake won't let us redefine an existing task, so we remove its no-op from the psake context and define our own under the same name (depends on Build, like the original). The new task counts $_.Severity correctly and throws at the configured FailBuildOnSeverityLevel (Error).
  • Point ScriptAnalysis.SettingsPath at the repo's own PSScriptAnalyzerSettings.psd1 (PowerShellBuild defaulted to its bundled ruleset).
  • Drop the obsolete Linux/macOS analysis-disable — loading a settings file no longer crashes PSScriptAnalyzer there (verified on pwsh 7.5).
  • Switch the standalone CI lint job from -Settings PSGallery to the same settings file, so every analysis path shares one ruleset.

Test Plan

  • build.psake.ps1 parses; CI YAML + embedded pwsh parse
  • Remove-then-redefine of the shared Analyze task verified end-to-end on psake 4.9.1 (pinned CI version) and 5.0.4
  • Severity/threshold logic unit-tested (throws on Error; passes Warning/Info at the Error threshold)
  • Repo ruleset against source: 0 errors (5 warnings, 4 info) — won't fail at the Error threshold
  • CI green on all platforms (this validates analysis of the built module, which can't be fully exercised locally)

Notes

Breaking Changes

None expected. Analysis now genuinely runs on all platforms at the Error threshold; current findings are all Warning/Information.

Summary by CodeRabbit

  • Chores
    • Moved linting into the build pipeline, removing the separate CI lint job.
    • Uses centralized repository analysis settings for consistency.
    • Linting can be conditionally skipped via configuration.
    • Build now enforces code-quality rules and fails based on configured severity levels.

Review Change Stack

Copilot AI review requested due to automatic review settings May 29, 2026 04:52
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 29, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: eb63d5b8-e90e-4c6f-ac22-a5f810168588

📥 Commits

Reviewing files that changed from the base of the PR and between 993a385 and 305fb9e.

📒 Files selected for processing (2)
  • .github/workflows/CI.yaml
  • build.psake.ps1
💤 Files with no reviewable changes (1)
  • .github/workflows/CI.yaml

📝 Walkthrough

Walkthrough

CI lint job removed. build.psake.ps1 now points ScriptAnalyzer settings to the repo PSScriptAnalyzerSettings.psd1, adds a Lint psake task that runs Invoke-ScriptAnalyzer and fails per severity, and the Test task depends on Lint.

Changes

PSScriptAnalyzer Enforcement Consolidation

Layer / File(s) Summary
Build ScriptAnalysis settings
build.psake.ps1
Sets PSBPreference.Test.ScriptAnalysis.SettingsPath to repository PSScriptAnalyzerSettings.psd1.
Lint psake task and Test wiring
build.psake.ps1
Adds a Lint task that runs Invoke-ScriptAnalyzer on PSBPreference.Build.ModuleOutDir, formats results, maps ParseError to Error, counts severities, and fails the build according to FailBuildOnSeverityLevel. Updates Test task dependency from AnalyzeLint.
Remove CI lint job
.github/workflows/CI.yaml
Removes the lint job from the CI workflow jobs list so CI proceeds directly to the unit-tests job.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

I’m a rabbit who inspects each script,
Hopping from CI to build — neat and swift,
Settings point home, lint runs in one place,
No extra CI job to chase,
A tidy hop, a cleaner drift. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main objectives: enforcing PSScriptAnalyzer in the build process and unifying the lint ruleset across CI and build tasks.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/real-inbuild-scriptanalysis

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes PowerShell script linting (PSScriptAnalyzer) consistently enforceable across both the psake build pipeline and the standalone CI lint job, using a single repository-owned ruleset.

Changes:

  • Replaces PowerShellBuild’s non-enforcing Analyze psake task with a repo-defined Analyze task that correctly counts severities and fails the build at the configured threshold.
  • Configures in-build analysis to use the repo’s PSScriptAnalyzerSettings.psd1 (instead of PowerShellBuild’s bundled defaults) and re-enables analysis on non-Windows platforms.
  • Updates the CI lint job to use the same settings file for consistent rules between CI and in-build analysis.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
build.psake.ps1 Sets ScriptAnalysis to use the repo ruleset and replaces the Analyze task to enforce failures by severity.
.github/workflows/CI.yaml Switches the lint job to use ./PSScriptAnalyzerSettings.psd1 for the same ruleset as the build.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Base automatically changed from ci/pin-pssa-version-in-lint-job to main May 29, 2026 05:29
@tablackburn tablackburn force-pushed the ci/real-inbuild-scriptanalysis branch 2 times, most recently from 0ae9803 to 38eee88 Compare May 29, 2026 05:59
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@build.psake.ps1`:
- Line 143: The error counting line using $errorCount =
@($analysisResult).Where({ $_.Severity -eq 'Error' }).Count misses parse/syntax
failures because those have Severity 'ParseError'; update the predicate that
computes $errorCount (referencing $analysisResult and $errorCount) to treat
'ParseError' as an error too (e.g. check Severity is 'Error' or 'ParseError', or
use membership against @('Error','ParseError')) so parse errors are included in
the count.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9a53c2b6-690c-4cad-b09f-5b9a4153a9d1

📥 Commits

Reviewing files that changed from the base of the PR and between 6713031 and 993a385.

📒 Files selected for processing (2)
  • .github/workflows/CI.yaml
  • build.psake.ps1
💤 Files with no reviewable changes (1)
  • .github/workflows/CI.yaml

Comment thread build.psake.ps1 Outdated
@tablackburn tablackburn force-pushed the ci/real-inbuild-scriptanalysis branch from 993a385 to 7d7445a Compare May 29, 2026 06:34
…t job

PowerShellBuild's 'Analyze' task never fails the build: its
Test-PSBuildScriptAnalysis filters results with `$_Severity` (an
undefined variable) instead of `$_.Severity`, so error/warning counts
are always zero. Present in every released version (through 0.8.0) and
on main, so a version bump cannot fix it. Reported upstream:
psake/PowerShellBuild#125.

Add a custom 'Lint' task that runs Invoke-ScriptAnalyzer directly and
fails at the configured FailBuildOnSeverityLevel ('Error'), counting
ParseError (syntax-broken files) as errors too. The Test task depends
on 'Lint' instead of PowerShellBuild's no-op 'Analyze' (which psake
loads via -FromModule and won't let us redefine; it is left unused).
This mirrors how other PowerShellBuild consumers define their own
analysis task rather than relying on the module's no-op.

Point ScriptAnalysis.SettingsPath at the repo's own
PSScriptAnalyzerSettings.psd1 (PowerShellBuild defaulted to its bundled
ruleset) and drop the obsolete Linux/macOS analysis-disable.

Remove the standalone 'PSScriptAnalyzer Lint' CI job: now that analysis
runs and enforces during the build on every platform, a separate job
re-running PSScriptAnalyzer on the source tree is redundant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tablackburn tablackburn force-pushed the ci/real-inbuild-scriptanalysis branch from 7d7445a to 305fb9e Compare May 29, 2026 07:29
@tablackburn tablackburn merged commit d162f70 into main May 29, 2026
14 checks passed
@tablackburn tablackburn deleted the ci/real-inbuild-scriptanalysis branch May 29, 2026 16:57
tablackburn added a commit that referenced this pull request May 29, 2026
…ver (#66)

The integration-tests job hits a live, shared Plex server. Running its
ubuntu and windows matrix legs at once (or across overlapping runs)
causes state races - e.g. collection ItemCount assertions flaking when
two jobs mutate the same server simultaneously (seen on PR #64).

Add a job-level concurrency group to integration-tests with a constant
group (not keyed by matrix.os or ref) so every integration job in the
repo serializes, and cancel-in-progress: false so an in-progress suite
is never cancelled - newer runs queue and wait. Scoped to this job only;
unit tests and lint are deterministic and keep running in parallel.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants