Skip to content

ci: queue integration tests so they don't race on the shared Plex server#66

Merged
tablackburn merged 2 commits into
mainfrom
ci/queue-integration-tests
May 29, 2026
Merged

ci: queue integration tests so they don't race on the shared Plex server#66
tablackburn merged 2 commits into
mainfrom
ci/queue-integration-tests

Conversation

@tablackburn
Copy link
Copy Markdown
Owner

@tablackburn tablackburn commented May 29, 2026

Summary

  • Add a job-level concurrency group to the integration-tests job so it never runs concurrently against the shared, live Plex server.
  • group: ${{ github.workflow }}-integration (constant — not keyed by matrix.os or ref) → serializes the ubuntu + windows matrix legs of a run and integration across overlapping runs/PRs.
  • cancel-in-progress: false → queue, never cancel an in-progress suite.
  • Scoped to integration only; unit tests + lint stay parallel (they're deterministic — integration is the only flaky job, because it's the only one hitting an external service).

Why

PR #64's windows integration leg flaked on Collection ItemCount assertions while the ubuntu leg passed on the same commit — two jobs mutating the same Plex server at once. Queuing eliminates that race.

Notes

  • This is PlexAutomationToolkit-specific: it's the only repo with an integration-tests job, so this does not belong in the template.
  • GitHub allows one pending run per concurrency group; if integration runs from several commits pile up, an older pending one can be superseded — acceptable here (newest wins; a re-run recovers it).
  • Touches CI.yaml, as does open PR fix: enforce PSScriptAnalyzer in the build and unify the lint ruleset #64 (different section) — whichever merges second takes a trivial rebase.

Test Plan

  • CI green; integration jobs observed running one-at-a-time rather than in parallel

Summary by CodeRabbit

  • Chores
    • Updated integration testing workflow configuration to prevent concurrent test runs and avoid interrupting in-progress executions.

Review Change Stack

The integration-tests job hits a live, shared Plex server. Running its
ubuntu and windows matrix legs at once (or across overlapping runs)
causes state races - e.g. collection ItemCount assertions flaking when
two jobs mutate the same server simultaneously (seen on PR #64).

Add a job-level concurrency group to integration-tests with a constant
group (not keyed by matrix.os or ref) so every integration job in the
repo serializes, and cancel-in-progress: false so an in-progress suite
is never cancelled - newer runs queue and wait. Scoped to this job only;
unit tests and lint are deterministic and keep running in parallel.
Copilot AI review requested due to automatic review settings May 29, 2026 07:50
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 29, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 22d28bad-c47b-406b-8de6-9bb7b0e9a240

📥 Commits

Reviewing files that changed from the base of the PR and between bb93bca and f03f3bf.

📒 Files selected for processing (1)
  • .github/workflows/CI.yaml

📝 Walkthrough

Walkthrough

This pull request adds a concurrency policy to the GitHub Actions integration-tests job to prevent concurrent execution. The shared live Plex server cannot handle simultaneous test runs, so the workflow now serializes integration test jobs across all matrix variations.

Changes

Integration Test Concurrency Policy

Layer / File(s) Summary
Integration test job concurrency policy
.github/workflows/CI.yaml
The integration-tests job defines a fixed concurrency group and sets cancel-in-progress: false to serialize runs and prevent concurrent access to the shared Plex server.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Poem

🐰 One test at a time, that's the way,
No race conditions here, hooray!
Plex server shared, but not in strife—
Concurrency locks bring order to life! 🔒

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and specifically describes the main change: adding concurrency queuing to integration tests to prevent race conditions on a shared Plex server.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/queue-integration-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a job-level concurrency group to the integration-tests job in .github/workflows/CI.yaml so that integration runs against the shared live Plex server are serialized rather than racing in parallel across matrix legs and overlapping workflow runs.

Changes:

  • Add concurrency block to integration-tests with a constant group (${{ github.workflow }}-integration) not keyed on matrix.os or ref.
  • Set cancel-in-progress: false so an in-flight integration suite is never interrupted; newer runs queue.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@tablackburn
Copy link
Copy Markdown
Owner Author

@CodeRabbit review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 29, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@tablackburn tablackburn merged commit 34ce73b into main May 29, 2026
18 of 19 checks passed
@tablackburn tablackburn deleted the ci/queue-integration-tests branch May 29, 2026 23:51
tablackburn added a commit that referenced this pull request May 30, 2026
…l reference (#67)

PSScriptAnalyzer 1.25.0 intermittently throws "Object reference not set to
an instance of an object." from inside the analyzer engine, failing the
Lint task on otherwise-green builds. It has struck both Linux (#65) and
Windows (#66) CI runners at random, so it is not platform-specific and a
non-Windows disable would not cover it.

Retry the analysis up to three times on that specific null reference, while
letting any genuine analysis error surface immediately.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants