Skip to content

Consistent phpunit runners for local and CI#493

Open
kasparsd wants to merge 65 commits into
fairpm:release_1.4.1from
kasparsd:add-phpstan-fresh
Open

Consistent phpunit runners for local and CI#493
kasparsd wants to merge 65 commits into
fairpm:release_1.4.1from
kasparsd:add-phpstan-fresh

Conversation

@kasparsd
Copy link
Copy Markdown
Contributor

@kasparsd kasparsd commented May 15, 2026

Fixes #479, fixes #383.

A follow-up to #382.

Development workflow consistency:

  • Standardizes local development and CI around wp-env so both paths use the same environment bootstrapping and runtime logic.
  • Adds a small npm command layer for common tasks like CLI access, linting, formatting, PHPUnit, and coverage, so developers and CI invoke the same commands.
  • Updates the coding standards workflow to run through the wp-env-based toolchain instead of relying on a separate runner-level PHP setup.

Local setup and onboarding:

  • Commits a Node version via .nvmrc and upgrades @wordpress/env, reducing local setup drift and keeping machines closer to CI.
  • Expands the contributing docs to make wp-env the default local workflow, covering start/log/stop commands, PHPUnit usage, coverage, and version overrides.
  • Supports both .wp-env.override.json and WP_ENV_PHP_VERSION / WP_ENV_CORE env vars, making local environment customization straightforward without changing repo defaults.

Local PHPUnit execution:

  • Moves PHPUnit execution into the wp-env tests container, removing the old split between custom local setup and CI-only test installation.
  • Adds a dedicated tests:cli path and install flow for test dependencies, so local test runs are easier to reason about and reuse.
  • Aligns the single-site and multisite PHPUnit config files with this workflow, including cleaner cache and coverage paths.

WordPress and PHP version matrix support:

  • Makes it possible to run PHPUnit locally against different WordPress core and PHP combinations by driving wp-env from override files or environment variables.
  • Broadens Composer dev tooling to support multiple PHPUnit versions and include the WP-CLI bundle needed for this workflow.
  • Replaces the hardcoded GitHub Actions exclusion list with a generated PHP/WordPress compatibility matrix, centralizing supported combinations in one script.

CI parity with local testing:

  • Uses the generated matrix in GitHub Actions so each job boots the requested WordPress and PHP combination the same way it can be done locally.
  • Refreshes Composer dependencies per matrix job so the right PHPUnit version is installed for that exact WordPress/PHP pairing.
  • Extends the same wp-env-based approach to coverage runs, improving parity between local verification and CI.

CI timing

Previously the phpunit setup in CI didn't use wp-env leading to two different ways of running the same tests. Switching to wp-env ensures that we can run the tests with any combination of PHP and WP core both locally and during CI.

Running the phpunit test matrix in CI before the change took 1min 42s:

matrix-before

while now it takes 3min 16s (or 2min 30s in recent runs):

matrix-after

which is a minor increase but offers consistency and predictability.

@github-actions
Copy link
Copy Markdown
Contributor

@kasparsd kasparsd force-pushed the add-phpstan-fresh branch from 783b8d0 to 15e03a5 Compare May 15, 2026 13:43
@cdils cdils changed the base branch from main to release_1.5.0 May 15, 2026 13:52
@cdils cdils added this to the 1.5.0 milestone May 15, 2026
kasparsd added 26 commits May 15, 2026 16:55
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
During CI we require in versions as needed

Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
kasparsd added 16 commits May 15, 2026 16:55
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
@kasparsd kasparsd force-pushed the add-phpstan-fresh branch from 051ece5 to 7ec0085 Compare May 15, 2026 13:55
uses: actions/setup-node@v6
with:
cache: 'npm'
node-version-file: '.nvmrc'
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Introduce .nvmrc to lock the version of Node to a known version for this project.

- uses: actions/checkout@v6
- name: Generate PHP and WP testing matrix
id: generate
run: echo "matrix=$(node ./bin/github-actions-matrix.js)" >> "$GITHUB_OUTPUT"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a simple node script to generate the test matrix since it is easier to read and maintain.

- name: Install Composer dependencies
run: npm run test:php:install-deps

- name: Generate coverage report
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add in coverage reporting.

Comment thread inc/icons/svg.php
return '';
}

function sanitize_hex_color( $color ): string {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix a few remaining phpstan errors.

Comment thread composer.json
"require-dev": {
"yoast/phpunit-polyfills": "*",
"nimut/phpunit-merger": "*",
"yoast/phpunit-polyfills": "^4.0",
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lock everything to the versions supported by the min-supported PHP of the project (8.0). During CI we delete the lock file and update the requirements to work with the relevant versions of PHP.

Comment thread composer.json
],
"test:multisite": [
"Composer\\Config::disableProcessTimeout",
"@php ./vendor/phpunit/phpunit/phpunit -c tests/phpunit/multisite.xml"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Composer is smart enough to try the local vendor/bin binaries before using globals so we make this consistent with how phpcs and phpstan is called in this file.

Comment thread phpcs.xml.dist
<!-- Exclude third-party dependencies -->
<exclude-pattern>inc/updater/class-lite\.php</exclude-pattern>
<exclude-pattern>inc/updater/class-lite.php</exclude-pattern>
<exclude-pattern>inc/updater/class-base58btc.php</exclude-pattern>
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the missing excludes and skip escaping . since it isn't necessary.

Comment thread phpcs.xml.dist

<!-- Exclude the unit tests from the file name rules -->
<rule ref="WordPress.Files.FileName">
<exclude-pattern>tests/phpunit/*</exclude-pattern>
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trailing /* is also not necessary -- it matches by fragment by default.

Comment thread phpstan.dist.neon
parameters:
tmpDir: .cache/phpstan
level: 0
scanDirectories:
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure phpstan is aware of the wp-cli signatures.

</exclude>
</groups>
<coverage includeUncoveredFiles="true" processUncoveredFiles="false" pathCoverage="false" cacheDirectory="./cache">
<coverage includeUncoveredFiles="true" processUncoveredFiles="false" pathCoverage="false" cacheDirectory="tests/phpunit/cache">
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this consistent with what phpunit.xml.dist was using to cache the results.

kasparsd added 4 commits May 15, 2026 17:05
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
@kasparsd kasparsd changed the base branch from release_1.5.0 to release_1.4.1 May 22, 2026 13:45
@chuckadams
Copy link
Copy Markdown
Contributor

I personally have less than zero interest in using wp-env as the standard environment: a dependency on node in order to run bash scripts in order to run PHP. If this is the easiest way to switch WP versions quickly in CI, especially if it can cache the different versions, then I'm not going to stand in the way, but I would very much prefer to avoid gratuitous Node dependencies as far as the officially sanctioned dev workflow goes, and would rather see something like DDEV be it instead.

@kasparsd
Copy link
Copy Markdown
Contributor Author

if this is the easiest way to switch WP versions quickly in CI, especially if it can cache the different versions

Yeah, this setup (and the updated documentation) allows us to consistently test the plugin against any version of PHP and WP core. Previously the setup used locally (wp-env) was different from what the CI would use, so there was no clear way to replicate failing builds.

I absolutely don't love wp-env but it gets the job done and is a shared convention that others know how to debug and use. The previous mix of bash scripts, system dependencies installed only in CI to run the same tests and linters was arguably more convoluted and harder to maintain.

@Ipstenu Ipstenu modified the milestones: 1.5.0, 1.4.1 May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🚀 Feature Request: Review and update Unit tests 🚀 Feature Request: Use wp-env tooling for PHPUnit CI runs

4 participants