Refresh the testing stack#40
Merged
Merged
Conversation
Cap PHPUnit at ^9.6 deliberately: the integration suite extends WP_UnitTestCase from the WordPress core test framework, which still targets yoast/phpunit-polyfills 1.x and PHPUnit 9.6 or earlier. Track roave/security-advisories' renamed dev-latest branch, which also lets minimum-stability rise to stable. Require mockery/mockery explicitly, as the base TestCase uses MockeryPHPUnitIntegration directly but the package only arrived transitively via Brain Monkey. Replace the hand-rolled bin/php-lint with php-parallel-lint; its grep pattern was a character class rather than the intended alternation, so it could never match parse errors anyway. Rewrite bin/xml-lint with strict mode so a failed xmllint can no longer exit 0, and validate phpunit.xml.dist against its bundled schema too. Add a wp-env-phpunit helper that derives the plugin mount path from the clone directory name, instead of hardcoding plugin-slug, so integration tests work whatever the repository is cloned as. Rename the scripts to the standard names used across plugins (cs-fix, test:unit, test:integration), add multisite, coverage, and Rector entries, and describe each one so composer list is self-documenting. Normalise composer.json with the newly added ergebnis/composer-normalize.
Add @Covers annotations to both FooTest classes so the existing beStrictAboutCoversAnnotation setting actually enforces something, and use assertSame for the string comparison so the type is checked as well as the value. Correct the base TestCase @SInCE tags to match the plugin's actual 0.1.0 version. Explain in phpunit.xml.dist why it stays on the 9.6 schema (the WordPress core test framework still targets PHPUnit 9.6 or earlier via phpunit-polyfills 1.x), drop the dead logging block, and write a Clover report alongside the text coverage output for tooling to consume. Teach the bootstrap to recognise the --testsuite=integration single-token form as well as the two-token form, and tolerate a missing argv, documenting the contract in the header.
Every mutant escaped, so mutation testing has been silently useless: Brain Monkey loads Patchwork on the first Monkey\setUp() call, and Patchwork's stream wrapper permanently replaces Infection's include interceptor, meaning classes autoloaded during a test received the original code rather than the mutant. CI never noticed because no MSI thresholds were set. Eagerly load the src classes in the unit bootstrap, which runs after Infection enables its interceptor but before Patchwork loads, so the mutated code is already in memory when tests execute. With the example tests now killing all three generated mutants, gate minMsi and minCoveredMsi at the measured 100%, and add the schema reference so editors can validate the config.
Pin the development environment to the supported minimums (WordPress 6.9 on PHP 8.4) and the tests environment to the latest (WordPress 7.0 on PHP 8.5), so day-to-day work happens against what the plugin guarantees while tests give early warning of compatibility issues. Both core refs exist as tags on the WordPress/WordPress mirror.
The Rector php84 set arriving in the code-standards branch flags the wrapped ( new Testee() ) form, so move to the native syntax now and keep the combined rector dry-run clean.
This was referenced Jun 11, 2026
The pre-refresh workflow calls composer unit, composer integration and bin/php-lint directly, so renaming the scripts would leave this branch red until the CI refresh lands. Deprecated aliases and a thin php-lint shim keep every merge step green; the CI refresh removes them when it switches the workflow to the new names.
This was referenced Jun 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This refreshes the testing toolchain while honouring one deliberate ceiling: PHPUnit stays at ^9.6 because the integration suite extends WP_UnitTestCase from the WordPress core test framework, which still targets phpunit-polyfills 1.x and PHPUnit 9.6 or earlier.
The headline find is that mutation testing has been silently useless. Brain Monkey loads Patchwork on the first
Monkey\setUp()call, and Patchwork's stream wrapper permanently replaces Infection's include interceptor, so every mutant escaped — CI stayed green only because no thresholds were set. The unit bootstrap now eagerly loads thesrcclasses while Infection's interceptor is still in place, after which all three generated mutants are killed. The measured MSI and covered MSI are both 100%, sominMsiandminCoveredMsiare gated at exactly that.Elsewhere, the Composer scripts adopt the standard names used across plugins (
cs-fix,test:unit,test:integration, plus new multisite, coverage, and Rector entries), each with a description socomposer listis self-documenting. A newbin/wp-env-phpunithelper derives the wp-env mount path from the clone directory name instead of hardcodingplugin-slug. PHP syntax linting moves to php-parallel-lint, replacing a hand-rolled script whose grep pattern was a character class and could never match a parse error, andbin/xml-lintgains strict mode plus schema validation ofphpunit.xml.dist. Mockery is now an explicit dependency since the base TestCase uses it directly, roave/security-advisories tracks its renameddev-latestbranch (letting minimum-stability rise to stable), and composer.json is normalised via ergebnis/composer-normalize. Two notes on that: a script namednormalizewould be skipped by Composer in favour of the plugin's own command, so none is defined —composer normalizeworks directly; and yoast/wp-test-utils was considered but skipped, as its latest release (1.2.1) pins phpunit-polyfills ^1.1.5 against this repo's ^4, so the hand-rolled TestCase stays.The example tests gain
@coversannotations (docblock style, deliberately not attributes, per the PHPUnit ceiling), making the existingbeStrictAboutCoversAnnotationsetting enforceable, and the bootstrap now recognises the--testsuite=integrationsingle-token form. Finally, wp-env pins the dev environment to the supported minimums (WordPress 6.9, PHP 8.4) and the tests environment to the latest (WordPress 7.0, PHP 8.5); both refs exist as tags on the WordPress/WordPress mirror.The CI workflow still calls
composer unit,composer integration, andbin/php-lint; those references are being updated separately.During the transition the old CI entry points keep working:
unitandintegrationremain as deprecated aliases andbin/php-lintbecomes a thin shim over parallel-lint, all removed by #35 when the workflow switches to the new names.