Empty PHP settings from the parent process override per-test `-d` settings forwarded to child processes (breaks PCOV coverage)

[mbeccati]

Q	A
PHPUnit version	12.5+
PHP version	8.3+
Installation Method	git

Summary

Three tests are failing when pcov is enabled. xdebug builds are passing:

https://github.com/mbeccati/php-latest-builds/actions/runs/26357169506

Current behavior

1) /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/code-coverage/disable-coverage-targeting.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
   2026-05-24 09:12:59
 
  Summary:
-  Classes: 100.00% (2/2)
-  Methods: 100.00% (2/2)
-  Lines:   100.00% (2/2)
-
-PHPUnit\TestFixture\DisableCoverageTargeting\Bar
-  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  1/  1)
-PHPUnit\TestFixture\DisableCoverageTargeting\Foo
-  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  1/  1)
+  Classes:  0.00% (0/2)
+  Methods:  0.00% (0/2)
+  Lines:    0.00% (0/2)

/home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/code-coverage/disable-coverage-targeting.phpt:42
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:381
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:381
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/TestRunner.php:64
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/Application.php:254

2) /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/code-coverage/include-in-code-coverage.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
   2026-05-24 09:13:00
 
  Summary:
-  Classes: 100.00% (1/1)
-  Methods: 100.00% (1/1)
-  Lines:   100.00% (1/1)
-
-PHPUnit\TestFixture\Event\IncludeInCodeCoverage\Foo
-  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  1/  1)
+  Classes:  0.00% (0/1)
+  Methods:  0.00% (0/1)
+  Lines:    0.00% (0/1)

/home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/code-coverage/include-in-code-coverage.phpt:41
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:381
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:381
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/TestRunner.php:64
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/Application.php:254

3) /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/regression/5218.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
   2026-05-24 09:15:02
 
  Summary:
-  Classes: 100.00% (1/1)
-  Methods: 100.00% (1/1)
-  Lines:   100.00% (1/1)
-
-PHPUnit\TestFixture\Issue5218\Issue5218
-  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  1/  1)
+  Classes:  0.00% (0/1)
+  Methods:  0.00% (0/1)
+  Lines:    0.00% (0/1)

/home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/regression/5218.phpt:39
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:381
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:381
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/TestRunner.php:64
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/Application.php:254

How to reproduce

Run test suite using php + pcov

Expected behavior

No failures. I currently have little time and I apologise if I can't help more than this. Failures started with the build from May 21, 12:12 PM GMT+2.

[sebastianbergmann]

This is a regression introduced by the sebastian/environment 8.1.0 → 8.1.1 dependency bump (commit 7aae6a0ed, May 21 10:47 +0200; right before your failing 12:12 build), and it is the same bug as sebastianbergmann/environment#99.

Runtime::getCurrentSettings() changed how it skips settings:

// 8.1.0 — empty values were skipped
if ($set === false || $set === '') { continue; }

// 8.1.1 — only false is skipped; empty strings now flow through
if ($set === false) { continue; }

All three tests are PHPT tests. Their --FILE-- section runs in a child PHP process spawned by JobRunner, and each sets the PCOV target directory via --INI--, e.g.:

pcov.directory=tests/end-to-end/code-coverage/_files/disable-coverage-targeting

That --INI-- value lands in the Job's phpSettings (src/Runner/Phpt/TestCase.php:135-137, :177). Then JobRunner::buildCommand() appends the outer process's PCOV settings after the job's own settings (src/Util/PHP/JobRunner.php:228-238):

$phpSettings = array_merge(
    $phpSettings,                                            // includes --INI-- pcov.directory=<fixture>
    $runtime->getCurrentSettings(array_keys($pcovSettings)), // appended AFTER
);

The outer test-suite process runs with PCOV loaded but pcov.directory unset and ini_get() returns ''. Under php -n, compiledDefaults() cannot see PCOV (it is loaded only from php.ini), so the empty value is not recognized as a default. The chain is exactly issue #99:

1. ini_get('pcov.directory') → ''
2. 8.1.1 no longer skips empty → proceeds
3. not in ini files, absent from compiled defaults (null === '' is false) → treated as a real override
4. getCurrentSettings() returns pcov.directory= (empty)

Because getCurrentSettings() is merged after the job settings, the child receives:

-d pcov.directory=<fixture> ... -d pcov.directory=

PHP applies -d flags in order, so the last (empty) one wins. PCOV is left with an empty directory, instruments nothing, and reports 0%.

Xdebug builds pass because the PCOV branch is not taken, and Xdebug coverage does not depend on pcov.directory.

The May 21 commit "Quote PHP setting values containing =" (1dc3edbc1) is a red herring here: these directory values contain no =, ;, or ", so processSettingValue() leaves them untouched. The trigger is the version bump of sebastian/environment. Both 12.5 (sebastian/environment 8.1.1) and main (sebastian/environment ^9.3.1) are affected; no fix has landed yet.

Solutions

1. In sebastian/environment (true fix, also closes Runtime::getCurrentSettings() forwards empty-string overrides for php.ini-only extensions environment#99): restore skipping of empty values in getCurrentSettings() when there is no INI evidence on either side; i.e. do not forward name= when the value is empty and the setting is absent from both the loaded ini files and the compiled defaults. This fixes both the PCOV failures here and the ZTS-APCu fatal in Runtime::getCurrentSettings() forwards empty-string overrides for php.ini-only extensions environment#99.

2. In PHPUnit JobRunner::buildCommand() (defensive hardening): merge the auto-detected PCOV/Xdebug settings before the job's own settings so explicit per-test --INI-- values always take precedence. This stops outer-process settings from clobbering explicit --INI-- overrides, but does not address the broader empty-forwarding problem (e.g. plain process isolation with no --INI--).

[sebastianbergmann]

I currently have little time and I apologise if I can't help more than this.

Thank you for bringing this to my attention! This is/was already helpful :)

[sebastianbergmann]

I am about to push a patch implementing the aforementioned defensive hardening / special case handling for PCOV and Xdebug INI settings passed for collecting code coverage for PHPT tests.

The general case should be fixed by sebastianbergmann/environment#100.

[mbeccati]

Thanks @sebastianbergmann.

The latest run however had more regressions:

1) /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/event/invalid-coverage-metadata.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
 Test Runner Configured
 Event Facade Sealed
 Test Suite Loaded (1 test)
-Static Analysis for Code Coverage Started
-Static Analysis for Code Coverage Finished (0 cache hits, 1 cache misses)
-Test Runner Started
-Test Suite Sorted
-Test Runner Execution Started (1 test)
-Test Suite Started (%s%etests%eend-to-end%eevent%e_files%einvalid-coverage-metadata%ephpunit.xml, 1 test)
-Test Suite Started (default, 1 test)
-Test Suite Started (PHPUnit\TestFixture\Event\InvalidCoverageMetadata\InvalidCoverageMetadataTest, 1 test)
-Test Preparation Started (PHPUnit\TestFixture\Event\InvalidCoverageMetadata\InvalidCoverageMetadataTest::testOne)
-Test Prepared (PHPUnit\TestFixture\Event\InvalidCoverageMetadata\InvalidCoverageMetadataTest::testOne)
-Test Passed (PHPUnit\TestFixture\Event\InvalidCoverageMetadata\InvalidCoverageMetadataTest::testOne)
-Test Triggered PHPUnit Warning (PHPUnit\TestFixture\Event\InvalidCoverageMetadata\InvalidCoverageMetadataTest::testOne)
-Class PHPUnit\TestFixture\Event\InvalidCoverageMetadata\This\Does\Not\Exist is not a valid target for code coverage
-Test Finished (PHPUnit\TestFixture\Event\InvalidCoverageMetadata\InvalidCoverageMetadataTest::testOne)
-Test Suite Finished (PHPUnit\TestFixture\Event\InvalidCoverageMetadata\InvalidCoverageMetadataTest, 1 test)
-Test Suite Finished (default, 1 test)
-Test Suite Finished (%s%etests%eend-to-end%eevent%e_files%einvalid-coverage-metadata%ephpunit.xml, 1 test)
-Test Runner Execution Finished
-Test Runner Finished
+Test Runner Triggered Warning (No code coverage driver available)
 PHPUnit Finished (Shell Exit Code: 1)

/home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/event/invalid-coverage-metadata.phpt:29
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:374
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:374
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/TestRunner.php:64
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/Application.php:229

2) /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/generic/empty-source-filter.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
 Runtime:       PHP 8.3.31-dev
 Configuration: /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/_files/empty-source-filter/phpunit.xml
 
-.                                                                   1 / 1 (100%)
-
-Time: %s, Memory: %s
-
 There was 1 PHPUnit test runner warning:
 
-1) Configured source filter (include-path: %snonexistent-directory) does not match any files, code coverage will not be processed
+1) No code coverage driver available
 
-OK, but there were issues!
-Tests: 1, Assertions: 1, PHPUnit Warnings: 1.
+No tests executed!

/home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/generic/empty-source-filter.phpt:26
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:374
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:374
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/TestRunner.php:64
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/Application.php:229

3) /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/regression/5218.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
 
 Runtime:       PHP 8.3.31-dev
 
-.                                                                   1 / 1 (100%)
+There was 1 PHPUnit test runner warning:
 
-Time: %s, Memory: %s MB
+1) No code coverage driver available
 
-OK (1 test, 1 assertion)
-
-
-Code Coverage Report:
-  %s
-
- Summary:
-  Classes: 100.00% (1/1)
-  Methods: 100.00% (1/1)
-  Lines:   100.00% (1/1)
-
-PHPUnit\TestFixture\Issue5218\Issue5218
-  Methods: 100.00% ( 1/ 1)   Lines: 100.00% (  1/  1)
+No tests executed!

/home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/regression/5218.phpt:28
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:374
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:374
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/TestRunner.php:64
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/Application.php:229

4) /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/regression/5351.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
 Runtime:       PHP 8.3.31-dev
 Configuration: /home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/regression/5351/phpunit.xml
 
-W                                                                   1 / 1 (100%)
+There was 1 PHPUnit test runner warning:
 
-Time: %s, Memory: %s MB
+1) No code coverage driver available
 
-1 test triggered 1 PHPUnit warning:
-
-1) PHPUnit\TestFixture\Issue5351\GreeterTest::testGreets
-Class PHPUnit\TestFixture\Issue5351\DoesNotExist is not a valid target for code coverage
-
-%sGreeterTest.php:18
-
-OK, but there were issues!
-Tests: 1, Assertions: 1, PHPUnit Warnings: 1.
-
-
-Code Coverage Report:
-  %s
-
- Summary:
-  Classes:  0.00% (0/1)
-  Methods:  0.00% (0/1)
-  Lines:    0.00% (0/1)
+No tests executed!

/home/runner/work/php-latest-builds/php-latest-builds/tests/end-to-end/regression/5351.phpt:25
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:374
/home/runner/work/php-latest-builds/php-latest-builds/src/Framework/TestSuite.php:374
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/TestRunner.php:64
/home/runner/work/php-latest-builds/php-latest-builds/src/TextUI/Application.php:229

[sebastianbergmann]

It seems to work for me both locally and in PHPUnit's own pipelines: https://github.com/sebastianbergmann/phpunit/actions

I do not know how you run PHPUnit's tests in your pipelines. I am genuinely grateful for the many useful bug reports I have received from you that originate in your pipelines: they have caught real problems. But I cannot debug and fix an issue like this when I am unable to reproduce it on my local machine, or at least in PHPUnit's own pipeline.

The three tests that fail for you ...

• tests/end-to-end/event/invalid-coverage-metadata.phpt
• tests/end-to-end/generic/empty-source-filter.phpt
• tests/end-to-end/regression/5218.phpt

... all have a --SKIPIF-- condition that skips the execution of the test when no code coverage driver is available:

// tests/end-to-end/event/invalid-coverage-metadata.phpt
if (!extension_loaded('pcov')) {
    print "skip: this test requires pcov\n";
}

// tests/end-to-end/generic/empty-source-filter.phpt
if (!extension_loaded('xdebug') && !extension_loaded('pcov')) {
    print 'skip: Extension Xdebug or PCOV must be loaded.';
}

// tests/end-to-end/regression/5218.phpt
require __DIR__ . '/../../_files/skip-if-requires-code-coverage-driver.php';

In your pipeline, these skip conditions apparently do not skip the execution of the tests in question. They are executed even though no code coverage driver is available, and they therefore fail. I cannot reproduce that: on my machine, and in PHPUnit's own pipeline, the skip condition works as intended: the tests are skipped when no driver is present, and they pass when a driver (Xdebug) is present.

[sebastianbergmann]

The new failures, all now reporting "No code coverage driver available") are reproducible, and they share the root cause of the original report: this time it surfaces through pcov.enabled rather than pcov.directory.

Here is what is going on, and in particular why these tests run instead of being skipped:

The --SKIPIF-- guards only check extension_loaded('pcov') / extension_loaded('xdebug'). In your pcov matrix legs the extension genuinely is loaded (you append extension=pcov.so to php.ini), so the guard correctly concludes that a driver is present and does not skip.

PHPUnit's coverage layer, however, uses a stricter condition. SebastianBergmann\CodeCoverage\Driver\Selector::forLineCoverage() asks Runtime::hasPCOV(), which is:

extension_loaded('pcov') && ini_get('pcov.enabled') === '1'

The PHPT --FILE-- body runs in a child process spawned by JobRunner, and the PCOV INI settings forwarded from the parent into that child (via sebastian/environment's Runtime::getCurrentSettings()) do not reproduce pcov.enabled === '1' there. So in the child extension_loaded('pcov') is still true (the --SKIPIF-- guard passes) while hasPCOV() is false (no usable driver), and the test runs and then fails with "No code coverage driver available". This is the same parent→child INI-forwarding regression as sebastianbergmann/environment#99 (sebastian/environment 8.1.0 → 8.1.1), just landing on a different setting.

The reason I did not catch this is a gap in PHPUnit's own pipeline: these PHPT tests only ever run with Xdebug, in the single dedicated code coverage job; PCOV appears only in the PHAR job, which runs a different subset of tests. So the "PCOV loaded via php.ini, whole suite" path your pipeline exercises is one mine never does. Thank you for surfacing it.

The fix is to make the child process actually receive a working driver configuration (the JobRunner change plus sebastianbergmann/environment#100), not to relax the --SKIPIF-- guards: these tests exist to prove that coverage is collected, so making them skip when the driver is misconfigured would only hide regressions like this one.

I will follow up here once the sebastian/environment fix is released.

However, any feedback on sebastianbergmann/environment#100 would be very much appreciated. Maybe you could try it in your pipeline? Thank you!

[mbeccati]

@sebastianbergmann I've added the following to the build:

  cd vendor/sebastian/environment
  (curl -sL https://patch-diff.githubusercontent.com/raw/sebastianbergmann/environment/pull/100.diff | patch -p1) || true

But I'm still getting the "No code coverage driver available" failures.

[sebastianbergmann]

I have now released new versions of sebastian/environment with sebastianbergmann/environment#100 because sebastianbergmann/environment#100 (comment) gave me enough confidence.

I will shortly push improvements for the skip conditions of the coverage-related PHPT tests in question.