Failed test control methods do not stop testing of class

[oschwald]

The documentation states, " If a test control method fails, the class/method will fail and testing for that class should stop." However, testing for the class does not stop. I believe this used to be the case, but it doesn't seem to be anymore.

Example:

package TestsFor::Fail;

use Test::Class::Moose;
use Throwable::Error;

sub test_setup {
    fail();
}

sub test_ok {
    ok(1);
}

package main;

use Test::Class::Moose::CLI;
Test::Class::Moose::CLI->new_with_options->run;

Output:

1..1
not ok 1 - TestsFor::Fail {
    1..1
    not ok 1
    #   Failed test at /home/greg/.plenv/versions/5.28.2/lib/perl5/site_perl/5.28.2/Test/Class/Moose/Role/Executor.pm line 168.
    not ok 2 - TestsFor::Fail->test_setup failed
    # Failed test 'TestsFor::Fail->test_setup failed'
    # at /home/greg/.plenv/versions/5.28.2/lib/perl5/site_perl/5.28.2/Test/Class/Moose/Role/Executor.pm line 168.
    # Tests may not be run in test control methods (test_setup) at /home/greg/.plenv/versions/5.28.2/lib/perl5/site_perl/5.28.2/Test/Class/Moose/Role/Executor.pm line 411.
    ok 3 - test_ok {
        ok 1
        1..1
    }
}

# Failed test 'TestsFor::Fail'
# at /home/greg/.plenv/versions/5.28.2/lib/perl5/site_perl/5.28.2/Test2/Tools/AsyncSubtest.pm line 23.
# IPC is waiting for children to finish...
# Looks like you failed 1 test of 1.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests 

Test Summary Report
-------------------
test.t (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=1, Tests=1,  1 wallclock secs ( 0.01 usr  0.00 sys +  0.40 cusr  0.04 csys =  0.45 CPU)
Result: FAIL

As you can see, the test_ok method is still run. We use the test control methods to set up our fixtures, and this issue can result in a lot of unnecessary noise when the fixture setup fails.

[autarch]

Yes, definitely a bug. A PR to fix this would be welcome.