Skip to content

Crashing in subtests does not fail the subtest #1075

@mediaalpha-blackmore

Description

@mediaalpha-blackmore

Given the following, to simulate a subroutine or external module call that may die:

use Test::More tests=>1;
subtest 'B'=>sub {ok(1);...;ok(1)};

The output from prove -v is

1..1
# Subtest: B
    ok 1
    1..1
ok 1 - B
Unimplemented at ./crashdemo.t line 10.
# Looks like your test exited with 255 just after 1.

The desired behavior would be for subtest B to not report okay:

1..1
# Subtest: B
    ok 1
    # Subtest died: Unimplemented at ./crashdemo.t line 10.
    1..1
    # Looks like you failed 1 test of 1.
not ok 1 - B

#   Failed test 'B'
#   at ./crashdemo.t line 10.
Unimplemented at ./crashdemo.t line 10.
# Looks like your test exited with 255 just after 1.

This may be possible around https://github.com/Test-More/test-more/blob/master/lib/Test/Builder.pm#L407 by setting $st_hub->{_passing} = 0; and/or $st_hub->{failed} = 1; in the crashing case, but I don't know enough about the ordering or the steps to submit a PR.

Note that proper test plans can produce the desired output, in cases where an accurate plan count is possible:

subtest 'B'=>sub {plan tests=>2; ok(1);...;ok(1)};

# Subtest: B
    1..2
    ok 1
    # Looks like you planned 2 tests but ran 1.
not ok 1 - B

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions