Skip to content

Human-readable name of test case does not appear in results.json #21

@bryceadam1

Description

@bryceadam1

Description
Naming test cases with a new-line character on the first line causes the name of the function to be placed in the results.json file instead of the human-readable name in the docstring of the testcase.

To Reproduce
Create a testcase and name it with a docstring that has a new-line character on the first line. This file is called tests.py.

import unittest
from gradescope_utils.autograder_utils.decorators import weight, number

class TestAutograder(unittest.TestCase):
    @weight(1.0)
    @number("1")
    def test_prob1(self):
        """
        A human-readable name.
        """
        pass

Running the above with the JSONTestRunner provided in gradescope_utils produces the following results.json:

{
    "tests": [
        {
            "name": "test_prob1 (tests.TestAutograder)",
            "score": 1.0,
            "max_score": 1.0,
            "number": "1"
        },
    ],
    "leaderboard": [],
    "execution_time": "0.00",
    "score": 10.0
}

Expected behavior
Should have produced the following results.json file:

{
    "tests": [
        {
            "name": "A human-readable name.",
            "score": 1.0,
            "max_score": 1.0,
            "number": "1"
        },
    ],
    "leaderboard": [],
    "execution_time": "0.00",
    "score": 10.0
}

Additional context
Removing the new-line character seems to fix the issue. I.e. the following produces the expected result.

import unittest
from gradescope_utils.autograder_utils.decorators import weight, number

class TestAutograder(unittest.TestCase):
    @weight(1.0)
    @number("1")
    def test_prob1(self):
        """A human-readable name."""
        pass

This also seems to only be the case inside of Gradescope's docker containers used for actually grading student submissions. This did not happen when I ran the code on my macbook.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions