Description
Naming test cases with a new-line character on the first line causes the name of the function to be placed in the results.json file instead of the human-readable name in the docstring of the testcase.
To Reproduce
Create a testcase and name it with a docstring that has a new-line character on the first line. This file is called tests.py.
import unittest
from gradescope_utils.autograder_utils.decorators import weight, number
class TestAutograder(unittest.TestCase):
@weight(1.0)
@number("1")
def test_prob1(self):
"""
A human-readable name.
"""
pass
Running the above with the JSONTestRunner provided in gradescope_utils produces the following results.json:
{
"tests": [
{
"name": "test_prob1 (tests.TestAutograder)",
"score": 1.0,
"max_score": 1.0,
"number": "1"
},
],
"leaderboard": [],
"execution_time": "0.00",
"score": 10.0
}
Expected behavior
Should have produced the following results.json file:
{
"tests": [
{
"name": "A human-readable name.",
"score": 1.0,
"max_score": 1.0,
"number": "1"
},
],
"leaderboard": [],
"execution_time": "0.00",
"score": 10.0
}
Additional context
Removing the new-line character seems to fix the issue. I.e. the following produces the expected result.
import unittest
from gradescope_utils.autograder_utils.decorators import weight, number
class TestAutograder(unittest.TestCase):
@weight(1.0)
@number("1")
def test_prob1(self):
"""A human-readable name."""
pass
This also seems to only be the case inside of Gradescope's docker containers used for actually grading student submissions. This did not happen when I ran the code on my macbook.
Description
Naming test cases with a new-line character on the first line causes the name of the function to be placed in the
results.jsonfile instead of the human-readable name in the docstring of the testcase.To Reproduce
Create a testcase and name it with a docstring that has a new-line character on the first line. This file is called
tests.py.Running the above with the
JSONTestRunnerprovided ingradescope_utilsproduces the following results.json:{ "tests": [ { "name": "test_prob1 (tests.TestAutograder)", "score": 1.0, "max_score": 1.0, "number": "1" }, ], "leaderboard": [], "execution_time": "0.00", "score": 10.0 }Expected behavior
Should have produced the following
results.jsonfile:{ "tests": [ { "name": "A human-readable name.", "score": 1.0, "max_score": 1.0, "number": "1" }, ], "leaderboard": [], "execution_time": "0.00", "score": 10.0 }Additional context
Removing the new-line character seems to fix the issue. I.e. the following produces the expected result.
This also seems to only be the case inside of Gradescope's docker containers used for actually grading student submissions. This did not happen when I ran the code on my macbook.