Validators

Validators are Python scripts that run on the test machines. They are developed by the assignment creators, in order to check the behavior or output of student code submissions for correctness.

Since OpenSubmit makes no assumption about your style of student code evaluation, these scripts are rather generic. They:

• Need to have the name validator.py
• Need to start the student code in the same directory as the validator file, assuming that it is already compiled.
• Need to terminate with exit code 0 in case the student code execution went fine.

Validators can also be uploaded in the teacher backend as ZIP / TGZ archive file, which allows to deploy custom validator libraries (e.g. profiling tools) to the test machine. The executor daemon on the test machine makes sure that all validator files are uncompressed in the same directory as the student code.

The OpenSubmit daemon on the execution host calls the validator script with a command-line argument. This is a file name that can be used by the validator script to store performance information. Data contained in that file is sent to the OpenSubmit web server as additional performance information, which can be analyzed in the backend.