API Documentation

The Autograder provides a FastAPI-based REST API for integration with learning management systems.

Starting the API

# Development
uvicorn web.main:app --reload

# Production
docker-compose up

API documentation (Swagger UI) available at: http://localhost:8000/docs

Authentication

Most endpoints are public and require no authentication. Two endpoints designed for machine-to-machine integration (GitHub Action external mode) are protected with a Bearer token.

Protected Endpoints

Endpoint	Method
`/api/v1/configs/id/{config_id}`	GET
`/api/v1/submissions/external-results`	POST

Setup

Set the AUTOGRADER_INTEGRATION_TOKEN environment variable on the server:

# Generate a strong random token
export AUTOGRADER_INTEGRATION_TOKEN=$(openssl rand -hex 32)

When the variable is not set or empty, the server will fail to start the integration auth configuration and protected endpoints will return 503 Service Unavailable. The token must be configured and non-empty before using integration endpoints.

Usage

Include the token in the Authorization header:

curl -H "Authorization: Bearer $AUTOGRADER_INTEGRATION_TOKEN" \
     http://localhost:8000/api/v1/configs/id/1

Error Responses

Status	Condition
`401 Unauthorized`	Missing or invalid token

Endpoints Overview

Method	Endpoint	Description	Auth
`POST`	`/api/v1/configs`	Create a grading configuration
`GET`	`/api/v1/configs`	List all grading configurations
`GET`	`/api/v1/configs/id/{config_id}`	Get configuration by internal ID	✓
`GET`	`/api/v1/configs/{external_assignment_id}`	Get configuration by assignment ID
`PUT`	`/api/v1/configs/{config_id}`	Update a grading configuration
`PUT`	`/api/v1/configs/external/{external_assignment_id}`	Update a grading configuration by external assignment ID
`POST`	`/api/v1/submissions`	Submit code for grading
`GET`	`/api/v1/submissions/{submission_id}`	Get submission details and results
`GET`	`/api/v1/submissions/user/{external_user_id}`	List submissions by user
`POST`	`/api/v1/submissions/external-results`	Ingest externally computed grading results	✓
`POST`	`/api/v1/execute`	Execute code without grading (DCE)
`GET`	`/api/v1/templates`	List available grading templates
`GET`	`/api/v1/templates/{template_name}`	Get template details
`GET`	`/api/v1/health`	Health check
`GET`	`/api/v1/ready`	Readiness check

Grading Configurations

Create Grading Configuration

POST /api/v1/configs
Content-Type: application/json

Request Body:

{
  "external_assignment_id": "assignment-42",
  "template_name": "input_output",
  "criteria_config": {
    "base": {
      "weight": 100,
      "subjects": [
        {
          "subject_name": "Functionality",
          "weight": 60,
          "tests": [
            {
              "name": "expect_output",
              "parameters": {
                "inputs": ["Alice"],
                "expected_output": "Hello, Alice!",
                "program_command": "python3 main.py"
              },
              "weight": 100
            }
          ]
        }
      ]
    }
  },
  "languages": ["python", "java"],
  "setup_config": {
    "required_files": ["main.py"],
    "setup_commands": []
  },
  "include_feedback": true,
  "feedback_config": {
    "general": {
      "report_title": "Evaluation Report",
      "show_score": true,
      "show_passed_tests": false,
      "add_report_summary": true,
      "online_content": [
        {
          "url": "https://docs.python.org/3/tutorial/",
          "description": "Python Tutorial"
        }
      ]
    }
  }
}

Request Fields:

Field	Type	Required	Description
`external_assignment_id`	string	✓	External assignment ID from your LMS/platform
`template_name`	string	✓	Template to use (`input_output`, `web_dev`, `api_testing`)
`criteria_config`	object	✓	Grading criteria tree configuration
`languages`	list[string]	✓	Supported languages: `python`, `java`, `node`, `cpp`
`setup_config`	object	✗	Setup configuration for preflight checks
`include_feedback`	boolean	✗	Whether to generate a feedback report after grading (default: `true`)
`feedback_config`	object	✗	Feedback preferences — see FeedbackConfig Reference

Response (201 Created):

{
  "id": 1,
  "external_assignment_id": "assignment-42",
  "template_name": "input_output",
  "criteria_config": { ... },
  "languages": ["python", "java"],
  "setup_config": { ... },
  "include_feedback": true,
  "feedback_config": { ... },
  "version": 1,
  "created_at": "2026-02-16T10:00:00Z",
  "updated_at": "2026-02-16T10:00:00Z",
  "is_active": true
}

Error (400): Configuration for this assignment already exists.

List Grading Configurations

GET /api/v1/configs?limit=100&offset=0

Query Parameters:

Parameter	Type	Default	Description
`limit`	int	100	Maximum results to return
`offset`	int	0	Pagination offset

Response (200 OK):

[
  {
    "id": 1,
    "external_assignment_id": "assignment-42",
    "template_name": "input_output",
    "criteria_config": { ... },
    "languages": ["python"],
    "setup_config": null,
    "version": 1,
    "created_at": "2026-02-16T10:00:00Z",
    "updated_at": "2026-02-16T10:00:00Z",
    "is_active": true
  }
]

Get Grading Configuration by Internal ID

GET /api/v1/configs/id/{config_id}
Authorization: Bearer <token>

Protected — requires a valid integration token (see Authentication).

Fetch a grading configuration using its internal database ID. This is used by external/private mode (e.g. GitHub Action) when only the grading_config_id is known.

Response (200 OK):

{
  "id": 1,
  "external_assignment_id": "assignment-42",
  "template_name": "input_output",
  "criteria_config": { ... },
  "languages": ["python", "java"],
  "setup_config": { ... },
  "include_feedback": true,
  "feedback_config": { ... },
  "version": 1,
  "created_at": "2026-02-16T10:00:00Z",
  "updated_at": "2026-02-16T10:00:00Z",
  "is_active": true
}

Error (404): Configuration not found.

Get Grading Configuration

GET /api/v1/configs/{external_assignment_id}

Response (200 OK):

{
  "id": 1,
  "external_assignment_id": "assignment-42",
  "template_name": "input_output",
  "criteria_config": { ... },
  "languages": ["python", "java"],
  "setup_config": null,
  "version": 1,
  "created_at": "2026-02-16T10:00:00Z",
  "updated_at": "2026-02-16T10:00:00Z",
  "is_active": true
}

Error (404): Configuration not found.

Update Grading Configuration

PUT /api/v1/configs/{config_id}
Content-Type: application/json

Request Body (all fields optional):

{
  "template_name": "web_dev",
  "criteria_config": { ... },
  "languages": ["python", "node"],
  "setup_config": { ... },
  "include_feedback": false,
  "feedback_config": { ... },
  "is_active": false
}

Response (200 OK): Returns the updated configuration object (same schema as create response).

Error (404): Configuration not found.

Update Grading Configuration by External ID

PUT /api/v1/configs/external/{external_assignment_id}
Content-Type: application/json

Use this endpoint when you only know the LMS-specific assignment identifier and need to update the associated grading configuration without looking up its internal database ID. It accepts the same payload as the standard update route and performs a partial update via exclude_unset behavior.

Request Body (all fields optional):

{
  "template_name": "web_dev",
  "criteria_config": { ... },
  "languages": ["python", "node"],
  "setup_config": { ... },
  "include_feedback": true,
  "feedback_config": { ... },
  "is_active": false
}

Response (200 OK): Updated configuration object (same schema as the create response).

Error (404): Configuration not found for the supplied external_assignment_id.

FeedbackConfig Reference

The feedback_config object controls how the DefaultReporter generates the post-grading Markdown report. All fields are optional.

{
  "general": {
    "report_title": "Evaluation Report",
    "show_score": true,
    "show_passed_tests": false,
    "add_report_summary": true,
    "online_content": [
      {
        "url": "https://docs.python.org/3/tutorial/",
        "description": "Python Official Tutorial"
      }
    ]
  },
  "default": {
    "category_headers": {
      "base": "✅ Essential Requirements",
      "penalty": "❌ Points to Improve",
      "bonus": "⭐ Extra Points"
    }
  }
}

`general` fields

Field	Type	Default	Description
`report_title`	string	`"Evaluation Report"`	Custom title at the top of the report
`show_score`	boolean	`true`	Append the final score section at the bottom of the report
`show_passed_tests`	boolean	`false`	Include tests that passed (score = 100) in the results
`add_report_summary`	boolean	`true`	Add a summary section with template name and score breakdown
`online_content`	list[object]	`[]`	Learning resources appended at the end of the report

`online_content` object fields

Field	Type	Description
`url`	string	Full URL of the resource
`description`	string	Human-readable label for the link
`linked_tests`	list[string]	Optional: List of test names. If provided, the resource is only shown if at least one of these tests fails.

Resources without linked_tests are always displayed at the end of the report. Resources with linked_tests act as "targeted hints" that only appear when relevant.

`default` fields

Field	Type	Description
`category_headers`	object	Override section headers per category (`base`, `penalty`, `bonus`)

Submissions

Submit for Grading

POST /api/v1/submissions
Content-Type: application/json

Request Body:

{
  "external_assignment_id": "assignment-42",
  "external_user_id": "user_001",
  "username": "student123",
  "files": [
    {
      "filename": "main.py",
      "content": "print('Hello World')"
    }
  ],
  "language": "python",
  "metadata": {
    "attempt": 1
  }
}

Request Fields:

Field	Type	Required	Description
`external_assignment_id`	string	✓	External assignment ID (must match an existing grading config)
`external_user_id`	string	✓	External user ID from your platform
`username`	string	✓	Username of the submitter
`files`	list[object]	✓	List of files with `filename` and `content`
`language`	string	✗	Language override (defaults to first language in config)
`metadata`	object	✗	Optional metadata to attach to the submission

Response (200 OK):

{
  "id": 1,
  "grading_config_id": 1,
  "external_user_id": "user_001",
  "username": "student123",
  "status": "pending",
  "submitted_at": "2026-02-16T10:05:00Z",
  "graded_at": null,
  "final_score": null,
  "feedback": null,
  "result_tree": null,
  "focus": null
}

Note: Grading runs asynchronously in the background. Poll the submission endpoint to check for results.

Errors:

404: Grading configuration for the given assignment not found
400: Specified language is not supported for this assignment

Get Submission Results

GET /api/v1/submissions/{submission_id}

Response (Success — grading completed):

{
  "id": 1,
  "grading_config_id": 1,
  "external_user_id": "user_001",
  "username": "student123",
  "status": "completed",
  "submitted_at": "2026-02-16T10:05:00Z",
  "graded_at": "2026-02-16T10:05:03Z",
  "final_score": 85.5,
  "feedback": "## Grade: 85.5/100\n\n### ✅ Base Tests...",
  "result_tree": {
    "final_score": 85.5,
    "root": {
      "base": { ... },
      "bonus": { ... },
      "penalty": { ... }
    }
  },
  "focus": {
    "high_impact": [ ... ],
    "medium_impact": [ ... ],
    "low_impact": [ ... ]
  },
  "submission_files": {
    "main.py": "print('Hello World')"
  },
  "submission_metadata": null,
  "pipeline_execution": {
    "status": "success",
    "failed_at_step": null,
    "total_steps_planned": 7,
    "steps_completed": 7,
    "execution_time_ms": 4521,
    "steps": [
      {
        "name": "LOAD_TEMPLATE",
        "status": "success",
        "message": "Template loaded successfully"
      },
      {
        "name": "BUILD_TREE",
        "status": "success",
        "message": "Criteria tree built successfully"
      },
      {
        "name": "SANDBOX",
        "status": "success",
        "message": "Sandbox created and attached to pipeline"
      },
      {
        "name": "PRE_FLIGHT",
        "status": "success",
        "message": "All preflight checks passed"
      },
      {
        "name": "GRADE",
        "status": "success",
        "message": "Grading completed: 85.5/100"
      },
      {
        "name": "FEEDBACK",
        "status": "success",
        "message": "Feedback generated"
      }
    ]
  }
}

Response (Preflight Failure):

{
  "id": 2,
  "grading_config_id": 1,
  "external_user_id": "user_001",
  "username": "student123",
  "status": "failed",
  "submitted_at": "2026-02-16T10:05:00Z",
  "graded_at": null,
  "final_score": 0,
  "feedback": "## Preflight Check Failed\n\n### Setup Command Failed: Compile Calculator.java\n...",
  "result_tree": null,
  "focus": null,
  "submission_files": { ... },
  "submission_metadata": null,
  "pipeline_execution": {
    "status": "failed",
    "failed_at_step": "PRE_FLIGHT",
    "total_steps_planned": 7,
    "steps_completed": 3,
    "execution_time_ms": 1523,
    "steps": [
      {
        "name": "PRE_FLIGHT",
        "status": "fail",
        "message": "Setup command 'Compile Calculator.java' failed with exit code 1",
        "error_details": {
          "error_type": "setup_command_failed",
          "phase": "setup_commands",
          "command_name": "Compile Calculator.java",
          "failed_command": {
            "command": "javac Calculator.java",
            "exit_code": 1,
            "stderr": "Calculator.java:4: error: ';' expected..."
          }
        }
      }
    ]
  }
}

Response Fields (Detail View):

Field	Type	Description
`id`	int	Submission ID
`grading_config_id`	int	Associated grading configuration ID
`external_user_id`	string	External user identifier
`username`	string	Submitter's username
`status`	string	`pending`, `processing`, `completed`, or `failed`
`submitted_at`	datetime	Submission timestamp
`graded_at`	datetime\|null	Grading completion timestamp
`final_score`	float\|null	Final grade (0-100+, includes bonus)
`feedback`	string\|null	Human-readable feedback report
`result_tree`	object\|null	Detailed grading results (null if grading didn't run)
`focus`	object\|null	Focus analysis grouping failed tests by impact
`submission_files`	object	Submitted files as `{filename: content}` map
`submission_metadata`	object\|null	Optional metadata attached at submission time
`pipeline_execution`	object\|null	Pipeline execution details with step-by-step status

Error (404): Submission not found.

List User Submissions

GET /api/v1/submissions/user/{external_user_id}?limit=100&offset=0

Query Parameters:

Parameter	Type	Default	Description
`limit`	int	100	Maximum results to return
`offset`	int	0	Pagination offset

Response (200 OK):

[
  {
    "id": 1,
    "grading_config_id": 1,
    "external_user_id": "user_001",
    "username": "student123",
    "status": "completed",
    "submitted_at": "2026-02-16T10:05:00Z",
    "graded_at": "2026-02-16T10:05:03Z",
    "final_score": 85.5,
    "feedback": "...",
    "result_tree": { ... },
    "focus": null
  }
]

Ingest External Grading Results

POST /api/v1/submissions/external-results
Content-Type: application/json
Authorization: Bearer <token>

🔒 Protected — requires a valid integration token (see Authentication).

Persist grading results computed outside the cloud instance (e.g. GitHub Action running in external/private mode). Creates both a submission and a submission_result row, making the result visible through the standard submission retrieval endpoints.

Request Body:

{
  "grading_config_id": 1,
  "external_user_id": "user_001",
  "username": "student123",
  "language": "python",
  "status": "completed",
  "final_score": 85.5,
  "feedback": "## Grade: 85.5/100\n...",
  "result_tree": { ... },
  "focus": { ... },
  "pipeline_execution": { ... },
  "execution_time_ms": 4521,
  "error_message": null,
  "submission_metadata": {
    "repo": "org/repo",
    "run_id": "123"
  }
}

Request Fields:

Field	Type	Required	Description
`grading_config_id`	int	✓	Internal grading configuration ID
`external_user_id`	string	✓	External user ID from your platform
`username`	string	✓	Username of the submitter
`language`	string	✓	Language used for grading (must be supported by the config)
`status`	string	✓	`completed` or `failed`
`final_score`	float	✓	Final grading score (≥ 0)
`feedback`	string	✗	Generated feedback text
`result_tree`	object	✗	Scored result tree
`focus`	object	✗	Failed tests sorted by impact
`pipeline_execution`	object	✗	Pipeline step execution details
`execution_time_ms`	int	✓	Total execution time in milliseconds (≥ 0)
`error_message`	string	✗	Error message for failed runs
`submission_metadata`	object	✗	Repository/run metadata

Response (200 OK):

{
  "submission_id": 5,
  "grading_config_id": 1,
  "external_user_id": "user_001",
  "username": "student123",
  "status": "completed",
  "final_score": 85.5,
  "graded_at": "2026-02-16T10:05:03Z",
  "execution_time_ms": 4521
}

Errors:

404: Grading configuration not found
400: Language not supported for this configuration
422: Validation error (invalid status, negative score, missing required fields)

Deliberate Code Execution (DCE)

Execute code in a sandbox without grading. This is a stateless feature designed for testing and debugging — no data is persisted.

📚 For complete DCE documentation with examples in all languages, see Deliberate Code Execution.

Execute Code

POST /api/v1/execute
Content-Type: application/json

Request Body:

{
  "language": "python",
  "submission_files": [
    {
      "filename": "main.py",
      "content": "name = input('Enter name: ')\nprint(f'Hello, {name}!')"
    }
  ],
  "program_command": "python main.py",
  "test_cases": [["Alice"], ["Bob"]]
}

Request Fields:

Field	Type	Required	Description
`language`	string	✓	Programming language: `python`, `java`, `node`, or `cpp`
`submission_files`	list[object]	✓	Files with `filename` and `content`
`program_command`	string	✓	Command to execute (e.g., `python main.py`)
`test_cases`	list[list[string]]	✗	Optional test cases. Each test case is a list of stdin inputs. If omitted, the program runs once with no input.

Response (200 OK):

{
  "results": [
    {
      "output": "Enter name: Hello, Alice!\n",
      "category": "success",
      "error_message": null,
      "execution_time": 0.123
    },
    {
      "output": "Enter name: Hello, Bob!\n",
      "category": "success",
      "error_message": null,
      "execution_time": 0.118
    }
  ]
}

Response Fields:

Field	Type	Description
`results`	list[object]	One result per test case (or one result if no test cases provided)
`results[].output`	string	Combined stdout/stderr output
`results[].category`	string	`success`, `runtime_error`, `compilation_error`, `timeout`, or `system_error`
`results[].error_message`	string\|null	Error details if execution failed
`results[].execution_time`	float	Execution time in seconds

Errors:

400: Unsupported language
500: Internal server error during execution

Example — Simple Python:

curl -X POST http://localhost:8000/api/v1/execute \
  -H "Content-Type: application/json" \
  -d '{
    "language": "python",
    "submission_files": [
      {"filename": "main.py", "content": "print(\"Hello, World!\")"}
    ],
    "program_command": "python main.py"
  }'

Example — Java with Compilation:

curl -X POST http://localhost:8000/api/v1/execute \
  -H "Content-Type: application/json" \
  -d '{
    "language": "java",
    "submission_files": [
      {
        "filename": "Main.java",
        "content": "public class Main { public static void main(String[] args) { System.out.println(\"Hello!\"); } }"
      }
    ],
    "program_command": "javac Main.java && java Main"
  }'

Templates

List Available Templates

GET /api/v1/templates

Response (200 OK):

{
  "templates": [
    {
      "name": "input_output",
      "description": "...",
      "test_functions": [ ... ]
    },
    {
      "name": "web_dev",
      "description": "...",
      "test_functions": [ ... ]
    }
  ]
}

Error (503): Template service not initialized.

Get Template Details

GET /api/v1/templates/{template_name}

Response (200 OK): Returns detailed information about the template including all available test functions and their parameters.

Errors:

404: Template not found
503: Template service not initialized

Health & Readiness

Health Check

GET /api/v1/health

Response (200 OK):

{
  "status": "healthy",
  "timestamp": "2026-02-16T10:00:00Z",
  "version": "1.0.0"
}

Readiness Check

GET /api/v1/ready

Response (200 OK):

{
  "ready": true,
  "timestamp": "2026-02-16T10:00:00Z"
}

Response (503 Service Unavailable):

{
  "ready": false,
  "timestamp": "2026-02-16T10:00:00Z"
}

External Mode Integration

This section describes how the Autograder GitHub Action in external mode interacts with the API. The action makes exactly two authenticated calls per grading run.

Integration sequence

1. GET  /api/v1/configs/id/{grading_config_id}     # fetch grading config
2. POST /api/v1/submissions/external-results        # submit result (or failure)

Both endpoints require the Authorization: Bearer <token> header where the token matches AUTOGRADER_INTEGRATION_TOKEN on the server.

Config fetch

GET /api/v1/configs/id/{grading_config_id}
Authorization: Bearer <token>

Returns the full grading configuration object (see Get Grading Configuration by Internal ID). The action uses the following fields from the response:

Field	Used for
`id`	Sent back as `grading_config_id` in the result payload
`template_name`	Selects the grading template
`criteria_config`	Grading rubric
`languages`	Validates `submission-language`; defaults to `languages[0]` if omitted
`include_feedback`	Whether to run the feedback step
`setup_config`	Pre-flight setup commands and required files
`feedback_config`	Reporter configuration

Result submission (success)

POST /api/v1/submissions/external-results
Authorization: Bearer <token>
Content-Type: application/json

{
  "grading_config_id": 42,
  "external_user_id": "github-actor-login",
  "username": "github-actor-login",
  "language": "python",
  "status": "completed",
  "final_score": 85.5,
  "feedback": "## Grade: 85.5/100\n...",
  "result_tree": { ... },
  "focus": { ... },
  "pipeline_execution": { ... },
  "execution_time_ms": 3200,
  "error_message": null,
  "submission_metadata": {
    "repository": "org/repo",
    "commit_sha": "abc123",
    "run_id": "999",
    "actor": "student1",
    "ref": "refs/heads/main"
  }
}

Result submission (failure)

When grading fails, the action posts a failure payload before exiting non-zero:

{
  "grading_config_id": 42,
  "external_user_id": "github-actor-login",
  "username": "github-actor-login",
  "language": "python",
  "status": "failed",
  "final_score": 0.0,
  "feedback": null,
  "result_tree": null,
  "focus": null,
  "pipeline_execution": null,
  "execution_time_ms": 1500,
  "error_message": "Sandbox timed out after 30 s",
  "submission_metadata": { ... }
}

Error handling

Scenario	Action behaviour
`401` on config fetch	Exits non-zero; no result posted
`404` on config fetch	Exits non-zero; no result posted
5xx / network error on config fetch	Retries with exponential back-off; then exits non-zero
Grading raises exception	Posts failure payload, then exits non-zero
`401` / `4xx` on result submission	Exits non-zero (failure recorded in Action log)

Error Responses

All endpoints use FastAPI's standard error response format:

{
  "detail": "Error message describing what went wrong"
}

Common HTTP Status Codes

Code	Meaning
`200 OK`	Request succeeded
`400 Bad Request`	Invalid request data (e.g., unsupported language, duplicate config)
`404 Not Found`	Resource not found
`422 Unprocessable Entity`	Validation error (Pydantic schema validation failed)
`500 Internal Server Error`	Server error
`503 Service Unavailable`	Service not ready (e.g., template service not initialized)

Validation Error Format (422)

When Pydantic validation fails, the response includes detailed field-level errors:

{
  "detail": [
    {
      "loc": ["body", "languages", 0],
      "msg": "Unsupported language 'ruby'. Supported languages are: python, java, node, cpp",
      "type": "value_error"
    }
  ]
}

FilesExpand file tree

API.md

Latest commit

History

API.md

File metadata and controls

API Documentation

Starting the API

Authentication

Protected Endpoints

Setup

Usage

Error Responses

Endpoints Overview

Grading Configurations

Create Grading Configuration

List Grading Configurations

Get Grading Configuration by Internal ID

Get Grading Configuration

Update Grading Configuration

Update Grading Configuration by External ID

FeedbackConfig Reference

general fields

online_content object fields

default fields

Submissions

Submit for Grading

Get Submission Results

List User Submissions

Ingest External Grading Results

Deliberate Code Execution (DCE)

Execute Code

Templates

List Available Templates

Get Template Details

Health & Readiness

Health Check

Readiness Check

External Mode Integration

Integration sequence

Config fetch

Result submission (success)

Result submission (failure)

Error handling

Error Responses

Common HTTP Status Codes

Validation Error Format (422)

`general` fields

`online_content` object fields

`default` fields