Hackeroos AI Agent Judge

Hackeroos intern project: This project was built by Mihir Shrivas during a Hackeroos internship, based on product and judging-workflow direction from Kasey Robinson (bitpixi). This repository is a fork of Mihirshrivas/Hackeroos-AI-Agent-Judge and keeps the intern's implementation credit visible.

Hackeroos AI Agent Judge is an MVP browser app and API for reviewing hackathon submissions with an agent-assisted judging workflow. It evaluates one criterion at a time, records supporting evidence, and produces structured feedback and leaderboard output for human judges to review.

What it explores

• Criterion-by-criterion assessment instead of one opaque overall prompt.
• Structured scores, evidence, feedback, and suggested improvements.
• Browser-based submission intake and Devpost CSV import.
• Batch judging, leaderboard views, and CSV export.
• An offline deterministic fallback for local testing without an API key.

The current criteria cover relevance and impact, originality and creativity, technical execution, design and presentation, and feasibility and potential.

Project layout

The working application is in hackeroos-judge/:

• api.py provides the FastAPI browser app and API.
• judge.py contains the judging workflow.
• batch_judge.py processes multiple submissions.
• import_devpost_csv.py handles Devpost exports.
• design/ contains the project report, diagrams, and wireframes.

Run locally

cd hackeroos-judge
python -m pip install -r requirements.txt

Create a .env file if you want to use an OpenAI model:

OPENAI_API_KEY=your_key_here
OPENAI_MODEL=gpt-5.2

Run the browser app:

python -m uvicorn api:app --reload

Then open http://127.0.0.1:8000. Without an API key, the MVP uses its offline fallback so the workflow can still be tested.

Human judgement remains final

This is an exploratory judging aid, not an autonomous decision-maker. Model output can miss context or overvalue what is easy to describe, so scores and feedback should be treated as supporting material for human judges rather than final results.

Attribution

• Intern implementation: Mihir Shrivas
• Product and judging-workflow direction: Kasey Robinson (bitpixi)
• Upstream repository: Mihirshrivas/Hackeroos-AI-Agent-Judge