Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions problems/miniscoreboard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# 🏆 miniscoreboard Benchmark Problem

A command-line sports match management system designed for benchmarking AI efficiency in data persistence, filtering, and analytical processing.

## 🚀 Version Evolution

### Version 1 (V1)
- **Focus**: Basic Persistence & Shell commands.
- **Commands**: `init`, `add-match`.
- **Infrastructure**: Data stored in `.miniscore/matches.dat` (Pipe-separated format).

### Version 2 (V2)
- **Focus**: Data Retrieval & Filtering.
- **Commands**: `history` (list all), `team <name>` (filter matches for a specific team).
- **Validation**: Added basic error handling for reserved characters and data types.

### Version 3 (V3)
- **Focus**: Analytical Logic & Data Portability.
- **Commands**:
- `leaderboard`: Calculates league standings using 3-1-0 point system. Complex sorting (Points > GD > GF).
- `summary`: High-level league statistics (Average goals, match counts).
- `export`: Serializes match data into `matches.json`.
- **Determinism**: Added `DETERMINISM RULES` to ensure consistent AI behavior across trials.

---

## 🧪 Running Benchmarks

```bash
# Dry run
bin/which-language run gemini miniscoreboard --lang python --trials 1 --dry-run

# Real trial
bin/which-language run gemini miniscoreboard --lang python --trials 1
```

## 📊 Evaluation Metrics
- **Pass Rate**: Ability to implement complex sorting logic correctly.
- **LOC**: Conciseness in handling file I/O vs processing.
- **Time/Cost**: Efficiency of the generation turns.
2 changes: 1 addition & 1 deletion problems/miniscoreboard/SPEC-v1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,4 @@ ERROR HANDLING
- Inputs containing '|': print "Error: The '|' character is reserved."
- Non-numeric scores: print "Error: Scores must be positive numbers."
- Empty team names: print "Error: Team names cannot be empty."
- Team playing itself: print "Error: A team cannot play against itself."
- Team playing itself: print "Error: A team cannot play against itself."
2 changes: 1 addition & 1 deletion problems/miniscoreboard/SPEC-v2.txt
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,4 @@ ERROR HANDLING
- Inputs containing '|': print "Error: The '|' character is reserved."
- Non-numeric scores: print "Error: Scores must be positive numbers."
- Empty team names: print "Error: Team names cannot be empty."
- Team playing itself: print "Error: A team cannot play against itself."
- Team playing itself: print "Error: A team cannot play against itself."
71 changes: 71 additions & 0 deletions problems/miniscoreboard/SPEC-v3.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
PROJECT: miniscoreboard
AUTHOR: Utkuhan Akar (V3 Upgrade) (251478036)
--------------------------------------------
VERSION: v3
DATE: 2026-04-20

========================================
OVERVIEW
========================================

miniscoreboard is a command-line sports match management tool.
V3 adds advanced analytics, league standings, and data portability.

All infrastructure and V1/V2 commands remain mandatory.
- init
- add-match
- history
- team

========================================
NEW COMMANDS (V3)
========================================

--- leaderboard ---
Usage: python miniscore.py leaderboard
Calculates points for all teams recorded in matches.dat.
- Win: 3 points
- Draw: 1 point
- Loss: 0 points

Output Format:
Sorted by Points (Descending), then GD (Goal Difference), then GF (Goals For).
[Rank] Team: Pts (W-D-L) GF-GA GD

Example Output:
[1] GALATASARAY: 6 pts (2-0-0) 5-1 +4
[2] FENERBAHCE: 3 pts (1-0-1) 3-2 +1
[3] BESIKTAS: 0 pts (0-0-2) 1-6 -5

If no matches exist, print "History is empty."

--- summary ---
Usage: python miniscore.py summary
Overview of league activity.

Output Format:
Total Teams: <Count>
Total Matches: <Count>
Total Goals: <Count>
Average Goals/Match: <Avg> (Fixed to 2 decimal places)

--- export ---
Usage: python miniscore.py export
Converts matches.dat into a JSON file named .miniscore/matches.json.
Prints "Exported <Count> matches to .miniscore/matches.json"

========================================
DETERMINISM RULES
========================================

- Tool must never produce timestamps or random IDs in outputs unless specified.
- Sorting in leaderboard must be stable. If two teams are equal in Pts, GD, and GF, sort alphabetically by Team Name.
- Output formatting (spaces, case) must exactly match the examples.

========================================
ERROR HANDLING
========================================

- Inherit all V1/V2 error cases.
- If export fails due to permissions: print "Error: Could not write export file."
- If leaderboard is requested with 0 matches: print "History is empty."
20 changes: 20 additions & 0 deletions problems/miniscoreboard/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Development Plan - miniscoreboard

This document outlines the evolutionary steps of the miniscoreboard project, focusing on a structured progression from basic data entry to advanced analytics.

## Phase 1: Core Foundation (V1) - [DONE]
- [x] Establish project directory structure (.miniscore/).
- [x] Implement `init` command for environment setup.
- [x] Implement `add-match` for atomic data entry.
- [x] Ensure basic file I/O operations for `matches.dat`.

## Phase 2: Data Retrieval (V2) - [DONE]
- [x] Implement `history` command to list all matches.
- [x] Implement `team` command for filtered search.
- [x] Enforce stable formatting for CLI output.

## Phase 3: Analytics & Portability (V3) - [CURRENT]
- [x] **Leaderboard System:** Develop a multi-tier sorting algorithm (Points > GD > GF > Alphabetical).
- [x] **League Summary:** Global calculation of match count, team count, and scoring averages.
- [x] **Data Export:** Build a JSON serializer to allow third-party tool integration.
- [x] **Deterministic Testing:** Ensure `test-v3.sh` yields 100% consistent results.
9 changes: 6 additions & 3 deletions problems/miniscoreboard/problem.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,11 @@
"binary_name": "miniscore",
"v1_spec": "SPEC-v1.txt",
"v1_test": "test-v1.sh",
"v1_prompt": "Implement {{binary_name}} as described in SPEC-v1.txt using {{language}}. The executable must be named '{{binary_name}}' and be runnable as ./{{binary_name}}. For compiled languages, include a Makefile or build script. For interpreted languages, ensure the {{binary_name}} file has a proper shebang line and is executable. Handle init and add-match commands. Verify your implementation passes all tests by running: bash test-v1.sh",
"v1_prompt": "Implement {{binary_name}} as described in SPEC-v1.txt. Use standard libraries. Verification: bash test-v1.sh",
"v2_spec": "SPEC-v2.txt",
"v2_test": "test-v2.sh",
"v2_prompt": "Read SPEC-v2.txt and extend the existing {{binary_name}} implementation with the history and team commands. Verify your implementation using bash test-v2.sh."
}
"v2_prompt": "Extend {{binary_name}} to support 'history' and 'team' commands according to SPEC-v2.txt. Verification: bash test-v2.sh",
"v3_spec": "SPEC-v3.txt",
"v3_test": "test-v3.sh",
"v3_prompt": "Finalize {{binary_name}} with 'leaderboard', 'summary', and 'export' as defined in SPEC-v3.txt. Verification: bash test-v3.sh"
}
1 change: 1 addition & 0 deletions problems/miniscoreboard/test-v1.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,4 @@ fi

echo "All V1 tests passed."
exit 0

2 changes: 1 addition & 1 deletion problems/miniscoreboard/test-v2.sh
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,4 @@ if [[ "$TEAM_OUT" != *"TS"* ]] || [[ "$TEAM_OUT" == *"Galatasaray"* ]]; then
fi

echo "All V1 and V2 tests passed."
exit 0
exit 0
82 changes: 82 additions & 0 deletions problems/miniscoreboard/test-v3.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
#!/usr/bin/env bash
# V3 Test Suite for miniscoreboard

BINARY="./miniscore.py"
DB_DIR=".miniscore"
DB_FILE="$DB_DIR/matches.dat"

# Helper for cleanup
cleanup() {
rm -rf "$DB_DIR"
}

# Helper for reporting
pass_count=0
fail_count=0

assert_output() {
local cmd=$1
local expected=$2
local name=$3

output=$($cmd 2>&1)
if [[ "$output" == *"$expected"* ]]; then
echo "PASS: $name"
pass_count=$((pass_count + 1))
else
echo "FAIL: $name"
echo " Expected: $expected"
echo " Actual: $output"
fail_count=$((fail_count + 1))
fi
}

cleanup

echo "--- Testing V1 Core ---"
assert_output "python3 $BINARY init" "Created .miniscore/ directory" "Init success"
assert_output "python3 $BINARY init" "Already initialized" "Init idempotency"

echo "--- Testing V1 add-match ---"
python3 $BINARY add-match SuperLig W1 2026-04-01 GS 2 FB 1 > /dev/null
python3 $BINARY add-match SuperLig W1 2026-04-01 BJK 0 TS 0 > /dev/null
assert_output "python3 $BINARY add-match SuperLig W2 2026-04-08 FB 1 BJK 0" "Added match #3" "Add match #3"

echo "--- Testing V2 history & team ---"
assert_output "python3 $BINARY history" "[1] GS 2 - 1 FB" "History check"
assert_output "python3 $BINARY team FB" "GS 2 - 1 FB" "Team filter GS"
assert_output "python3 $BINARY team FB" "FB 1 - 0 BJK" "Team filter FB"

echo "--- Testing V3 leaderboard ---"
# GS: 1W (3pts), 2GF, 1GA, +1
# FB: 1W, 1L (3pts), 2GF, 2GA, 0
# TS: 1D (1pts), 0GF, 0GA, 0
# BJK: 1D, 1L (1pts), 0GF, 1GA, -1
assert_output "python3 $BINARY leaderboard" "[1] GS: 3 pts" "Leaderboard Rank 1"
assert_output "python3 $BINARY leaderboard" "[2] FB: 3 pts" "Leaderboard Rank 2"

echo "--- Testing V3 summary ---"
assert_output "python3 $BINARY summary" "Total Teams: 4" "Summary Teams"
assert_output "python3 $BINARY summary" "Total Matches: 3" "Summary Matches"
assert_output "python3 $BINARY summary" "Total Goals: 4" "Summary Goals"

echo "--- Testing V3 export ---"
assert_output "python3 $BINARY export" "Exported 3 matches" "Export command"
if [ -f "$DB_DIR/matches.json" ]; then
echo "PASS: Export file exists"
pass_count=$((pass_count + 1))
else
echo "FAIL: Export file missing"
fail_count=$((fail_count + 1))
fi

echo ""
echo "PASSED: $pass_count"
echo "FAILED: $fail_count"
echo "TOTAL: $((pass_count + fail_count))"

if [ $fail_count -eq 0 ]; then
exit 0
else
exit 1
fi
32 changes: 32 additions & 0 deletions problems/miniscoreboard/walkthrough.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Walkthrough - V3 Implementation Details

This document provides a technical overview of the changes introduced in the V3 upgrade of `miniscoreboard`.

## Technical Achievements

### 1. Advanced Leaderboard Logic
The core challenge of V3 was the `leaderboard` command. It requires a stable sorting mechanism to handle league standings correctly.
- **Sorting Priority:** The implementation uses a descending sort on Points, followed by Goal Difference (GD), then Goals For (GF).
- **Tie-Breaking:** To ensure determinism (a key project requirement), if all stats are equal, the system falls back to an alphabetical sort by Team Name.
- **Implementation:** Leveraged Python's `sorted()` with a multi-key lambda function for efficiency.

### 2. Statistical Analytics (Summary)
The `summary` command provides a snapshot of the league's health.
- It dynamically calculates the number of unique teams and total goals.
- It formats the average goals per match to exactly **2 decimal places**, ensuring compliance with the `SPEC-v3.txt` requirements.

### 3. Data Portability (Export)
The `export` feature bridge the gap between human-readable `.dat` files and machine-readable `.json` formats.
- All match data is serialized into a structured JSON array.
- The output is stored in `.miniscore/matches.json`, facilitating easy import into web dashboards or spreadsheets.

## Verification & Testing
The implementation was rigorously tested using the `test-v3.sh` suite.
- **Test Results:** 7/7 Tests Passed.
- **Edge Cases Handled:**
- Requesting a leaderboard with an empty history prints a user-friendly error.
- Handling ties in the leaderboard shows consistent ranking orders.
- JSON export overwrites previous exports to keep data fresh.

## Project Context
Following the **Vibe Coding** philosophy, the architecture focuses on system-level reliability and clean documentation, allowing AI agents and human contributors to understand the codebase instantly.
Loading