Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
REPO_EVALUATOR_HOST=0.0.0.0
REPO_EVALUATOR_PORT=8000
REPO_EVALUATOR_LOG_LEVEL=info
52 changes: 52 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: CI

on:
push:
branches: [main, feature/*]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.12", "3.13"]

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"

- name: Run tests with coverage
run: pytest

- name: Run linting
run: ruff check .

docker-build:
runs-on: ubuntu-latest
needs: test

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build Docker image
uses: docker/build-push-action@v5
with:
context: .
push: false
tags: repo-evaluator:latest
29 changes: 29 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
FROM python:3.12-slim AS builder

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends git && rm -rf /var/lib/apt/lists/*

COPY pyproject.toml ./
COPY README.md ./
COPY src/ ./src/

RUN pip install --no-cache-dir -e "."

FROM python:3.12-slim

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends git docker.io && rm -rf /var/lib/apt/lists/*

COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY src/ ./src/
COPY pyproject.toml ./
COPY README.md ./

RUN pip install --no-cache-dir -e "."

EXPOSE 8000

CMD ["uvicorn", "repo_evaluator.main:app", "--host", "0.0.0.0", "--port", "8000"]
120 changes: 110 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,19 @@

Evaluate Python repositories for health, structure, and Docker readiness.

## Features

- **Repository Analysis**: Clone and inspect any public GitHub repository.
- **Health Scoring**: Automated scoring based on README, Dockerfile, docker-compose, dependency files, tests, Python files, collected tests, and Docker build success.
- **Docker Validation**: Detects `Dockerfile` presence and attempts a build to verify it works.
- **CLI Tool**: Run evaluations from the terminal with JSON or Markdown output.
- **REST API**: FastAPI-based HTTP API for programmatic access.
- **CI/CD Ready**: GitHub Actions workflow included for automated testing and Docker validation.

## Install

Requires Python 3.12+.

```bash
pip install -e ".[dev]"
```
Expand All @@ -13,27 +24,116 @@ pip install -e ".[dev]"
### CLI

```bash
# Evaluate a repo and output JSON (default)
repo-eval evaluate https://github.com/example/repo --output json

# Evaluate a repo and output Markdown
repo-eval evaluate https://github.com/example/repo --output markdown
```

### API

```bash
uvicorn repo_evaluator.main:app --reload
# Start the server
uvicorn repo_evaluator.main:app --reload --host 0.0.0.0 --port 8000
```

Then POST to `/evaluate` with `{"repo_url": "https://github.com/example/repo"}`.
Then POST to `http://localhost:8000/api/v1/evaluate` with:

```json
{
"repo_url": "https://github.com/example/repo"
}
```

Health check: `GET http://localhost:8000/health` (also available at `/api/v1/health`)

### Docker

```bash
# Build and run with Docker Compose
docker-compose up --build

# Or build manually
docker build -t repo-evaluator .
docker run -p 8000:8000 repo-evaluator
```

## Scoring Rubric

| Category | Weight |
|----------|--------|
| README | 15 |
| Dockerfile | 15 |
| Tests | 20 |
| Python files | 10 |
| Docker build | 20 |
| Structure | 20 |
| Category | Weight | Criteria |
|----------|--------|----------|
| README | 15 | `README.md` (or `.rst`, `.txt`) present |
| Dockerfile | 15 | `Dockerfile` present |
| docker-compose | 10 | `docker-compose.yml` (or `.yaml`) present |
| Dependency file | 10 | `requirements.txt`, `pyproject.toml`, `setup.py`, or `setup.cfg` present |
| Tests directory | 10 | `tests/` or `test/` directory present |
| Python files | 10 | At least one `.py` file in the repo |
| Collected tests | 15 | `pytest --collect-only` finds > 0 tests |
| Docker build | 15 | `docker build` succeeds for the repo |

**Max score: 100**

## Development

```bash
# Run tests with coverage
pytest

# Run linting
ruff check .

# Run type checking
mypy src/repo_evaluator
```

## Environment Variables

Copy `.env.example` to `.env` and adjust as needed:

```bash
cp .env.example .env
```

| Variable | Description | Default |
|----------|-------------|---------|
| `REPO_EVALUATOR_HOST` | API bind host | `0.0.0.0` |
| `REPO_EVALUATOR_PORT` | API bind port | `8000` |
| `REPO_EVALUATOR_LOG_LEVEL` | Logging level | `info` |

## CI/CD

This project includes a GitHub Actions workflow (`.github/workflows/ci.yml`) that:

1. Runs the test suite with pytest and enforces 70% coverage.
2. Validates the Docker image builds successfully.
3. Runs linting with ruff.

## Project Structure

```
python-repo-evaluator/
├── src/repo_evaluator/ # Source code
│ ├── __init__.py
│ ├── analyzer.py # Repository analysis
│ ├── api.py # FastAPI routes
│ ├── cli.py # Typer CLI
│ ├── docker_validator.py # Docker build validation
│ ├── formatters.py # JSON / Markdown output
│ ├── main.py # FastAPI app factory
│ ├── models.py # Pydantic models
│ └── scorer.py # Health scoring logic
├── tests/ # Test suite (pytest)
├── scripts/
│ └── run-local.sh # Local API startup helper
├── .github/workflows/
│ └── ci.yml # GitHub Actions CI
├── Dockerfile # Multi-stage Python 3.12 build
├── docker-compose.yml # Local orchestration
├── .env.example # Environment template
├── pyproject.toml # Project config & dependencies
└── README.md # This file
```

## License

Expand Down
10 changes: 10 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
services:
repo-evaluator:
build: .
ports:
- "8000:8000"
env_file:
- .env
volumes:
- /var/run/docker.sock:/var/run/docker.sock
restart: unless-stopped
31 changes: 31 additions & 0 deletions scripts/run-local.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env bash
set -euo pipefail

# Local development startup script for python-repo-evaluator API

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
VENV_PATH="${PROJECT_ROOT}/.venv"

if [ ! -d "$VENV_PATH" ]; then
echo "Virtual environment not found at $VENV_PATH"
echo "Run: python -m venv .venv && source .venv/bin/activate && pip install -e \".[dev]\""
exit 1
fi

source "$VENV_PATH/bin/activate"

HOST="${REPO_EVALUATOR_HOST:-0.0.0.0}"
PORT="${REPO_EVALUATOR_PORT:-8000}"
LOG_LEVEL="${REPO_EVALUATOR_LOG_LEVEL:-info}"

echo "Starting python-repo-evaluator API on http://${HOST}:${PORT}"
echo "Health check: http://${HOST}:${PORT}/api/v1/health"
echo "Evaluate endpoint: POST http://${HOST}:${PORT}/api/v1/evaluate"
echo "Press Ctrl+C to stop"

exec uvicorn repo_evaluator.main:app \
--host "$HOST" \
--port "$PORT" \
--log-level "$LOG_LEVEL" \
--reload
35 changes: 32 additions & 3 deletions src/repo_evaluator/analyzer.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
from __future__ import annotations

import shutil
import subprocess
import tempfile
from pathlib import Path
from typing import Any

from git import Repo

Expand All @@ -25,6 +25,7 @@ def analyze(
key_files = self._detect_key_files(temp_dir)
python_files = self._count_python_files(temp_dir)
tests = self._detect_tests(temp_dir)
collected_tests = self._collect_tests(temp_dir)
docker = DockerResult(present=key_files["Dockerfile"])
if docker_validator and docker.present:
docker = docker_validator.validate(temp_dir)
Expand All @@ -35,6 +36,7 @@ def analyze(
key_files=key_files,
python_files=python_files,
tests=tests,
collected_tests=collected_tests,
docker=docker,
)
finally:
Expand All @@ -43,7 +45,7 @@ def analyze(
def _clone(self, repo_url: str) -> Path:
"""Clone repository into a temporary directory and return its path."""
temp_dir = Path(tempfile.mkdtemp(prefix="repo_evaluator_"))
Repo.clone_from(repo_url, temp_dir)
Repo.clone_from(repo_url, temp_dir, depth=1)
return temp_dir

def _detect_language(self, repo_path: Path) -> str | None:
Expand Down Expand Up @@ -76,8 +78,27 @@ def _detect_tests(self, repo_path: Path) -> TestResult:
return TestResult(framework="pytest", count=count)
return TestResult()

def _collect_tests(self, repo_path: Path) -> int:
"""Attempt to collect tests with pytest --collect-only."""
try:
result = subprocess.run(
["python", "-m", "pytest", "--collect-only", "-q"],
cwd=repo_path,
capture_output=True,
text=True,
timeout=30,
)
if result.returncode != 0:
return 0
# pytest -q outputs one line per collected test/module; count non-empty lines
lines = [line for line in result.stdout.splitlines() if line.strip()]
# Filter out summary lines like "no tests ran"
return len([line for line in lines if "test" in line.lower() or ".py" in line])
except Exception:
return 0

def _detect_key_files(self, repo_path: Path) -> dict[str, bool]:
"""Check presence of README, Dockerfile, and docker-compose."""
"""Check presence of key project files and directories."""
return {
"README": any(
(repo_path / name).is_file()
Expand All @@ -88,4 +109,12 @@ def _detect_key_files(self, repo_path: Path) -> dict[str, bool]:
(repo_path / name).is_file()
for name in ("docker-compose.yml", "docker-compose.yaml")
),
"dependency_file": any(
(repo_path / name).is_file()
for name in ("requirements.txt", "pyproject.toml", "setup.py", "setup.cfg")
),
"tests_directory": any(
(repo_path / name).is_dir()
for name in ("tests", "test")
),
}
1 change: 0 additions & 1 deletion src/repo_evaluator/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
from repo_evaluator.models import RepoReport
from repo_evaluator.scorer import HealthScorer


router = APIRouter()


Expand Down
5 changes: 1 addition & 4 deletions src/repo_evaluator/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,11 @@

from __future__ import annotations

import sys

import typer

from repo_evaluator.analyzer import RepoAnalyzer
from repo_evaluator.docker_validator import DockerValidator
from repo_evaluator.formatters import to_json, to_markdown
from repo_evaluator.models import OutputFormat
from repo_evaluator.scorer import HealthScorer

app = typer.Typer(help="Evaluate Python repositories for health, structure, and Docker readiness.")
Expand Down Expand Up @@ -45,7 +42,7 @@ def evaluate(
typer.echo(to_markdown(report))
except Exception as exc:
typer.echo(f"Error: {exc}", err=True)
raise typer.Exit(code=1)
raise typer.Exit(code=1) from exc


if __name__ == "__main__":
Expand Down
Loading
Loading