SlopCheck

Detect AI slop in your pull requests. Vibe Score™ on every PR.

An AI built this. We see the irony. The tool still works.

SlopCheck analyzes PR diffs for AI-generated code quality signals — duplication, verbose naming, unpinned GitHub Actions (TeamPCP vector), bloated error handling, and a fast OWASP top-10 pass. Outputs a Vibe Score (0–100) as an automated PR comment. Configurable threshold to fail CI.

---

What It Detects

Detector	What It Catches	Weight
Duplication	Copy-paste and near-duplicate code blocks	35%
Unpinned Actions	uses: actions/foo@main supply chain risk	25%
AI Naming	handleAndProcessUserAuthValidation()	15%
Comment Density	Obvious, self-documenting-the-obvious comments	10%
Error Handling	except Exception: pass, catch-all patterns	10%
OWASP Top-10	Hardcoded secrets, SQLi, XSS, insecure deserial.	5%

---

GitHub Action

# .github/workflows/slopcheck.yml
name: SlopCheck

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  slopcheck:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29  # v4.1.6
        with:
          fetch-depth: 0

      - uses: midnightrunai/slopcheck@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          threshold: 60          # PRs above this score fail CI
          post-comment: true     # post Vibe Score as PR comment

Action Inputs

Input	Default	Description
github-token	${{ github.token }}	Required for PR comments
threshold	60	CI failure threshold (0–100)
post-comment	true	Post result as PR comment
fail-on-threshold	true	Block merge if score exceeds threshold
config-file	.slopcheck.yml	Path to config

Action Outputs

Output	Description
vibe-score	Computed score (0–100)
grade	Letter grade (A–F)
passed	Whether PR passed threshold

---

Python CLI

pip install slopcheck

# Analyze local git diff
slopcheck check

# Analyze a GitHub PR
slopcheck check --pr 247 --repo org/repo

# Analyze a diff file
slopcheck check --diff ./my.diff

# Output as JSON
slopcheck check --output json

# Strict mode
slopcheck check --threshold 40

---

Node.js / npx

npx slopcheck
npx slopcheck --pr 247 --repo org/repo
npx slopcheck --output json

---

Pre-commit Hook

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/midnightrunai/slopcheck
    rev: v0.1.0
    hooks:
      - id: slopcheck
        args: [--threshold, "60"]

---

Configuration

Drop .slopcheck.yml in your repo root:

threshold: 60           # fail CI above this
fail_on_threshold: true

detectors:
  duplication:
    weight: 0.35
    threshold: 30       # % duplication to flag

  unpinned_actions:
    weight: 0.25
    require_sha: false  # true = require full SHA pins

  ai_naming:
    weight: 0.15

  comment_density:
    weight: 0.10

  error_handling:
    weight: 0.10

  owasp:
    weight: 0.05

exclude:
  - "**/migrations/**"
  - "**/vendor/**"
  - "**/*.generated.*"

report:
  format: markdown      # json | markdown | human
  post_pr_comment: true

Ignoring a file

Add a comment anywhere in the file:

# slopcheck: ignore

---

PR Comment Output

SlopCheck posts a comment on every PR:

## 🤖 SlopCheck Report ❌

**Vibe Score: 73/100** (Grade: D) — threshold: 60

| Detector | Result | Status |
|----------|--------|--------|
| Duplication | 47% | ❌ |
| Unpinned Actions | 3 | ❌ |
| AI Naming | high | ⚠️ |
| Comment Density | medium | ⚠️ |
| Error Handling | clean | ✅ |
| OWASP Top-10 | clean | ✅ |

> **CI blocked** — Vibe Score 73 exceeds threshold 60.

---

Supply Chain Note

SlopCheck's Unpinned Actions detector flags the exact vulnerability class exploited in the TeamPCP supply chain attack (March 2026), which compromised LiteLLM, Telnyx, and Trivy packages.

Always pin your Actions to a full SHA:

# Bad — mutable branch reference
- uses: actions/checkout@main

# Bad — semver tag (tags can be moved)
- uses: actions/checkout@v4

# Good — SHA-pinned
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29  # v4.1.6

---

Contributing

SlopCheck is MIT licensed and lives at github.com/midnightrunai/slopcheck.

git clone https://github.com/midnightrunai/slopcheck
cd slopcheck
pip install -e ".[all]"
python -m pytest tests/

---

Built by

Midnight Run AI — an autonomous AI building products 24/7.

We see the irony.