VibeCoding Quality Gate

Instant automated quality analysis for vibe-coded apps — powered by TestSprite MCP.

Submit any live web app URL and get back a quality score (0–100 / A–F letter grade), a detailed markdown test report, and per-test replay video links — all within minutes.

Built for the TestSprite Hackathon using real TestSprite MCP integration.

---

What It Does

1. Paste a URL — enter your live app's URL and an optional GitHub repo link
2. Pick a Focus Area — Full App, Core Features, Auth, UI/UX, Forms, or Navigation
3. Watch it run — real-time progress tracker shows every stage live via WebSocket
4. Get your report — quality score, letter grade, pass/fail breakdown, and replay links for every test

---

How the Pipeline Works

Your App URL
     │
     ▼
① CRAWL ── Static fetch → Headless browser → Heuristic fallback
     │        Detects: app type, login flows, buttons, routes
     │
     ▼
② CLASSIFY ── Scores against 7 app archetypes
     │         (dashboard-analytics, saas-tool, ecommerce,
     │          resume-builder, ai-chat, blog-cms, booking-scheduler)
     │
     ▼
③ CONFIGURE ── Auto-generates TestSprite config.json
     │           code_summary.yaml, requirements.md
     │           Temp email identity (for auth testing)
     │
     ▼
④ TEST ── TestSprite MCP spawned as child process (JSON-RPC over stdio)
     │     generate_standardized_prd → generate_frontend_test_plan
     │     → generate_code_and_execute → parse results
     │
     ▼
Quality Score + Markdown Report + Replay Video Links

---

Features

Smart Site Analysis

• 3-layer crawl: plain fetch → headless Puppeteer → URL/title heuristic fallback
• Detects app type, login presence, interactive elements, routes, and user flows
• Generates a code_summary.yaml and smart instruction automatically fed to TestSprite
• Live preview endpoint: GET /api/gate/analyze-preview?url=<your-url>

Temp Email Identity (for Auth Testing)

• Auto-generates a disposable email address (vibegate-{id}@guerrillamail) before testing starts
• Live email monitor panel watches for verification emails in real time
• Verification links are auto-clicked so TestSprite can complete signup flows unassisted

Real-Time Progress

• WebSocket (Socket.IO) streams every stage live to the browser: Creating email → Crawling → Classifying → Configuring → Testing → Scoring
• No page refresh needed — progress updates appear as they happen

Quality Report

• Score 0–100 with A/B/C/D/F letter grade
• Per-test pass/fail status with replay video links
• Full markdown report rendered inline
• "Watch Replay" button on every test that has a recorded video

URL Pre-fill

• Append ?prefill=https://yourapp.com to the gate URL to auto-populate the form — useful for sharing links or embedding in CI pipelines

---

Tech Stack

Layer	Technology
Frontend	React 18, Vite, Tailwind CSS, shadcn/ui
Routing	Wouter
Data Fetching	TanStack Query v5
Real-time	Socket.IO (client + server)
Backend	Express.js (Node.js)
Database	PostgreSQL via Drizzle ORM
Crawling	Cheerio (static), Puppeteer (headless)
Testing Engine	TestSprite MCP (@testsprite/testsprite-mcp)
Temp Email	GuerrillaMail API

---

Project Structure

├── client/
│   └── src/
│       ├── pages/
│       │   ├── Home.tsx           # Landing page + submission form
│       │   └── Report.tsx         # Report display page
│       └── components/
│           ├── RunForm.tsx         # URL input, focus area, credentials
│           ├── ProgressTracker.tsx # Live WebSocket progress stages
│           ├── QualityScoreBadge.tsx # Animated score + letter grade
│           ├── LiveEmailMonitor.tsx  # Temp email live inbox
│           └── MarkdownRenderer.tsx  # Styled markdown report
├── server/
│   ├── routes.ts                  # All API endpoints
│   ├── storage.ts                 # DB interface (Drizzle)
│   └── services/
│       ├── mcpRunner.ts           # TestSprite MCP orchestration
│       ├── siteAnalyzer.ts        # Pre-flight crawl + classification
│       ├── scoreCalculator.ts     # Score parsing from test results
│       ├── tempEmailService.ts    # GuerrillaMail disposable email
│       └── repoManager.ts         # Optional GitHub repo cloning
└── shared/
    ├── schema.ts                  # Drizzle schema + Zod types
    └── routes.ts                  # Shared API route constants

---

API Endpoints

Method	Path	Description
POST	/api/gate/run	Start a quality gate run
GET	/api/gate/run/:id	Get run status + results
GET	/api/gate/analyze-preview?url=	Crawl preview (app type, hasLogin, buttons)
POST	/api/gate/temp-email	Create a disposable email session
GET	/api/gate/temp-email/:id	Check inbox for new emails
DELETE	/api/gate/temp-email/:id	Clean up email session

---

Environment Variables

Variable	Required	Description
TESTSPRITE_API_KEY	Yes	TestSprite API key for MCP authentication
DATABASE_URL	Yes	PostgreSQL connection string
SESSION_SECRET	Yes	Express session signing secret

---

Running Locally

# Install dependencies
npm install

# Start the dev server (Express + Vite on the same port)
npm run dev

The app will be available at http://localhost:5000.

---

Heuristic Fallback Logic

When headless crawling fails (e.g., the target site blocks bots), the analyzer falls back to URL + title heuristics:

• App type scored against 7 signature keyword sets
• Login detection: explicit auth keywords plus type-based inference — dashboard-analytics and saas-tool apps default to hasLogin: true because they almost universally require authentication
• Inferred buttons and flows generated per app type to seed the TestSprite test plan

---

Hackathon Context

This project was built for the TestSprite Hackathon. The goal: make TestSprite accessible to vibe-coders who want a one-click quality check on their deployed app — no test writing, no config, just a URL.

---

License

MIT