PromptPilot

PromptPilot is a prompt intelligence profile for people who work with AI every day.

Instead of generating another generic prompt template, PromptPilot reads your real AI conversations and shows what your prompting behavior says about you: how clearly you ask, where you leave context out, how specific your constraints are, and what to change in the next prompt.

Live Application

PromptPilot v1 is live on Vercel:

• Web app: https://promptpilot-web.vercel.app/
• Production API: https://promptpilot-api.vercel.app

Open the app, paste or upload a prompt session, and run Judge My Prompts to generate a prompt intelligence report.

Why PromptPilot Exists

AI tools are getting better, but most users still treat prompting as guesswork. The problem is not only "how do I write one better prompt?" The deeper problem is that people do not get feedback on their prompting habits.

PromptPilot solves that by turning prompt history into a mirror:

• It finds patterns in how you ask for help.
• It scores the clarity, context, constraints, evidence, and actionability of your prompts.
• It points to evidence from your own imported sessions instead of giving vague advice.
• It gives you a next-prompt recipe you can use immediately.

The result is less prompt theater and more prompt self-awareness.

What It Does

• Imports sessions from Codex, Claude, ChatGPT, Cursor, Gemini, Windsurf, Markdown, JSON, pasted text, or plain text files.
• Redacts obvious secrets before sessions become analysis material.
• Uses Gemini-backed judgment in production, with deterministic local scoring as a development fallback.
• Generates style scores, behavior patterns, evidence excerpts, recommendations, comparisons, and a next-prompt recipe.
• Keeps import and report history so you can compare how your prompting style evolves.
• Exposes profile controls for insights, profile Q&A, correction, export, and reset flows.

Product Surface

PromptPilot v1 focuses on three screens:

• Judge: paste or upload a session, choose the source platform, and run the prompt intelligence analysis.
• History: add, preview, reprocess, and delete imported sessions; review saved prompts.
• Settings: inspect profile insights, ask profile questions, refresh/export/reset profile data, and correct or hide inferred observations.

Legacy prompt-generation routes remain in the codebase for regression coverage, but the active product experience is the prompt intelligence workflow.

How To Use The Live App

1. Open promptpilot-web.vercel.app.
2. Go to Judge.
3. Paste a prompt session or upload a .txt, .md, .markdown, or .json file.
4. Select the platform and source type.
5. Click Judge My Prompts.
6. Review the headline, style scores, evidence excerpts, behavior patterns, recommendations, and next-prompt recipe.
7. Use the report history on Judge, the session ledger in History, and the profile controls in Settings to keep exploring the analysis.

Use sample or non-sensitive sessions when testing the public deployment.

Tech Stack

• apps/web: Next.js 16, React 19, TypeScript, Tailwind CSS, Base UI, and lucide-react.
• apps/api: FastAPI, SQLAlchemy, Pydantic, pgvector, Google Gen AI SDK, and Uvicorn.
• Production hosting: Vercel web and API projects.
• Production database: Neon Postgres with pgvector.
• Local infrastructure: Docker Compose Postgres with pgvector.
• Reports and verification history: execution-reports/.

Architecture

PromptPilot is a monorepo with a Next.js frontend and FastAPI backend.

apps/web  ->  Next.js application
apps/api  ->  FastAPI service
packages/shared  ->  shared TypeScript package
infra/docker-compose.yml  ->  local Postgres + pgvector
execution-reports  ->  build notes, deployment notes, and QA reports

The main flow is:

1. The user imports or pastes a prompt session.
2. The API normalizes messages and redacts obvious secrets.
3. Prompt intelligence analysis runs through Gemini when configured.
4. A deterministic fallback can score locally during development.
5. Reports are stored with evidence, scores, recommendations, and metadata.
6. The frontend renders the current judgment, import ledger, and report history.

Local Development

Install JavaScript dependencies:

pnpm.cmd install

Install or sync Python dependencies:

uv --directory apps/api sync

Start local Postgres:

docker compose -f infra/docker-compose.yml up -d

Create local environment files from the examples:

Copy-Item .env.example .env
Copy-Item apps/api/.env.example apps/api/.env
Copy-Item apps/web/.env.example apps/web/.env.local

Set the server-only Gemini API key in apps/api/.env or the repo root .env:

APP_ENV=development
DATABASE_URL=postgresql://prompt_engine:prompt_engine@localhost:5432/prompt_engine
LLM_PROVIDER=gemini
GEMINI_API_KEY=...
DEFAULT_MODEL=gemini-3.1-flash-live-preview
GEMINI_MAX_OUTPUT_TOKENS=3200
GEMINI_THINKING_LEVEL=low
GEMINI_TIMEOUT_SECONDS=30
ALLOWED_ORIGINS=http://localhost:3000,http://127.0.0.1:3000

Start the API:

pnpm.cmd run dev:api

Start the web app in another terminal:

pnpm.cmd run dev:web

Then open http://localhost:3000.

Verification

Run the core checks before shipping changes:

uv --directory apps/api run python -m compileall app
uv --directory apps/api run python ../../evals/promptfoo/phase14_regression.py
pnpm.cmd --dir apps/web lint
pnpm.cmd --dir apps/web build

For production API regression checks:

uv --directory apps/api run python ../../evals/promptfoo/phase14_regression.py --base-url https://promptpilot-api.vercel.app

Version 1 Scope

This is PromptPilot v1: the first live version of the product.

The v1 goal is to prove that imported prompt sessions can become useful behavioral feedback. It is intentionally focused on judging and explaining prompting habits, not on becoming a full prompt library, team analytics suite, or agent orchestration platform yet.

Future Scope

PromptPilot can grow in several directions:

• Personal prompt memory that improves recommendations across more sessions.
• Team and organization-level prompt intelligence dashboards.
• MCP and agent integrations that turn a user's profile into live instructions for tools like Codex, Cursor, Claude, and ChatGPT.
• Richer privacy controls, including selective redaction, encrypted storage, and bring-your-own-database options.
• Browser extension and desktop importers for smoother session capture.
• Prompt improvement experiments that track whether suggested changes actually improve downstream AI results.
• Shareable report snapshots for coaching, hiring, education, and AI workflow reviews.

Author

Built by Sameer Nagar.

License

PromptPilot is released under the MIT License.