A minimal local web application that lets students upload a course syllabus (PDF) and ask natural-language questions about it (e.g., exam dates, grading breakdown, policies). The app uses Retrieval-Augmented Generation (RAG) with the Gemini API to ensure answers are grounded strictly in the uploaded document.
- Upload a syllabus PDF
- Ask questions in plain English
- RAG pipeline (chunking + embeddings + vector search)
- Safety guardrails (system rules, injection checks, input limits)
- Telemetry logging (latency, tokens, RAG path)
- Offline evaluation with automated pass-rate reporting
User → Web UI → Flask API
→ PDF Parser → Chunking → Embeddings → Vector Store
→ Retriever → LLM → Answer
→ Telemetry Logger
- Backend: Python, Flask
- LLM: Gemini API
- RAG: FAISS
- Embeddings: Gemini embeddings
- PDF Parsing: pypdf
- Frontend: HTML, CSS, JS
- Python 3.9+
- Gemini API Key
Create a .env file in the root directory based on .env.example:
GEMINI_API_KEY=your_api_key_here
python run.pyThen open your browser at:
http://localhost:5000
- Test cases are stored in
tests.json - Run evaluation with:
python eval.pyThis prints a pass rate based on regex pattern matching.
- Strict system prompt: only answer from syllabus content
- Prompt injection detection
- Input length limits
- Graceful error handling
Each request logs:
- Timestamp
- Pathway (RAG)
- Latency
- Tokens / cost (if available)
syllabus-llm-app/
├── app.py
├── rag.py
├── llm.py
├── eval.py
├── logger.py
├── templates/
│ └── index.html
├── static/
│ ├── style.css
│ └── app.js
├── data/
│ ├── uploads/
│ └── vectors/
├── tests.json
├── sample_syllabus.pdf
├── requirements.txt
├── .env.example
└── README.md
- Single syllabus at a time
- Text-based answers only
- Accuracy depends on PDF text quality
Adrian Fudge