This project is based on the paper "Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks" and extends it into a practical comparison demo.
The system compares two ways of answering questions from uploaded documents:
- RAG: retrieve the most relevant chunks, then answer from those chunks.
- CAG: preload the full limited knowledge base, then answer without real-time retrieval.
- Adaptive: automatically choose RAG or CAG based on knowledge-base size and query complexity.
RAG is popular, but it adds retrieval latency and can fail when the wrong chunks are selected. CAG is simpler for small, stable document collections because the model can work from the full available context. This project demonstrates when each approach is better.
- Upload TXT, MD, or PDF files.
- Run RAG, CAG, Adaptive, or side-by-side Compare mode.
- View answer latency, confidence score, routing reason, and source snippets.
- Includes a sample knowledge base for instant demo.
Open index.html in a browser. This version supports pasted text or uploaded TXT/MD/CSV files.
pip install -r requirements.txt
streamlit run app.pyThen open the local URL shown by Streamlit.
python -m unittest discover tests- When should CAG be preferred over RAG?
- What are the main failure cases of RAG?
- Compare RAG and CAG for small knowledge bases.
- Which metrics should be used to evaluate this system?
Use 20-30 questions over the same document collection and record:
- answer relevance
- faithfulness to source
- latency
- retrieval quality
- whether the adaptive route selected the better method
- Replace lexical retrieval with embeddings and FAISS.
- Add a real LLM generation layer.
- Add automatic faithfulness scoring.
- Add query decomposition for complex questions.
- Cache long-context prompts for true CAG behavior with supported LLM APIs.