A modular backend for document ingestion, semantic search, and Retrieval-Augmented Generation built with FastAPI, FAISS, and OpenAI-compatible models.
This project implements the core architecture behind modern AI knowledge assistants:
documents are indexed into a vector store, relevant context is retrieved for each user question, and the final answer is generated by an LLM using that context.
Large Language Models are powerful, but they do not automatically know about private, internal, or domain-specific data.
This backend solves that problem using a RAG pipeline:
- Documents are cleaned and split into chunks.
- Each chunk is converted into an embedding.
- Embeddings are stored in a FAISS vector index.
- User questions are embedded and matched against stored knowledge.
- Relevant chunks are sent to an LLM as context.
- The LLM generates a grounded answer based on retrieved information.
The goal is to build a clean, extensible AI backend that can serve as the foundation for real-world document question-answering systems.
- Full document indexing pipeline
- Retrieval-Augmented Generation answer pipeline
- FAISS-based semantic vector search
- OpenAI-compatible embedding and chat model integration
- Clean service-oriented backend architecture
- Shared AI client for embedding and LLM services
- Transparent responses with retrieved context included
- Designed for extension into production AI applications
The indexing pipeline receives raw text and automatically:
- cleans and preprocesses the document
- splits it into configurable chunks
- generates embeddings for each chunk
- stores vectors and metadata in FAISS
The retrieval layer:
- embeds the user question
- searches the vector store
- returns the most relevant chunks
- includes metadata and distance scores for debugging and transparency
The answer pipeline:
- retrieves top-k relevant chunks
- builds a context-aware prompt
- calls the LLM
- returns a grounded answer along with supporting contexts
POST /documents/index{
"text": "Refunds are available within 14 days of purchase.",
"metadata": {
"source": "policy.txt"
},
"chunk_size": 500,
"chunk_overlap": 50
}{
"message": "Document indexed successfully",
"total_chunks": 1,
"total_vectors": 1
}POST /rag/answer{
"question": "What is the refund policy?",
"top_k": 3
}{
"question": "What is the refund policy?",
"answer": "According to the indexed document, refunds are available within 14 days of purchase.",
"contexts": [
{
"text": "Refunds are available within 14 days of purchase.",
"distance": 0.118,
"metadata": {
"source": "policy.txt"
}
}
]
}Document
↓
Clean & Chunk
↓
Generate Embeddings
↓
Store in FAISS
↓
Searchable Knowledge Base
User Question
↓
Generate Question Embedding
↓
Retrieve Relevant Chunks
↓
Build Context-Aware Prompt
↓
Generate Grounded Answer
Handles text preprocessing and chunking.
Generates embeddings for document chunks and user questions.
Manages FAISS indexing, vector persistence, metadata storage, and similarity search.
Coordinates the full document indexing workflow.
Handles answer generation using the configured chat model.
Coordinates the full retrieval-augmented answer generation pipeline.
The system is split into focused services instead of placing all logic inside API routes. This makes the codebase easier to maintain, test, and extend.
Embedding and generation services use a centralized AI client. This avoids duplicated provider setup and makes it easier to switch models or providers later.
The LLM receives retrieved document chunks as context, reducing hallucination and making answers more reliable.
The API returns retrieved contexts with the final answer. This helps with debugging, validation, source display, and future citation support.
- Python
- FastAPI
- Pydantic
- FAISS
- OpenAI-compatible API
- Uvicorn
EMBEDDING_MODEL=text-embedding-3-small
CHAT_MODEL=gpt-4o-miniModel names can be adjusted depending on the selected provider.
git clone https://github.com/ErfanMasoudiBA/rag-fastapi-vector-backend.git
cd rag-fastapi-vector-backend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reloadAPI documentation will be available at:
http://127.0.0.1:8000/docs
When changing the embedding model, the vector dimension may also change. If an existing FAISS index was created with a different dimension, you may see an error like:
Embedding dimension mismatch. Expected X, got Y
To fix this during development, remove the old vector store files and re-index documents:
data/vector.index
data/vector_metadata.json- Designed a complete RAG workflow from ingestion to answer generation
- Handled vector dimension consistency between embeddings and FAISS
- Separated AI provider logic from business services
- Built reusable orchestration layers for indexing and answering
- Returned retrieval context for better debugging and trust
- Created an extensible backend foundation instead of a simple chatbot wrapper
This backend can be adapted for:
- internal company knowledge assistants
- documentation Q&A systems
- support knowledge bases
- policy and contract assistants
- semantic search tools
- domain-specific AI copilots
- private data chat systems
Planned improvements:
- document IDs for update and delete operations
- metadata-based filtering
- retrieval score thresholding
- reranking retrieved chunks
- streaming LLM responses
- citation formatting
- multi-file ingestion
- evaluation pipeline for retrieval and answer quality
- authentication and rate limiting
This project demonstrates the implementation of a practical AI backend architecture using Retrieval-Augmented Generation.
It is more than a wrapper around an LLM API. It includes document processing, embedding generation, vector search, retrieval orchestration, and grounded answer generation.
A concise professional description:
Built a modular RAG backend with FastAPI, FAISS, and OpenAI-compatible models to enable document ingestion, semantic search, and grounded question answering over private knowledge sources.