Skip to content

BACKEND.md

Latest commit

 

History

History
189 lines (141 loc) · 4.6 KB

BACKEND.md

File metadata and controls

189 lines (141 loc) · 4.6 KB
Raw

Backend Development Guide

Architecture Overview

The backend integrates with the notebooks' rag_utils module for a unified implementation:

File Structure

backend/
├── main.py                    # FastAPI app initialization + startup hooks
├── api/
│   └── routes.py             # API endpoint definitions
├── rag/
│   ├── neural_rag.py         # Neural RAG (uses rag_utils)
│   └── internet_rag.py       # Internet RAG (uses rag_utils)
├── services/                 # Legacy services (optional, can be deprecated)
├── utils/
│   ├── logger.py             # Logging + .env loader
│   └── metrics.py            # Performance timing utilities
└── __init__.py

How It Works

  1. Shared RAG Utils: Both notebooks and backend import from notebooks/rag_utils.py
  2. Neural RAG: Uses FAISS vector index + SentenceTransformers embeddings
  3. Internet RAG: Uses Tavily Search API + BeautifulSoup web scraping
  4. LLM: Mistral-7B with 4-bit quantization (when RAG_USE_REAL_MODELS=1)

API Endpoints

GET  /              - Health check + system info
POST /api/rag/neural      - Query neural RAG
POST /api/rag/internet    - Query internet RAG
POST /api/rag/compare     - Compare both methods
GET  /docs          - Interactive API documentation

Request Format

{
  "query": "What is machine learning?",
  "k": 3,
  "answer_len": 50
}

Response Format

{
  "method": "neural",
  "query": "What is machine learning?",
  "answer": "Machine learning is a subset of artificial intelligence...",
  "contexts": ["passage1", "passage2", "passage3"],
  "metrics": {
    "retrieval_time": 0.123,
    "llm_time": 0.456,
    "context_size": 1200.0
  }
}

Running the Backend

Development Mode

# Install full dev dependencies
pip install -r requirements-dev.txt

# Run with auto-reload
python -m uvicorn backend.main:app --reload

Visit: http://localhost:8000/docs

Production Mode (Docker)

# Install minimal backend dependencies
pip install -r requirements.txt

# Build image
docker build -f docker/Dockerfile -t cloud-rag-api .

# Run container
docker run -p 8000:8000 \
  -e RAG_USE_REAL_MODELS=0 \
  -e TAVILY_API_KEY=your_key_here \
  cloud-rag-api

Environment Variables

Create .env from .env_example:

# Enable real SentenceTransformers/Transformers (slower, more accurate)
RAG_USE_REAL_MODELS=0

# Enable real MS MARCO dataset (large, ~2GB download)
RAG_USE_REAL_DATASET=0

# Tavily Search API key (for internet RAG)
TAVILY_API_KEY=your_tavily_api_key

# Hugging Face token (optional, for gated models)
HF_TOKEN=your_hf_token

# Logging level
LOG_LEVEL=INFO

Key Integration Points

Initialization Flow

  1. main.py: Loads .env at startup, initializes logger
  2. routes.py: Creates cached instances of NeuralRag/InternetRag on first request
  3. neural_rag.py: Calls build_neural_rag_runtime() from rag_utils once
  4. rag_utils.py: Loads dataset, embedder, FAISS index based on env flags

Lazy Loading

RAG instances are cached using @lru_cache to avoid reinitializing expensive resources:

  • FAISS index stays in memory across requests
  • SentenceTransformers model cached
  • MS MARCO dataset cached (if enabled)

Testing the API

Quick Health Check

curl http://localhost:8000/

Query Neural RAG

curl -X POST http://localhost:8000/api/rag/neural \
  -H "Content-Type: application/json" \
  -d '{"query": "What is FAISS?"}'

Query Internet RAG

curl -X POST http://localhost:8000/api/rag/internet \
  -H "Content-Type: application/json" \
  -d '{"query": "Latest AI news"}'

Compare Both Methods

curl -X POST http://localhost:8000/api/rag/compare \
  -H "Content-Type: application/json" \
  -d '{"query": "Explain machine learning"}'

Troubleshooting

FAISS Not Available

  • If using fallback embedder, FAISS is optional
  • For real models: pip install faiss-cpu

Out of Memory

  • Reduce dataset size: num_samples=20 in build_neural_rag_runtime()
  • Disable 4-bit quantization: set RAG_USE_REAL_MODELS=0

Tavily API Errors

  • Check TAVILY_API_KEY in .env
  • Internet RAG falls back to mock results if API fails

Import Errors

  • Ensure notebooks/rag_utils.py exists
  • Verify notebooks/ is added to sys.path in main.py

Future Enhancements

  • Add caching decorator for query results
  • Implement request rate limiting
  • Add authentication (API key or OAuth)
  • Create dashboard for performance comparison
  • Support for custom vector indexes (Pinecone, Weaviate, etc.)