Aegis Knowledge Mesh

Enterprise RAG (Retrieval-Augmented Generation) knowledge base with a ChatGPT-like UI, document ingestion, pgvector search, and source-cited answers.

Frontend (UI) Features

• Shell layout: header + full-height sidebar + feature content
• Theme: Light / Dark / System modes
• AI Chat (/query) (ChatGPT-like)
  • Multiple conversations with history stored in Postgres
  • Chat list in sidebar with per-chat options (Rename, Delete)
  • Modes: Fast / Thinking / Deep research / Web only (per chat)
  • Sticky composer at the bottom and scrollable message history
  • Copy assistant responses; resend a previous user prompt
  • Citations per answer (toggle Sources) with a persistent "auto-open sources" preference
• Documents (/documents)
  • Upload PDF/TXT/Markdown/CSV/XLSX
  • Paste text ingestion
  • View document metadata, processing stage, and chunk progress
  • Reprocess failed documents
  • Delete documents (and associated chunks)
• Search (/search)
  • Single-shot query (no chat history)
  • Modes + citations

Backend Features

• Document ingestion: extract text -> chunk -> embed -> store in Postgres + pgvector
• Vector retrieval: similarity search with tenant + visibility filtering
• RAG answering: retrieval -> prompt building -> LLM answer -> citations
  • Fast/Thinking can answer general questions and clearly label grounding when no sources are available
  • Deep research can include optional web snippets (if web search is configured)
  • Web only uses only web snippets (no knowledge-base retrieval)
• Answer safety (basic): simple grounding/citation checks and guardrails
• Audit logging: tracks key actions like uploads and queries
• In-process background document queue with configurable worker count

Environment & Secrets

• Copy .env.example to .env at the repo root before running docker-compose up.
• Fill strong, unique values for POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD, DATABASE_URL, and SECRET_KEY.
• Docker Compose sources .env for both the database container and backend service; .env is ignored via .gitignore so secrets stay local.

Evaluation & Experiments

• Retrieval evaluation dataset: experiments/data/retrieval_eval.json
• Precision@K script: experiments/retrieval_metrics.py
• Embedding comparison notebook: experiments/embedding_comparison.ipynb
• Design tradeoffs: docs/design_decisions.md

Frontend Routes

• GET /query - chat UI
• GET /documents - upload + manage documents
• GET /search - single-shot query UI

Backend API (HTTP)

Health

• GET / - service info
• GET /health - health check
• GET /docs - Swagger UI

Auth

• POST /auth/register
• POST /auth/login
• POST /auth/refresh

Users

• GET /users/
• GET /users/{user_id}
• POST /users/
• PUT /users/{user_id}

Documents

• POST /documents/ - upload file (multipart)
• POST /documents/text - ingest pasted text (multipart form)
• GET /documents/ - list
• GET /documents/{document_id} - details (includes progress fields)
• POST /documents/{document_id}/reprocess - re-run ingestion
• DELETE /documents/{document_id} - delete (admin only)

Search / Query

• POST /query/ - single-shot RAG query (supports mode)

Chats

• GET /chats/ - list chats
• POST /chats/ - create chat (supports mode)
• GET /chats/{chat_id} - chat + messages
• PATCH /chats/{chat_id} - rename
• DELETE /chats/{chat_id} - delete
• POST /chats/{chat_id}/query - send a message (supports mode, optional agentic)

Admin

• GET /admin/audit-logs
• GET /admin/stats

Agents (Agentic RAG)

• POST /agents/query - agent planner-controller loop (supports mode, max_steps, include_steps)

Query Modes

• fast: fastest responses, smaller retrieval
• thinking: best quality on your knowledge base (default)
• deep_research: larger retrieval + optional web snippets (requires WEB_SEARCH_PROVIDER + WEB_SEARCH_API_KEY)
• web_only: web snippets only (requires web search config)

Agentic RAG

• Chat UI includes an Agentic toggle that runs a planner-controller loop with auditable steps (persisted in Postgres as agent_runs and agent_steps).

OpenAI (Paid) Model Split

When LLM_PROVIDER=openai, you can split models by role:

• LLM_MODEL - default chat model (used if specific overrides are not set)
• LLM_MODEL_FAST / LLM_MODEL_THINKING / LLM_MODEL_DEEP_RESEARCH - per-mode answer model overrides
• AGENT_PLANNER_MODEL - agent planner model (JSON action selection)
• AGENT_SUMMARY_MODEL - agent summarization model (optional)