feat: Core AI Pipeline, Vector Relationships, and E2E Dockerization

[Prakhar-Sethi012]

Overview

This PR introduces the core AI and semantic search engines for the WhereTF backend, fully containerized for a zero-configuration local developer environment. All dependencies, including the local LLM and embedding models, are fully offline and privacy-preserving.

Architecture Highlights

• AI Query Expansion (HyDE): Implemented an interception layer on the POST /search/ route. Integrates an offline qwen2:0.5b model via Ollama to hallucinate highly technical context, vastly improving vector search accuracy.
• Semantic File Mapping: Engineered a C-level pgvector math engine utilizing cosine distance (<=>) to automatically map and score relationships between uploaded documents.
• E2E Containerization: Built a multi-container Docker infrastructure (FastAPI, PostgreSQL + pgvector, Ollama).
  • Implemented layer caching in the Dockerfile to drop build times.
  • Configured internal Docker DNS routing to allow EasyOCR to fetch initial weights.
  • Resolved WSL2 memory limits by migrating to a highly optimized micro-model.

Bug Fixes

• Variable Scope Guarding: Fixed an UnboundLocalError in the indexing service where file_obj cleanup would crash the pipeline if EasyOCR failed or returned empty content.
• Cold Start Timeouts: Increased urllib timeouts to allow the local CPU to handle the initial LLM weight-loading without dropping the client connection.

Testing Steps

1. Run docker compose up -d --build
2. Upload two related text files via POST /upload/.
3. Check GET /files/{id}/related to verify the pgvector similarity score.
4. Execute POST /search/ and verify the expanded_query field is populated by the local LLM.