PROJECT: Enterprise AI Assistant Platform

A FastAPI backend that handles multiple AI capabilities: document Q&A, SQL queries, voice interaction, and multi-agent coordination.

Pre-requisites

Python 3.13 or higher
uv package manager
Ollama - Local LLM runtime
LangSmith account (optional, for observability)
Vision-capable model for multimodal features (optional):
- ollama pull llava or ollama pull llava-phi3
- Alternative: GPT-4V, Claude 3 (requires API keys)
Docker and Docker Compose (optional)
Make (optional, for convenient commands)

Features

A production-ready AI platform with:

📚 9 specialized AI agents
🔧 10+ tools and integrations
💬 Real-time chat with streaming
🖼️ Vision and multimodal support
🔌 MCP for external services
📊 Full observability with LangSmith
🤝 Human-in-the-loop workflows
🌐 WebSocket real-time features

Feature Matrix

Feature	Status	Phase	Files
RAG	✅	1	`rag_agent.py`
SQL Agent	✅	2	`sql_agent.py`
Custom RAG	✅	3	`custom_rag_agent.py`
Web Search	✅	4	`tools/web_search.py`
Calculator	✅	4	`tools/calculator.py`
Wikipedia	✅	4	`tools/wikipedia.py`
Memory	✅	4	`tool_agent.py`
Multi-Agent	✅	5	`multi_agent_router.py`
Streaming	✅	5	`streaming_agent.py`
Human-in-Loop	✅	5	`human_in_loop_agent.py`
Vision	✅	6	`multimodal_agent.py`
MCP	✅	7	`mcp/mcp_client.py`, `mcp_agent.py`
LangSmith	✅	8	`langsmith_service.py`
WebSocket Chat	✅	9	`websocket_manager.py`, `chat_service.py`

Phase 1: RAG & Semantic Search

Document upload and indexing
Vector search & storage with ChromaDB
Basic RAG and Semantic Search with LangGraph LCEL
Document processing and chunking

Phase 2: SQL Agent

Natural language to SQL
Database schema introspection
Query execution and explanation

Phase 3: Custom RAG (LangGraph)

State-based workflow
Question analysis
Conditional routing
Clarification requests

Phase 4: Tools & Memory

Web search (DuckDuckGo)
Calculator
Wikipedia
Conversation memory with SQLite checkpoints
Short-Term Memory - SQLite checkpoint persistence
Conversation History - Thread-based tracking
Tool Agent - Orchestrates all tools with memory

Phase 5: Multi-Agent & Streaming

Multi-Agent Router - Intelligent routing to specialists
Real-time Streaming Agent - Real-time token/event streaming
Human-in-the-Loop with breakpoints - Breakpoints for sensitive operations
State Management - Advanced LangGraph workflows
Customer Support Agent with handoffs - Handoffs between specialists

Phase 6: Advanced Features

Long-term memory (user profiles, preferences, facts)
Parallel execution
Deep research with sub-graphs
Synthesis from multiple sources
Image Analysis - Describe and analyze images
Multi-Image Comparison - Compare multiple images
OCR - Extract text from images
Document + Image - Combine text and visual analysis
Vision Models - Support for GPT-4V, Claude 3, Llava

Phase 7: MCP Integration

MCP Client - Connect to external MCP servers
Server Management - Add/remove MCP servers dynamically
Tool Discovery - Auto-discover available tools
MCP Agent - AI-powered tool selection and execution
Pre-configured Presets - GitHub, Slack, Notion, Google Drive

Phase 8: LangSmith Integration (Observability)

Auto-Tracing - Automatic trace capture
Project Management - Organize runs by project
Feedback System - Collect user feedback
Dataset Creation - Build evaluation datasets
Run Analytics - View performance statistics
Error Tracking - Monitor failures

Phase 9: Real-Time WebSocket Chat

WebSocket Connections - Real-time bidirectional communication
Connection Management - Multi-device support
Message Broadcasting - Broadcast to conversation participants
Typing Indicators - Show when users are typing
Read Receipts - Track message read status
Multi-Device Sync
Agent Streaming - Stream AI responses in real-time
Message Persistence - SQLite database storage
Conversation History - Load past messages
Pagination - Efficient message loading
Multi-User Support - Multiple users per conversation
Reconnection Handling - Graceful disconnect/reconnect

Quick Start

Initial Setup

Install Ollama from https://ollama.ai

Pull required models:

ollama pull llama3.2
ollama pull nomic-embed-text

# For vision features (optional):
ollama pull llava
# Or smaller vision model:
ollama pull llava-phi3

Messaging feature

FastAPI server has WebSocket endpoints, but you need a client to connect to them. Test clients provide ability to:

Test that WebSocket chat works
See messages streaming in real-time
Test multiple users (open multiple browser tabs)
Debug connection issues

Different ways to run

Browser Client (EASIEST - No installation or dependencies needed!)

Start server:

uv run fastapi dev app/main.py

Open the HTML file in your browser: Just double-click on tests/websocket/test_websocket_client.html in folder

# OR in terminal
open tests/websocket/test_websocket_client.html

You'll see a nice chat interface:
- Click "Connect"
- Type messages
- See AI responses stream in real-time!

Python Client (For Testing from Terminal)

Install websockets:

uv add websockets --dev

Start server (in one terminal)

uv run fastapi dev app/main.py

Run the test:

python tests/websocket/test_websocket_client.py

Watch it connect and chat:

   🔌 Connecting to ws://localhost:8000/ws/chat/test-conversation?token=user123...
   ✅ Connected as user123!

   💬 Sending message...
   🤔 Agent is thinking...
   LangChain is a framework... (streaming in real-time)
   ✅ Agent response complete!

Node.js Client (For JavaScript Development)

Install ws package:

npm install ws

Run the test:

node tests/websocket/test_websocket_client.js

Testing Different Scenarios

Basic Chat

python tests/websocket/test_websocket_client.py basic

Connects Sends one message Receives AI response Disconnects

Multi-Turn Memory

python tests/websocket/test_websocket_client.py memory

Tests if AI remembers context:

"My name is Alice"
"What's my name?" → AI should say "Alice"

Typing Indicators

python tests/websocket/test_websocket_client.py typing

Tests typing status updates Shows "user is typing..." messages

LangSmith Setup (Optional)

Sign up at https://smith.langchain.com
Create API key

Add to .env:

LANGSMITH_ENABLED=true
LANGSMITH_API_KEY=your_key_here
LANGSMITH_PROJECT=my-project

MCP

Example MCP - Servers:

# GitHub
mcp_client.add_server("github", "http://localhost:3000/mcp", token)

# Slack
mcp_client.add_server("slack", "https://slack.com/api/mcp", token)

# Custom
mcp_client.add_server("my-service", "https://api.myservice.com/mcp")

Example MCP - GitHub Integration

# First, configure MCP server in services.py:
services.mcp_client.add_server(
    name="github",
    url="http://localhost:3000/mcp",
    api_key=os.getenv("GITHUB_TOKEN")
)

# Then use via API:
curl -X POST "http://localhost:8000/api/v1/mcp/query" \
  -H "Content-Type: application/json" \
  -d '{"question": "List my GitHub repositories"}'

MCP Server Setup

Example MCP server configurations available in mcp_client.py:

GitHub
Slack
Notion
Google Drive
Custom servers

Project Setup

# Install dependencies
make install
# OR
uv sync

# Setup environment
cp .env.example .env
# Edit .env with your keys

# Run server
make run
# OR
uv run fastapi dev app/main.py

# Run tests
make test-all
# OR
chmod +x tests/test_all_phases.sh && ./tests/test_all_phases.sh

# Test individual phases
make test-phase1
make test-phase4
make test-phase6

# Test streaming
make test-streaming

API Endpoints

Phase 1: RAG

POST /api/v1/upload-documents - Upload documents
POST /api/v1/semantic-search - Semantic search
POST /api/v1/rag-query - RAG query

Phase 2: SQL

POST /api/v1/sql-query - Natural language to SQL
GET /api/v1/database-schema - Get schema

Phase 3: Custom RAG

POST /api/v1/custom-rag - Advanced RAG with reasoning

Phase 4: Tools

POST /api/v1/tool-agent - Query with tools
GET /api/v1/tool-agent/history/{thread_id} - Get history
DELETE /api/v1/tool-agent/history/{thread_id} - Clear history

Phase 5: Multi-Agent

POST /api/v1/multi-agent - Intelligent routing
POST /api/v1/customer-support - Support agent
POST /api/v1/stream - Streaming responses
POST /api/v1/human-in-loop - HIL queries
POST /api/v1/human-in-loop/approve - Approve action

Phase 6: Memory, Parallelization & Vision

POST /api/v1/memory-agent - Query with long-term memory
POST /api/v1/memory-agent/profile - Create user profile
GET /api/v1/memory-agent/user/{user_id} - Get user info
POST /api/v1/parallel-search - Parallel search
POST /api/v1/research - Deep research
POST /api/v1/vision/analyze - Analyze single image
POST /api/v1/vision/compare - Compare multiple images
POST /api/v1/vision/ocr - Extract text from image

Phase 7: MCP Integration

GET /api/v1/mcp/servers - List MCP servers
GET /api/v1/mcp/tools - List available tools
POST /api/v1/mcp/query - Query using MCP tools
POST /api/v1/mcp/servers - Add MCP server
DELETE /api/v1/mcp/servers/{name} - Remove MCP server

Phase 8: LangSmith Integration

GET /api/v1/langsmith/stats - Get project statistics
GET /api/v1/langsmith/runs - List recent runs
POST /api/v1/langsmith/feedback - Add feedback to run
GET /api/v1/langsmith/run/{run_id} - Get run details

All agent calls are automatically traced when LangSmith is enabled. View traces at: https://smith.langchain.com

Phase 9: Real-Time WebSocket Chat

WebSocket Endpoint:

WS /ws/chat/{conversation_id}?token=user_id - Real-time chat

REST Endpoints:

POST /api/v1/chat/conversations - Create conversation
GET /api/v1/chat/conversations/{id} - Get conversation
GET /api/v1/chat/conversations/{id}/messages - Get messages
GET /api/v1/chat/users/{user_id}/conversations - List user convos
GET /api/v1/chat/stats - WebSocket stats

Message Types:

// User message
{type: "message", content: "Hello"}

// Typing indicator
{type: "typing", is_typing: true}

// Read receipt
{type: "read", message_id: "msg_123"}

// Agent streaming
{type: "agent_stream", content: "chunk..."}

// Agent complete
{type: "agent_complete", content: "full response"}

Project Structure:

└── langchain_learn
    └── app
        └── core
            ├── __init__.py
            ├── config.py
            └── services.py
        └── models
            ├── __init__.py
            └── chat.py
        └── routes
            ├── __init__.py
            ├── agents.py
            └── websocket.py
        └── services
            └── agents
                ├── __init__.py
                ├── custom_rag_agent.py
                ├── human_in_loop_agent.py
                ├── mcp_agent.py
                ├── memory_agent.py
                ├── multi_agent_router.py
                ├── multimodal_agent.py
                ├── parallel_agent.py
                ├── rag_agent.py
                ├── research_agent.py
                ├── sql_agent.py
                ├── streaming_agent.py
                ├── support_agent.py
                └── tool_agent.py
            └── mcp
                ├── __init__.py
                └── mcp_client.py
            └── memory
                ├── __init__.py
                └── long_term_memory.py
            └── tools
                ├── __init__.py
                ├── calculator.py
                ├── database_tools.py
                ├── web_search.py
                └── wikipedia.py
            ├── __init__.py
            ├── chat_service.py
            ├── database.py
            ├── langsmith_service.py
            ├── vector_store.py
            └── websocket_manager.py
        └── utils
            ├── __init__.py
            └── utc_now.py
        ├── __init__.py
        ├── dependencies.py
        ├── main.py
        └── schemas.py
    └── data
    └── tests
        ├── stress_test.sh
        ├── test_all_phases.sh
        ├── test_phase1.sh
        ├── test_phase2.sh
        ├── test_phase3.sh
        ├── test_phase4.sh
        └── test_streaming.sh
    ├── .env.example
    ├── .gitignore
    ├── .python-version
    ├── Makefile
    ├── prek.toml
    ├── pyproject.toml
    ├── README.md
    └── uv.lock

Development

# Format code
make format

# Lint
make lint

# Health check
make health

Testing

# Full test suite
make test-all

# Streaming tests
make test-streaming

# Stress test
make test-stress

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
app		app
tests		tests
.env.example		.env.example
.gitignore		.gitignore
.python-version		.python-version
Makefile		Makefile
README.md		README.md
Untitled-1.md		Untitled-1.md
Untitled-2.md		Untitled-2.md
prek.toml		prek.toml
pyproject.toml		pyproject.toml
uv.lock		uv.lock

oyinetare/langchain_learn

Folders and files

Latest commit

History

Repository files navigation

PROJECT: Enterprise AI Assistant Platform

Pre-requisites

Features

Feature Matrix

Phase 1: RAG & Semantic Search

Phase 2: SQL Agent

Phase 3: Custom RAG (LangGraph)

Phase 4: Tools & Memory

Phase 5: Multi-Agent & Streaming

Phase 6: Advanced Features

Phase 7: MCP Integration

Phase 8: LangSmith Integration (Observability)

Phase 9: Real-Time WebSocket Chat

Quick Start

Initial Setup

Messaging feature

Testing Different Scenarios

LangSmith Setup (Optional)

MCP

Project Setup

API Endpoints

Phase 1: RAG

Phase 2: SQL

Phase 3: Custom RAG

Phase 4: Tools

Phase 5: Multi-Agent

Phase 6: Memory, Parallelization & Vision

Phase 7: MCP Integration

Phase 8: LangSmith Integration

Phase 9: Real-Time WebSocket Chat

Development

Testing

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages