🚀 LLM Engineering Mastery

Architect: Jobiryasir
The ultimate blueprint for building, scaling, and deploying production-grade, local-first AI systems.

---

🎯 Project Vision

This repository is a Centralized Knowledge Hub (200+ Resources) designed for AI Architects. It bridges the gap between theoretical research and real-world system engineering.

---

🏛️ The Master Resource Index

⚙️ 1. Orchestration & RAG Pipelines (40 Resources)

Name	Description
LangChain	Universal LLM application framework.
LlamaIndex	RAG & Data indexing framework.
DSPy	Programmatic prompt optimization.
Haystack	Production-ready search/QA pipelines.
Unstructured	Extracting data from PDF/PPTX/HTML.
Txtai	AI-powered workflow & search engine.
Ragas	RAG evaluation metrics.
LangGraph	Stateful, cyclic LLM agent workflows.
Instructor	Structured data output using Pydantic.
Guidance	Control over LLM generation flows.
Outlines	Reliable generation (Regex/JSON).
LiteLLM	Unified interface for 100+ LLM APIs.
Semantic Kernel	Microsoft's integration framework.
Mem0	Persistent memory for LLM apps.
Pathway	Real-time data processing for RAG.
Docling	Transform documents for RAG.
PyRAG	Simple RAG pipeline starter.
Embedchain	RAG framework for custom data.
GPT4All	Local RAG ecosystem.
Langflow	UI-based LLM flow builder.
Flowise	Drag-and-drop orchestration UI.
Verba	RAG-based chatbot interface.
Rag-Fusion	Multi-query search.
FastEmbed	Light embedding generation.
Jina Embeddings	High-performance models.
Voyage AI	Optimized RAG retrieval.
Ares	Automated RAG evaluation.
TruLens	Evaluation for LLM apps.
Arize Phoenix	RAG observability & tracing.
DeepEval	Unit testing for LLMs.
Promptfoo	Systematic prompt testing.
Guardrails AI	Validating LLM outputs.
NeMo Guardrails	Controlling conversational flow.
LLM-Guard	Security suite for LLMs.
Giskard	Testing for AI models.
PromptLayer	Prompt management.
Helicone	LLM observability & caching.
Portkey	AI gateway & observability.
LangSmith	Enterprise LLM tracing.
Weights & Biases	Experiment tracking.

---

🛠️ How to maintain this repo (Professional Rules)

1. Sandbox Rule: Create a Labs/ folder. Every tool you learn MUST have a corresponding script inside this folder.
2. Weekly Audit: Every Saturday, scan for new tools on Hugging Face Trending and update this README.
3. Architecture Mindset: Don't just learn a tool; learn how to connect it. (Example: LangChain + ChromaDB + FastEmbed = A complete RAG pipeline).

---

🤖 2. Agentic Reasoning & Autonomy (40 Resources)

Name	Description
CrewAI	Multi-agent collaborative framework.
AutoGen	Conversational agents with code execution.
LangGraph	Stateful, cyclic agent workflows.
Smolagents	Minimalist, highly effective agents.
OpenInterpreter	AI running code locally on your OS.
MetaGPT	Multi-agent software company simulation.
AutoGPT	The original autonomous agent project.
BabyAGI	Task-driven autonomous agent.
SuperAGI	Versatile autonomous agent framework.
TaskWeaver	Code-first agent framework.
GPT-Pilot	Autonomous developer agent.
Devin	Agentic software engineering tool.
Multi-On	Web-browsing autonomous agents.
Skyvern	Automating browser-based workflows.
Browser-use	Agents that control web browsers.
AgentOps	Observability for agentic workflows.
E2B	Secure code sandboxes for agents.
Swarm	Lightweight multi-agent orchestration.
VertexAI Agents	Enterprise agent framework.
Agency	Multi-agent framework for Node/TS.
Cerebral	Agentic reasoning for complex tasks.
Reasoning-Agents	Chain-of-thought implementations.
Agent-Protocol	Standardization for AI agents.
MemGPT	Agents with infinite memory.
Bee-Agent-Framework	Building powerful agents in Python.
Temporal	Durable execution for agent workflows.
Agno	Framework for building useful agents.
Local-Agent-Runner	Run agents entirely local-first.
Agent-Bench	Benchmark for evaluating agents.
ToolBench	Toolkit for instruction tuning agents.
ReAct-Framework	ReAct reasoning pattern.
Plan-and-Solve	Planning-heavy autonomous agents.
Reflection-Agent	Self-correcting agentic patterns.
Prompt-Agent	Autonomous prompt optimization.
Aider	Agentic coding assistant.
Continue	IDE-integrated agentic assistant.
Cursor	The leading agentic coding environment.
Plandex	Agentic planner for complex tasks.
Autonomous-Research	Research-oriented agent systems.
Semantic-Router	Fast semantic routing for agents.

---

⚡ 3. Inference & High-Performance Serving (40 Resources)

Name	Description
vLLM	High-throughput LLM serving engine.
Ollama	Simplest way to run local LLMs.
Text-Generation-WebUI	Full-featured local LLM interface.
llama.cpp	Hardware-optimized inference engine.
LocalAI	Self-hosted OpenAI-compatible API.
Triton Inference Server	Multi-framework production serving.
Text-Generation-Inference	Hugging Face's production LLM server.
DeepSpeed-MII	Low-latency inference via DeepSpeed.
Ray-Serve	Scalable serving for AI models.
TensorRT-LLM	NVIDIA's optimized LLM inference.
SGLang	High-speed structured generation.
Aviary	Managed LLM serving on Ray.
ExLlamaV2	Fastest inference for GPTQ/EXL2 models.
AutoGPTQ	Easy-to-use quantization for LLMs.
AutoAWQ	Activation-aware weight quantization.
BitNet	1-bit LLM inference optimization.
Candle	Rust-based ML framework for low-latency.
ExecuTorch	PyTorch inference on edge devices.
BentoML	High-performance model serving framework.
Ray-LLM	LLM serving specifically on Ray.
MLC-LLM	Universal deployment on any hardware.
OpenLLM	Serving LLMs in production easily.
Modal	Serverless infrastructure for inference.
RunPod-Serverless	Serverless GPU inference nodes.
Anyscale-Endpoints	Managed scalable LLM endpoints.
Groq-Inference	Ultra-low latency LPU inference.
Together-API	Distributed inference platform.
Fireworks-AI	Fastest API-based LLM serving.
Mistral-Inference	Official optimized Mistral serving.
Flash-Attention	Speed-up for long-context attention.
PagedAttention	Memory management optimization for LLMs.
Speculative-Decoding	Fast decoding using smaller draft models.
Quantization-Toolkit	Intel-optimized LLM inference.
LibTorch	C++ interface for PyTorch deployment.
ONNX-Runtime	Cross-platform model inference.
FasterTransformer	NVIDIA's high-speed transformer core.
LM-Deploy	High-performance deployment for LLMs.
KServe	Serverless inference on Kubernetes.
Seldon-Core	Advanced MLOps model serving.
Triton-Python-Backend	Custom Python logic for production servers.

---

🗄️ 4. Vector Databases & Memory (40 Resources)

Name	Description
ChromaDB	AI-native open-source vector store.
Qdrant	High-performance vector search engine.
Milvus	Cloud-native vector database for massive scale.
Pinecone	Managed vector search (Enterprise standard).
Weaviate	Vector database with GraphQL support.
Faiss	Facebook's library for similarity search.
LanceDB	Serverless, embeddable vector database.
Typesense	Fast, typo-tolerant search & vector store.
Pgvector	Vector similarity search for PostgreSQL.
Redis-Vector	Redis-based vector search capabilities.
Elasticsearch-Vector	Vector search engine for ELK stack.
Manticore-Search	Fast, open-source search engine.
Vespa	Big data processing and vector search.
Vald	Highly scalable distributed vector search.
DiskANN	Large-scale SSD-based vector search.
Annoy	Spotify's library for Approximate Nearest Neighbors.
Hnswlib	Fast HNSW algorithm implementation.
Pinecone-Client	Python client for Pinecone API.
Milvus-SDK	Python SDK for Milvus database.
Chroma-SDK	Official Python SDK for ChromaDB.
Qdrant-Client	Python client for Qdrant.
Mem0	Persistent memory layer for LLM agents.
Zep	Long-term memory for LLM apps.
LangChain-Memory	Built-in memory components in LangChain.
LlamaIndex-Storage	Data storage abstractions.
Vector-Admin	UI for managing vector databases.
Vector-Search-Bench	Benchmark suite for vector DBs.
Marqo	End-to-end vector search engine.
Jina-Reader	Turning any website into LLM-ready data.
Unstructured-Storage	Storage for processed data chunks.
KV-Store-RAG	Key-value store integration for RAG.
DuckDB-Vector	Analytical SQL + Vector capabilities.
SurrealDB	Multi-model DB with vector support.
Neo4j-Vector	Graph-based vector search implementation.
Cassandra-Vector	Vector similarity for NoSQL databases.
Milvus-Lite	Lightweight Milvus for local development.
Chroma-Server	HTTP server for ChromaDB deployment.
Vector-DB-Comparison	Comparison guide for vector stores.
Approximate-Search	Non-Metric Space Library for search.
Vector-Flow	Real-time vector indexing pipeline.

---

📈 5. Evaluation, Observability & Guardrails (40 Resources)

Name	Description
RAGAS	RAG pipeline evaluation (Faithfulness, Relevance).
Promptfoo	Unit testing for prompts and LLM outputs.
LangSmith	End-to-end tracing, debugging, and testing.
Arize Phoenix	Observability and evaluation for RAG.
DeepEval	Unit testing for LLMs.
Giskard	Testing for AI models and scanner for bias.
NeMo Guardrails	Adding safety and boundaries to LLM behavior.
LLM-Guard	Security suite for LLMs (Pii, Jailbreak detection).
Helicone	LLM observability & caching.
Portkey	AI gateway with observability.
PromptLayer	Prompt management and experiment logging.
Weights & Biases	Experiment tracking and model monitoring.
Garak	LLM vulnerability scanner (Red teaming).
Presidio	PII detection and anonymization for LLMs.
TruLens	Evaluation for LLM apps (Feedback functions).
Ares	Automated RAG evaluation framework.
Inspect AI	Framework for evaluating LLM capabilities.
LangWatch	LLM observability and evaluation.
Monitor-LLM	Tracing framework for RAG apps.
RAG-Evals	Standard benchmarks for RAG systems.
Pytest-LLM	Testing integration for Pytest.
Prompt-Optimizer	Optimizing prompts programmatically.
Llama-Guard	Safety input/output filtering for models.
Check-List	Behavioral testing for NLP models.
UpTrain	Monitoring and evaluation for LLMs.
LangFuse	Open-source tracing and eval.
Honeycomb	Observability for distributed AI systems.
Open-Telemetry	Standard for distributed system tracing.
Safety-Gym	Tools for evaluating AI safety.
Robust-Bench	Benchmark for adversarial robustness.
Prompt-Shield	Real-time guardrails for LLM inputs.
Red-Teaming-Framework	Automated red-teaming for LLMs.
Literal-AI	Observability and evaluation platform.
Lang-Sentry	Error tracking for LLM applications.
Prompt-Optimizer	LangChain's internal eval tools.
Evaluating-Agents	Evaluation for agent-based reasoning.
Adversarial-Prompt-Detection	Detecting jailbreak attempts.
Eval-Toolkit	Comprehensive LLM unit testing.
Monitoring-Dashboard	Visualizing LLM latency and quality.
Feedback-Loop	Implementing user-feedback loops.

---

🏗️ 6. MLOps & Advanced Infrastructure

Name	Description
Kubeflow	Machine learning toolkit for Kubernetes.
MLflow	Platform for the machine learning lifecycle.
DVC	Data Version Control for ML projects.
Ray	Scalable compute framework for AI/Python.
Pachyderm	Data versioning and pipelining.
Flyte	Workflow automation platform for ML.
ZenML	MLOps framework to orchestrate pipelines.
BentoML	Unified model serving framework.
Seldon Core	Deploying ML models on Kubernetes.
KServe	Serverless model inference on Kubernetes.
NVIDIA Triton	Multi-model production inference server.
Docker	Containerization for portable AI environments.
Terraform	Infrastructure as Code (IaC) for AI cloud.
Ansible	IT automation for cluster configuration.
Prometheus	Monitoring for AI system metrics.
Grafana	Visualization dashboard for system health.
Airflow	Workflow orchestration for data pipelines.
Prefect	Modern data workflow orchestration.
Dagster	Data-aware orchestrator for ML.
Great Expectations	Data quality and validation.
DeepLake	Vector database for AI dataset management.
LakeFS	Git-like branching for object storage.
Hugging Face Hub	Model and dataset versioning registry.
Neptune.ai	Metadata store for experiment tracking.
Comet ML	Tracking and monitoring for ML experiments.
ClearML	All-in-one MLOps suite.
CML	Continuous Machine Learning for CI/CD.
GitHub Actions	Automating CI/CD for model deployment.
Argo Workflows	Container-native workflows.
Hydra	Framework for configuring complex apps.
Ray-Tune	Scalable hyperparameter tuning.
Katib	Automated hyperparameter tuning on K8s.
Spack	Package manager for high-performance computing.
Slurm	Cluster management for large-scale training.
Singularity/Apptainer	Containers for scientific computing.
Prometheus-Client	Instrumenting Python code for monitoring.
Loki	Log aggregation for ML clusters.
NVIDIA-DCGM	Monitoring GPU performance in real-time.
Kubernetes	The foundation of scalable infra.
Tailscale	Secure private networking for AI clusters.

---

🛠️ The Architect's Workflow (Handling 200+ Links)

To maintain this professionally, follow these three rules:

1. The Sandbox Rule: Every tool you list here must have a corresponding folder in your repo (e.g., /Labs/ChromaDB_Test/). A link without code is just noise.
2. Standardization: All scripts in your repo should follow the same structure (Config, Core Logic, Evaluation).
3. The 3-Tier Audit:
   • Tier 1 (Core): Essential tools (LangChain, vLLM, Chroma).
   • Tier 2 (Expert): Specialised tools (DSPy, Ray, Distilabel).
   • Tier 3 (Research): Cutting-edge papers and experimental repos.

---

📅 Roadmap to Expert Architect

• 2026-Q3: Master RAG & Inference Serving.
• 2026-Q4: Build Multi-Agent Systems & Evaluation pipelines.
• 2027-Q1: Scaling with Ray, MLOps, and Synthetic Data.

---

Created by Jobiryasir | Architecting the future of Local-First, Private AI.