A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

Create characters in Unity with LLMs!

llama.go is like llama.cpp in pure Golang!

Build and run AI agents using Docker Compose. A collection of ready-to-use examples for orchestrating open-source LLMs, tools, and agent runtimes.

React Native binding of llama.cpp

A cosy home for your LLMs.

Self-evaluating interview for AI coders

Run AI ✨ assistant locally! with simple API for Node.js 🚀

LLama.cpp rust bindings

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.

This repo is to showcase how you can run a model locally and offline, free of OpenAI dependencies.

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

A Pure Rust based LLM (Any LLM based MLLM such as Spark-TTS) Inference Engine, powering by Candle framework.

Local ML voice chat using high-end models.

Run LLMs locally. A clojure wrapper for llama.cpp.

Booster - open accelerator for LLM models. Better inference and debugging for AI hackers

Complete offline AI ecosystem for Android: Chat (GGUF/LLMs), Images (Stable Diffusion 1.5), Voice (TTS/STT), and Knowledge (RAG Data-Packs). Or access 100+ cloud models via OpenRouter. Extensible plugins, zero subscriptions, no data harvesting. Open-source privacy-first AI on your terms.