Important

This is a fork of Ollama which we at NOLAI use for our Framework desktops. The Ollama team have stated that RPC is not their main priority so, we want to keep our own active branch maintaining it.

We sometimes will also add different features that we like. If you have any questions feel free to contact us.

Here is the docker image: https://hub.docker.com/r/julianvanderhorst/ollama-framework-desktop

Setup your frameworks

You need to have the correct ROCM install on your framework. Make sure to follow these two guides:

https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/install/installryz/native_linux/install-ryzen.html

and then

https://instinct.docs.amd.com/projects/container-toolkit/en/latest/container-runtime/quick-start-guide.html

Make sure your frameworks have a good and fast connection between them. You can use a good usb4 cable or network them together with a switch

RPC docker compose

services:
  ollama:
    volumes:
      - ollama:/root/.ollama
    container_name: ollama
    tty: true
    restart: unless-stopped
    image: julianvanderhorst/ollama-framework-desktop:latest
    network_mode: "host"
    environment:
      - export OLLAMA_LOAD_TIMEOUT=10m
      - OLLAMA_RPC_SERVERS=127.0.0.1:50053
      - OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/rocm_v7
      - LD_LIBRARY_PATH=/opt/rocm/lib:/usr/lib/ollama:/usr/lib/ollama/rocm_v7
    depends_on:
      - ollama_rpc_worker

  ollama_rpc_worker:
    container_name: ollama_rpc_worker_sam
    volumes:
      - ollama:/root/.ollama
    image: julianvanderhorst/ollama-framework-desktop:latest
    runtime: amd
    restart: unless-stopped
    network_mode: "host"
    environment:
      - AMD_VISIBLE_DEVICES=all
      - OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/rocm_v7
      - LD_LIBRARY_PATH=/opt/rocm/lib:/usr/lib/ollama:/usr/lib/ollama/rocm_v7
    command: rpc --host 0.0.0.0 --port 50053 --device ROCm0
volumes:
  ollama: {}

You can just add RPC clients to OLLAMA_RPC_SERVERS by appending them with a comma. 127.0.0.1:50053,192.168.1.1:50053,192.168.1.12:50053 etc.

Ollama

Start building with open models.

Download

macOS

curl -fsSL https://ollama.com/install.sh | sh

or download manually

Windows

irm https://ollama.com/install.ps1 | iex

or download manually

Linux

curl -fsSL https://ollama.com/install.sh | sh

Manual install instructions

Docker

The official Ollama Docker image ollama/ollama is available on Docker Hub.

Libraries

• ollama-python
• ollama-js

Community

• Discord
• 𝕏 (Twitter)
• Reddit

Get started

ollama

You'll be prompted to run a model or connect Ollama to your existing agents or applications such as claude, codex, openclaw and more.

Coding

To launch a specific integration:

ollama launch claude

Supported integrations include Claude Code, Codex, Droid, and OpenCode.

AI assistant

Use OpenClaw to turn Ollama into a personal AI assistant across WhatsApp, Telegram, Slack, Discord, and more:

ollama launch openclaw

Chat with a model

Run and chat with Gemma 3:

ollama run gemma3

See ollama.com/library for the full list.

See the quickstart guide for more details.

REST API

Ollama has a REST API for running and managing models.

curl http://localhost:11434/api/chat -d '{
  "model": "gemma3",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'

See the API documentation for all endpoints.

Python

pip install ollama

from ollama import chat

response = chat(model='gemma3', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response.message.content)

JavaScript

npm i ollama

import ollama from "ollama";

const response = await ollama.chat({
  model: "gemma3",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);

Supported backends

• llama.cpp project founded by Georgi Gerganov.

Documentation

• CLI reference
• REST API reference
• Importing models
• Modelfile reference
• Building from source

Community Integrations

Want to add your project? Open a pull request.

Chat Interfaces

Web

• Open WebUI - Extensible, self-hosted AI interface
• Onyx - Connected AI workspace
• LibreChat - Enhanced ChatGPT clone with multi-provider support
• Lobe Chat - Modern chat framework with plugin ecosystem (docs)
• NextChat - Cross-platform ChatGPT UI (docs)
• Perplexica - AI-powered search engine, open-source Perplexity alternative
• big-AGI - AI suite for professionals
• Lollms WebUI - Multi-model web interface
• ChatOllama - Chatbot with knowledge bases
• Bionic GPT - On-premise AI platform
• Chatbot UI - ChatGPT-style web interface
• Hollama - Minimal web interface
• Chatbox - Desktop and web AI client
• chat - Chat web app for teams
• Ollama RAG Chatbot - Chat with multiple PDFs using RAG
• Tkinter-based client - Python desktop client

Desktop

• Dify.AI - LLM app development platform
• AnythingLLM - All-in-one AI app for Mac, Windows, and Linux
• Maid - Cross-platform mobile and desktop client
• Witsy - AI desktop app for Mac, Windows, and Linux
• Cherry Studio - Multi-provider desktop client
• Ollama App - Multi-platform client for desktop and mobile
• PyGPT - AI desktop assistant for Linux, Windows, and Mac
• Alpaca - GTK4 client for Linux and macOS
• SwiftChat - Cross-platform including iOS, Android, and Apple Vision Pro
• Enchanted - Native macOS and iOS client
• RWKV-Runner - Multi-model desktop runner
• Ollama Grid Search - Evaluate and compare models
• macai - macOS client for Ollama and ChatGPT
• AI Studio - Multi-provider desktop IDE
• Reins - Parameter tuning and reasoning model support
• ConfiChat - Privacy-focused with optional encryption
• LLocal.in - Electron desktop client
• MindMac - AI chat client for Mac
• Msty - Multi-model desktop client
• BoltAI for Mac - AI chat client for Mac
• IntelliBar - AI-powered assistant for macOS
• Kerlig AI - AI writing assistant for macOS
• Hillnote - Markdown-first AI workspace
• Perfect Memory AI - Productivity AI personalized by screen and meeting history

Mobile

• Ollama Android Chat - One-click Ollama on Android

SwiftChat, Enchanted, Maid, Ollama App, Reins, and ConfiChat listed above also support mobile platforms.

Code Editors & Development

• Cline - VS Code extension for multi-file/whole-repo coding
• Continue - Open-source AI code assistant for any IDE
• Void - Open source AI code editor, Cursor alternative
• Copilot for Obsidian - AI assistant for Obsidian
• twinny - Copilot and Copilot chat alternative
• gptel Emacs client - LLM client for Emacs
• Ollama Copilot - Use Ollama as GitHub Copilot
• Obsidian Local GPT - Local AI for Obsidian
• Ellama Emacs client - LLM tool for Emacs
• orbiton - Config-free text editor with Ollama tab completion
• AI ST Completion - Sublime Text 4 AI assistant
• VT Code - Rust-based terminal coding agent with Tree-sitter
• QodeAssist - AI coding assistant for Qt Creator
• AI Toolkit for VS Code - Microsoft-official VS Code extension
• Open Interpreter - Natural language interface for computers

Libraries & SDKs

• LiteLLM - Unified API for 100+ LLM providers
• Semantic Kernel - Microsoft AI orchestration SDK
• LangChain4j - Java LangChain (example)
• LangChainGo - Go LangChain (example)
• Spring AI - Spring framework AI support (docs)
• LangChain and LangChain.js with example
• Ollama for Ruby - Ruby LLM library
• any-llm - Unified LLM interface by Mozilla
• OllamaSharp for .NET - .NET SDK
• LangChainRust - Rust LangChain (example)
• Agents-Flex for Java - Java agent framework (example)
• Elixir LangChain - Elixir LangChain
• Ollama-rs for Rust - Rust SDK
• LangChain for .NET - .NET LangChain (example)
• chromem-go - Go vector database with Ollama embeddings (example)
• LangChainDart - Dart LangChain
• LlmTornado - Unified C# interface for multiple inference APIs
• Ollama4j for Java - Java SDK
• Ollama for Laravel - Laravel integration
• Ollama for Swift - Swift SDK
• LlamaIndex and LlamaIndexTS - Data framework for LLM apps
• Haystack - AI pipeline framework
• Firebase Genkit - Google AI framework
• Ollama-hpp for C++ - C++ SDK
• PromptingTools.jl - Julia LLM toolkit (example)
• Ollama for R - rollama - R SDK
• Portkey - AI gateway
• Testcontainers - Container-based testing
• LLPhant - PHP AI framework

Frameworks & Agents

• AutoGPT - Autonomous AI agent platform
• crewAI - Multi-agent orchestration framework
• Strands Agents - Model-driven agent building by AWS
• Cheshire Cat - AI assistant framework
• any-agent - Unified agent framework interface by Mozilla
• Stakpak - Open source DevOps agent
• Hexabot - Conversational AI builder
• Neuro SAN - Multi-agent orchestration (docs)

RAG & Knowledge Bases

• RAGFlow - RAG engine based on deep document understanding
• R2R - Open-source RAG engine
• MaxKB - Ready-to-use RAG chatbot
• Minima - On-premises or fully local RAG
• Chipper - AI interface with Haystack RAG
• ARGO - RAG and deep research on Mac/Windows/Linux
• Archyve - RAG-enabling document library
• Casibase - AI knowledge base with RAG and SSO
• BrainSoup - Native client with RAG and multi-agent automation

Bots & Messaging

• LangBot - Multi-platform messaging bots with agents and RAG
• AstrBot - Multi-platform chatbot with RAG and plugins
• Discord-Ollama Chat Bot - TypeScript Discord bot
• Ollama Telegram Bot - Telegram bot
• LLM Telegram Bot - Telegram bot for roleplay

Terminal & CLI

• aichat - All-in-one LLM CLI with Shell Assistant, RAG, and AI tools
• oterm - Terminal client for Ollama
• gollama - Go-based model manager for Ollama
• tlm - Local shell copilot
• tenere - TUI for LLMs
• ParLlama - TUI for Ollama
• llm-ollama - Plugin for Datasette's LLM CLI
• ShellOracle - Shell command suggestions
• LLM-X - Progressive web app for LLMs
• cmdh - Natural language to shell commands
• VT - Minimal multimodal AI chat app

Productivity & Apps

• AppFlowy - AI collaborative workspace, self-hostable Notion alternative
• Screenpipe - 24/7 screen and mic recording with AI-powered search
• Vibe - Transcribe and analyze meetings
• Page Assist - Chrome extension for AI-powered browsing
• NativeMind - Private, on-device browser AI assistant
• Ollama Fortress - Security proxy for Ollama
• 1Panel - Web-based Linux server management
• Writeopia - Text editor with Ollama integration
• QA-Pilot - GitHub code repository understanding
• Raycast extension - Ollama in Raycast
• Painting Droid - Painting app with AI integrations
• Serene Pub - AI roleplaying app
• Mayan EDMS - Document management with Ollama workflows
• TagSpaces - File management with AI tagging

Observability & Monitoring

• Opik - Debug, evaluate, and monitor LLM applications
• OpenLIT - OpenTelemetry-native monitoring for Ollama and GPUs
• Lunary - LLM observability with analytics and PII masking
• Langfuse - Open source LLM observability
• HoneyHive - AI observability and evaluation for agents
• MLflow Tracing - Open source LLM observability

Database & Embeddings

• pgai - PostgreSQL as a vector database (guide)
• MindsDB - Connect Ollama with 200+ data platforms
• chromem-go - Embeddable vector database for Go (example)
• Kangaroo - AI-powered SQL client

Infrastructure & Deployment

Cloud

• Google Cloud
• Fly.io
• Koyeb
• Harbor - Containerized LLM toolkit with Ollama as default backend

Package Managers

• Pacman
• Homebrew
• Nix package
• Helm Chart
• Gentoo
• Flox
• Guix channel