🧠 Local AI Assistant (Offline CLI)

A fully offline AI assistant that runs locally on your machine using lightweight LLMs like LLaMA, Mistral, or Qwen via Ollama.

No API keys. No internet dependency. Just local AI.

---

🚀 Features

• 🖥️ Runs entirely offline
• ⚡ Powered by local LLMs (via Ollama)
• 💬 CLI-based chat interface
• 🧠 Conversation memory (session-based)
• 🔌 Extensible for RAG and agent workflows

---

🏗️ Architecture

[CLI / Terminal] ↓ [Node.js App] ↓ [Ollama (Local AI Engine)] ↓ [Local LLM]

---

⚙️ Setup

1. Install Ollama

brew install ollama

Start the Ollama server:

ollama serve

2. Pull a Model

ollama pull qwen2.5:latest

Other options:

ollama pull llama3.2
ollama pull mistral
ollama pull phi3.5

3. Create the Project

mkdir local-ai-assistant
cd local-ai-assistant

npm init -y
npm install axios readline-sync

4. Create the Assistant Script

Create a file named assistant.js:

#!/usr/bin/env node

const axios = require('axios');
const readline = require('readline-sync');

let history = [];

async function chat() {
    while (true) {
        const input = readline.question("You: ");

        history.push({ role: "user", content: input });

        const response = await axios.post('http://localhost:11434/api/chat', {
            model: 'qwen2.5:latest',
            messages: history,
            stream: false
        });

        const reply = response.data.message?.content || "No response";

        history.push({ role: "assistant", content: reply });

        console.log("AI:", reply);
    }
}

chat();

5. Make It a Global CLI Tool

Add this to your package.json:

{
  "name": "local-ai",
  "bin": {
    "local-ai": "./assistant.js"
  }
}

Then run:

npm link

Now you can start your assistant from anywhere:

local-ai

Roadmap

• 📂 File-based memory (RAG)
• 🛠️ Tool usage (agent capabilities)
• 🔗 Laravel API integration
• 🐳 Docker deployment
• ☁️ AWS hosting

Connect

GitHub: https://github.com/abbeymaniak LinkedIn: https://linkedin.com/in/abiodun-paul-ogunnaike