JanitorAI Local LLM Proxy

Use local AI models with JanitorAI - Run Qwen, Llama, Mistral, and other LLMs locally via LM Studio

Run local LLMs (via LM Studio) with JanitorAI using nginx reverse proxy and a CSP bypass browser extension. No paid API keys needed - use your own hardware!

Keywords: JanitorAI local LLM, LM Studio JanitorAI, free JanitorAI API, JanitorAI reverse proxy, local AI roleplay

Overview

JanitorAI blocks custom API endpoints via Content Security Policy (CSP). This setup bypasses that restriction using:

1. LM Studio - Local LLM server (OpenAI-compatible API)
2. nginx - Reverse proxy that rewrites endpoints and handles CORS
3. ngrok - Exposes local server to the internet
4. CSP Bypass Extension - Firefox/Chrome extension that removes CSP restrictions

JanitorAI → ngrok → nginx:5001 → LM Studio:1234
                      ↓
              Rewrites /chat/completions
                 to /v1/chat/completions

Requirements

• Windows 10/11
• LM Studio 0.4.6+
• ngrok (free account)
• Firefox 109+ or Chrome/Brave
• A local LLM model (e.g., Qwen 3.5 9B, Llama, Mistral)

Quick Start

1. Install & Configure LM Studio

1. Download and install LM Studio
2. Download a model (e.g., Qwen 3.5 9B)
3. Load the model
4. Important: Disable "Thinking Mode" if using Qwen 3.x models
5. Start the local server (default port: 1234)

2. Install nginx

# Download nginx for Windows
curl -L -o nginx.zip https://nginx.org/download/nginx-1.24.0.zip

# Extract to C:\nginx-1.24.0
unzip nginx.zip -d C:\

3. Configure nginx

Replace C:\nginx-1.24.0\conf\nginx.conf with:

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/json;

    sendfile        on;
    keepalive_timeout  65;

    upstream lmstudio {
        server 127.0.0.1:1234;
    }

    server {
        listen       5001;
        server_name  localhost;

        # Hide upstream CORS headers to avoid duplicates
        proxy_hide_header 'Access-Control-Allow-Origin';
        proxy_hide_header 'Access-Control-Allow-Methods';
        proxy_hide_header 'Access-Control-Allow-Headers';

        # JanitorAI sends to /chat/completions - proxy to /v1/chat/completions
        location = /chat/completions {
            if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' '*';
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
                add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization';
                add_header 'Access-Control-Max-Age' 86400;
                return 204;
            }

            add_header 'Access-Control-Allow-Origin' '*' always;
            add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
            add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;

            proxy_pass http://lmstudio/v1/chat/completions;
            proxy_http_version 1.1;
            proxy_set_header Host $host;
            proxy_set_header Content-Type $content_type;
            proxy_set_header Authorization $http_authorization;
            proxy_buffering off;
            proxy_read_timeout 300;
        }

        # Handle /models without /v1
        location = /models {
            add_header 'Access-Control-Allow-Origin' '*' always;
            proxy_pass http://lmstudio/v1/models;
            proxy_http_version 1.1;
            proxy_set_header Host $host;
        }

        # Default - pass through as-is
        location / {
            if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' '*';
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
                add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization';
                add_header 'Access-Control-Max-Age' 86400;
                return 204;
            }

            add_header 'Access-Control-Allow-Origin' '*' always;
            add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
            add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization' always;

            proxy_pass http://lmstudio;
            proxy_http_version 1.1;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_buffering off;
            proxy_cache off;
            proxy_connect_timeout 300;
            proxy_send_timeout 300;
            proxy_read_timeout 300;
        }
    }
}

4. Start nginx

cd C:\nginx-1.24.0
nginx.exe

5. Install ngrok

1. Download ngrok
2. Create a free account and get your auth token
3. Configure ngrok:

   ngrok config add-authtoken YOUR_TOKEN

6. Start ngrok

ngrok http 5001

Copy the public URL (e.g., https://xxxx-xx-xx-xx-xx.ngrok-free.app)

7. Install CSP Bypass Extension

Download from: https://github.com/Dabaski/janitorai-csp-bypass

Firefox:

1. Clone or download the extension repo
2. Go to about:debugging
3. Click "This Firefox"
4. Click "Load Temporary Add-on..."
5. Select manifest.json from the extension folder

Chrome/Brave:

1. Clone or download the extension repo
2. Go to chrome://extensions
3. Enable "Developer mode"
4. Click "Load unpacked"
5. Select the extension folder

8. Configure JanitorAI

1. Go to JanitorAI
2. Open API Settings
3. Select "Proxy" or "OpenAI Compatible"
4. Enter:
   • API URL: https://your-ngrok-url.ngrok-free.app (no /v1)
   • API Key: Any value (e.g., sk-local)
   • Model: qwen/qwen3.5-9b (or your model name)

Troubleshooting

"Thinking Process:" in responses

• Disable thinking mode in LM Studio model settings
• Reload the model after changing settings

CORS errors

• Make sure nginx is running (not the Python proxy)
• Check only ONE process is on port 5001: netstat -ano | grep 5001

"Unexpected endpoint" errors

• nginx rewrites /chat/completions to /v1/chat/completions
• Use base URL without /v1 in JanitorAI

CSP still blocking

• Verify the extension is installed and enabled
• Check the extension icon shows "Enabled"
• Try refreshing JanitorAI page

ngrok URL changed

• Free tier URLs change on restart
• Check http://localhost:4040 for current URL
• Consider Cloudflare Tunnel for permanent URLs

Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  JanitorAI  │────▶│    ngrok    │────▶│    nginx    │────▶│  LM Studio  │
│  (Browser)  │     │   :443      │     │   :5001     │     │   :1234     │
└─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘
       │                                       │
       │                                       ▼
       │                              ┌─────────────────┐
       │                              │ Path Rewriting  │
       │                              │ /chat/completions│
       │                              │       ↓         │
       │                              │/v1/chat/completions│
       │                              └─────────────────┘
       ▼
┌─────────────────┐
│  CSP Bypass     │
│  Extension      │
│  (Removes CSP)  │
└─────────────────┘

Files

JanitorAI Proxy/
├── README.md                 # This file
├── config.txt                # Quick reference config
├── csp-bypass-extension/     # Firefox/Chrome extension
│   ├── manifest.json
│   ├── background.js
│   ├── popup/
│   ├── options/
│   ├── icons/
│   └── tests/
├── ngrok.exe                 # ngrok binary
├── proxy_server.py           # Alternative Python proxy (not needed)
├── test_connection.py        # Connection test script
├── test_page.html            # Browser test page
└── SillyTavern/              # Alternative frontend (optional)

Alternative: SillyTavern

If you prefer not to use JanitorAI, SillyTavern works directly with LM Studio without needing ngrok or CSP bypass:

cd SillyTavern
npm install
node server.js

Then open http://localhost:8000 and configure:

• API: Chat Completion > Custom (OpenAI-compatible)
• Endpoint: http://localhost:1234/v1
• Model: qwen/qwen3.5-9b

License

MIT

Credits

• LM Studio - Local LLM runtime
• nginx - Reverse proxy
• ngrok - Tunneling service