E2EE LLM Proxy

A lightweight local proxy that enables any OpenAI-compatible application to transparently use Venice AI's End-to-End Encrypted (E2EE) models.

Your prompts are encrypted before leaving your machine. Neither Venice nor anyone in between can read them — only the TEE enclave can.

Your app (plaintext) --> localhost:5111 --> [E2EE Proxy] --> Venice API (encrypted)
                                                |
                                      ECDH + AES-256-GCM
                                      TEE attestation verified

Why?

Venice AI offers E2EE models running inside Trusted Execution Environments (TEE). The E2EE protocol requires client-side cryptography (ECDH key exchange, AES-GCM encryption, streaming decryption) that standard OpenAI SDKs don't implement.

This proxy handles the full E2EE handshake transparently — any app that speaks the OpenAI /v1/chat/completions format gets end-to-end encryption for free.

Use cases:

• Private conversations where prompts must never be visible to the API provider
• Sensitive content generation (legal, medical, business strategy)
• Any OpenAI SDK app that needs confidential inference
• Reference implementation of the Venice E2EE protocol

Note: Venice E2EE models do not support function calling or tool use. This proxy is designed for chat and text generation, not for agentic coding tools.

Features

• Transparent E2EE — encrypts all prompts with ECDH (secp256k1) + AES-256-GCM before sending to Venice
• TEE attestation — verifies the model runs in a genuine hardware enclave before trusting its public key
• OpenAI-compatible — drop-in /v1/chat/completions endpoint for any client or SDK
• Streaming — full SSE streaming with real-time decryption of response chunks
• Session management — automatic key rotation (1 hour), retry on stale sessions, per-model isolation
• Security hardened — constant-time crypto (@noble/curves), no plaintext logging, bounded request size, localhost-only CORS

Supported E2EE Models

Any Venice model with the e2ee- prefix. Examples:

Model	Parameters	Context	Price (input/output per 1M tokens)
e2ee-gpt-oss-120b-p	117B MoE	128K	$0.13 / $0.65
e2ee-qwen3-5-122b-a10b	122B MoE	128K	$0.50 / $4.00
e2ee-glm-4-7-flash-p	—	198K	$0.13 / $0.55
e2ee-glm-4-7-p	—	128K	$1.10 / $4.15
e2ee-qwen3-30b-a3b-p	30B MoE	—	$0.19 / $0.69
e2ee-gpt-oss-20b-p	21B MoE	128K	$0.05 / $0.19

Quick Start

Prerequisites

• Node.js >= 20
• A Venice AI API key (referral link)

Install

git clone https://github.com/x1m4x/e2ee-llm-proxy.git
cd e2ee-llm-proxy
npm install

Run

VENICE_API_KEY="your-venice-api-key" node server.js

The proxy starts on http://127.0.0.1:5111/v1.

Environment Variables

Variable	Default	Description
VENICE_API_KEY	(required)	Your Venice AI API key
VENICE_MODEL	e2ee-gpt-oss-120b-p	Default E2EE model
PROXY_PORT	5111	Local proxy port

Integration Examples

Python (OpenAI SDK)

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:5111/v1", api_key="unused")
response = client.chat.completions.create(
    model="e2ee-gpt-oss-120b-p",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)
for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

Node.js (OpenAI SDK)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "http://127.0.0.1:5111/v1", apiKey: "unused" });
const stream = await client.chat.completions.create({
  model: "e2ee-gpt-oss-120b-p",
  messages: [{ role: "user", content: "Hello!" }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

curl

curl http://127.0.0.1:5111/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "e2ee-gpt-oss-120b-p",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

LangChain / LlamaIndex / Any OpenAI-compatible Framework

Point base_url to http://127.0.0.1:5111/v1 — no other changes needed.

How It Works

1. Key generation — on first request, the proxy generates an ephemeral secp256k1 key pair
2. TEE attestation — fetches and verifies hardware attestation from Venice, obtaining the model's public key
3. Encryption — each message is encrypted with a fresh ECDH shared secret + HKDF-SHA256 + AES-256-GCM
4. Request — the encrypted payload is sent to Venice with E2EE headers (X-Venice-TEE-Client-Pub-Key, etc.)
5. Decryption — streaming response chunks are decrypted in real-time and forwarded as standard SSE

Sessions auto-rotate every hour. Ephemeral encryption keys are zeroed after use.

API Endpoints

Endpoint	Method	Description
/v1/chat/completions	POST	Proxied chat completions with E2EE
/v1/models	GET	Lists available models
/health	GET	Proxy health check

Testing

npm test

82 tests covering cryptography (ECDH, AES-256-GCM round-trips), session management, request validation, HTTP endpoints, CORS, and security headers.

Security

• Crypto — all ECDH operations use @noble/curves (audited, constant-time). AES-256-GCM via @noble/ciphers. HKDF-SHA256 via @noble/hashes.
• No plaintext logging — prompts, responses, and credentials are never written to logs
• Localhost only — binds to 127.0.0.1, CORS restricted to localhost origins
• Request limits — 10 MB max body size, 60s upstream timeout
• Session isolation — per-model sessions with mutex to prevent race conditions
• Attestation caveat — TEE attestation is verified server-side by Venice. For production use, consider implementing client-side attestation quote verification against Intel TDX / AMD SEV-SNP root certificates.

E2EE Limitations (Venice Platform)

These are inherent to Venice's E2EE implementation, not the proxy:

Feature	Status
Streaming	Required (non-streaming not supported)
Function calling	Not supported
Structured outputs	Not supported
Web search	Disabled
File uploads	Not supported
Vision / images	Not supported (except e2ee-qwen3-vl-30b-a3b-p)

License

MIT