doppler-gpu

JavaScript and WGSL WebGPU inference for browser and Node, with CLI and OpenAI-compatible local server entry points. Doppler loads sharded RDRR model artifacts for text generation, embeddings, and reranking. Bun WebGPU support is experimental.

Try the live demo | npm | docs

Current evidence

Doppler is faster than Transformers.js on every model shown below, across Apple Metal and AMD Vulkan.

Metal evidence: Qwen 3.5 0.8B · Qwen 3 Embedding 0.6B · Qwen 3 Reranker 0.6B · Qwen 3.5 2B paired run · runtime profile

Vulkan evidence: Qwen 3.5 0.8B · Qwen 3 Embedding 0.6B · Qwen 3 Reranker 0.6B · Qwen 3.5 2B · Gemma 4

The links include output checks, load time, and hardware details. Qwen 3.5 2B Metal used 20 paired runs and a local Doppler model artifact. Transformers.js still loads the Vulkan embedding and reranker models faster. See the methodology and full results.

How it works

registry ID / model URL
          |
          v
+----------------------+    +----------------------+
| RDRR manifest        |--->| verified shards      |
| model + tokenizer    |    | OPFS / disk cache    |
| session + execution  |    +----------+-----------+
+----------------------+               |
                                       |
prompt / documents                     v
        +--------------------->+----------------------+
                               | JavaScript runtime   |
                               | prefill / decode / KV|
                               +----------+-----------+
                                          |
                                          v
                               +----------------------+
                               | WGSL / WebGPU        |
                               | selected kernels     |
                               +----------+-----------+
                                          |
                                          v
                              text / embeddings / scores

The manifest and runtime config select dtype and kernel paths before execution. Unsupported paths fail closed.

Quick start

Browser

Use the live demo link above. It runs entirely in the browser with no server required. Models load into the browser cache and work offline after the first download.

CLI

npx doppler-gpu

Downloads the default quickstart model, runs a local prompt, and prints the answer. Node quickstart artifacts are cached in ~/.cache/doppler-gpu/models after the first run; set DOPPLER_QUICKSTART_CACHE_DIR to move the cache or DOPPLER_QUICKSTART_CACHE=0 to disable it.

npx doppler-gpu "Summarize WebGPU in one sentence"
npx doppler-gpu --model qwen3-0.8b --prompt "Write a haiku about GPUs"
npx doppler-gpu --list-models

Root API

The dr facade is the primary app-facing API. doppler remains a compatibility alias. Advanced APIs live on explicit package subpaths.

import { dr } from 'doppler-gpu';

// Stream tokens
const model = await dr.load('qwen3-0.8b');
for await (const token of model.generate('Describe WebGPU briefly')) {
  process.stdout.write(token);
}

// One-shot
const text = await model.generateText('Explain WebGPU in one sentence');

OpenAI-compatible server

For existing apps, SDKs, and eval stacks that speak the OpenAI protocol:

npx doppler-serve --model qwen3-0.8b --port 8080

Then point any OpenAI client at http://localhost:8080/v1:

import OpenAI from 'openai';
const client = new OpenAI({ baseURL: 'http://localhost:8080/v1', apiKey: 'unused' });
const response = await client.chat.completions.create({
  model: 'qwen3-0.8b',
  messages: [{ role: 'user', content: 'Hello' }],
});

This compatibility bridge uses the same runtime contract as the browser and Node APIs.

Registry IDs resolve to hosted RDRR artifacts from Clocksmith/rdrr by default. See the Root API guide.

Support

The primary proof surface is the hosted browser demo, root dr API, quickstart CLI, OpenAI-compatible local server, and the hosted Qwen registry lanes below.

Registry alias	Artifact ID	Task
qwen3-0.8b	qwen-3-5-0-8b-q4k-ehaf16	Text generation
qwen3-embedding-0.6b	qwen-3-embedding-0-6b-q4k-ehf16-af32	Embeddings
qwen3-reranker-0.6b-q4k	qwen-3-reranker-0-6b-q4k-ehf16-af32	Reranking

Browser and Node are mainline runtime surfaces. Bun WebGPU is experimental. Use the model support matrix for verified models and the subsystem support matrix for public, experimental, and internal-only APIs.

Model roadmap

Current model priorities and promotion state live in the model roadmap. Exact registry IDs, runtime verification, and benchmark claims remain in the model support inventory and release matrix.

Documentation

• npm quickstart: run npx doppler-gpu --help
• Docs index (canonical navigation): docs/INDEX.md
• First-run workflow: docs/getting-started.md
• CLI reference: docs/cli.md
• Runtime config contract: docs/config.md
• Architecture: docs/architecture.md
• Model roadmap: docs/model-roadmap.md
• Model support matrix: docs/model-support-matrix.md

Environment requirements

• WebGPU is required.
• Browser: Current Chromium browsers with WebGPU enabled, including Chrome and Edge. WebGPU shipped in Chrome/Edge 113+. Firefox and Safari support varies.
• Node: Requires a WebGPU provider (webgpu npm package). Installed automatically as an optional dependency.

License

Apache License 2.0 (Apache-2.0). See LICENSE and NOTICE.