Vision-language model demos running entirely in the browser — no server, no Python, no GPU required.
Models are loaded from HuggingFace via Transformers.js and run using WebGPU (with automatic fallback to WASM).
Live site: https://pranavchokda.github.io/RunInBrowser/
| # | Demo | Description |
|---|---|---|
| 01 | SmolVLM | Upload an image and get a natural-language description from SmolVLM-256M-Instruct |
RunInBrowser/
├── .github/
│ └── workflows/
│ └── deploy.yml # Auto-deploy to GitHub Pages on push to main
├── examples/
│ └── 01_smolvlm/
│ ├── index.html # Demo page markup
│ ├── main.ts # Inference logic (model load, image processing, generation)
│ ├── style.css # UI styles
│ └── vite.demo.config.ts # Vite build config for this demo
├── dist/ # Production build output (generated, not committed)
├── index.html # Landing page linking to all demos
├── package.json
├── tsconfig.json
├── dev.sh # Helper script to start/stop dev servers
└── README.md
Each demo lives in its own examples/NN_name/ folder with its own Vite config, so they can be developed and built independently.
- Node.js 20+
- npm 10+
# 1. Install Node dependencies
npm install
# 2. Start the dev server
./dev.sh start
# 3. Open http://localhost:5173Or start the dev server directly with npm:
npm run dev./dev.sh start # Start dev server in the background (prints PID + URL)
./dev.sh stop # Stop the dev server
./dev.sh status # Check if the server is running
./dev.sh restart # Stop then startnpm run buildOutput goes to dist/. The GitHub Actions workflow runs this automatically on every push to main and deploys the result to GitHub Pages.
- Push the repo to GitHub (repo name:
RunInBrowser). - Go to Settings → Pages and set the source to GitHub Actions.
- Push to
main— the workflow in.github/workflows/deploy.ymlhandles the rest.
Note: If your repo name differs from
RunInBrowser, update thebasefield inexamples/01_smolvlm/vite.demo.config.tsto match.
- Load Model — weights are downloaded from HuggingFace (~300 MB for the 256M model) and cached in the browser's Cache Storage. Subsequent loads are instant.
- Runtime — inference runs via WebGPU when available, falling back to WebAssembly automatically. No data leaves your device.
- Usage — drop or upload an image, optionally edit the prompt, and click Describe.
This project is released under the MIT License.
- Hugging Face for Transformers.js and for hosting the SmolVLM model weights
- HuggingFaceTB for training and releasing SmolVLM-256M-Instruct
- Guido Zuidhof for
coi-serviceworker, which makes cross-origin isolation work on static hosts like GitHub Pages - Microsoft / ONNX Runtime team for ONNX Runtime Web, the inference engine underneath Transformers.js
- Vite for the fast dev/build tooling
- Claude (Anthropic) for assisting with the implementation, tooling setup, and documentation of this project
| Package | Author | License |
|---|---|---|
| @huggingface/transformers | Hugging Face | Apache 2.0 |
| SmolVLM-256M-Instruct | HuggingFaceTB | Apache 2.0 |
| onnxruntime-web | Microsoft | MIT |
| coi-serviceworker | Guido Zuidhof | MIT |
| vite | Evan You & Vite contributors | MIT |
Transformers.js
- Transformers.js docs — the library used for in-browser inference
- Hugging Face blog: Transformers.js — overview of v3 with WebGPU support
- SmolVLM model card — architecture and capabilities of the model used here
WebGPU
- MDN: WebGPU API — browser GPU compute API used for accelerated inference
- WebGPU spec — official W3C specification
ONNX Runtime Web
- ONNX Runtime Web docs — the inference engine underneath Transformers.js; explains WebGPU vs WASM execution providers and quantization formats (q4, fp16, etc.)
SharedArrayBuffer / Cross-Origin Isolation
- MDN: Cross-Origin Isolation guide — why COOP/COEP headers are required for
SharedArrayBuffer(multi-threaded WASM) - web.dev: Making your website cross-origin isolated — practical walkthrough of the headers
- coi-serviceworker — service worker workaround for hosts that don't support custom headers (used here for GitHub Pages)