cellpose-js

Browser-side cellular segmentation powered by Cellpose-SAM, running on WebGPU. Faithful TypeScript port of the Cellpose-SAM inference + dynamics pipeline, designed for in-browser microscopy workflows without a server round-trip.

Status: v0.1.0 — first end-to-end-working release. The full port from the implementation plan is complete: model loading + IndexedDB cache, preprocessing, WebGPU inference in a worker, tile averaging, flow dynamics, full-image label maps. SlimSAM-style compression and domain-specialized finetunes are out of scope — see the plan's §6 for the rationale.

Highlights

• Single-call API: await Cellpose.fromPretrained(modelUrl) → await cp.segment(image, opts) → a Uint32Array instance label map at source resolution.
• WebGPU inference via onnxruntime-web/webgpu. Measured ~277 ms / 256×256 tile on an M1 Max. Cold start ~2.3 s (one-time shader compile).
• Web Worker offload: inference doesn't block the UI thread; AbortSignal terminates the worker mid-run with sub-100 ms latency.
• Faithful Python parity for preprocess and dynamics — 14/14 vitest parity tests pass against numpy-generated .npy fixtures.
• IndexedDB cache for the 588 MB FP16 model: first visit fetches from your CDN; subsequent visits load from local storage in <2 s.

Browser requirements

• Chrome ≥135 (Feb 2025) or Safari ≥17.4. Native Float16Array is required to consume the FP16 ONNX graph IO.
• WebGPU available ('gpu' in navigator).
• onnxruntime-web ~1.26.0 as a peer dependency.

Older browsers fail fast with a clear UnsupportedEnvironmentError.

Install

npm install cellpose-js onnxruntime-web

You also need to host:

1. The model: cpsam_fp16.onnx (588 MB). Either upload to your own CDN, or use the public copy at https://huggingface.co/ballon999/cellpose-sam-onnx/resolve/main/cpsam_fp16.onnx.
2. ORT-web's WASM/JSEP sidecars: ORT dynamically imports .mjs and .wasm files at runtime. They must be served same-origin with your app (cross-origin dynamic import() is blocked). Either copy node_modules/onnxruntime-web/dist/ort-wasm-simd-threaded.{wasm,mjs,jsep.wasm,jsep.mjs,asyncify.wasm,asyncify.mjs} to your public assets, or proxy /ort/* to jsDelivr at build time (see examples/demo/vite.config.ts for the recipe).

Quickstart

import { Cellpose, configureOrt } from 'cellpose-js';

// One-time: tell ORT where to find its WASM sidecars.
configureOrt({ wasmPaths: '/ort/' });

// Load the model. Cached in IndexedDB after the first visit.
const cp = await Cellpose.fromPretrained(
  'https://your-cdn/cpsam_fp16.onnx',
  { preload: true }, // eager session create
);

// Segment an image.
const canvas = document.querySelector('canvas') as HTMLCanvasElement;
const ctx = canvas.getContext('2d')!;
const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);

const result = await cp.segment(
  { data: imageData.data, width: imageData.width, height: imageData.height, channels: 4 },
  {
    diameter: 30, // estimated cell diameter in source pixels (omit for native resolution)
    chan: 0, // primary channel (0 = grayscale)
    chan2: 0, // secondary channel (0 = none)
    dynamics: { cellprobThreshold: 0 }, // pixels above this enter the dynamical system
    onTileProgress: (done, total) => console.log(`tile ${done}/${total}`),
  },
);

console.log(`Found ${result.count} cells.`);
// result.masks      : Uint32Array — instance label map at source resolution, 0=background
// result.width      : number      — source image width
// result.height     : number      — source image height
// result.totalMs    : number      — wall-clock time for the segment() call
// result.tiles      : per-tile diagnostics (flow tensors, inference time)

Parameter quick-reference

chan / chan2

CPSAM was trained with channel-shuffling augmentation, so the choice rarely matters for segmentation quality. The legacy Cellpose 1–3 semantics are preserved:

Image type	chan	chan2
H&E histology, brightfield, phase contrast	0	0
Fluorescence: green cyto, blue nuclei	2	3
Fluorescence: red cyto, green nuclei	1	2
First run / unknown	0	0

diameter

Rescales the image so the median cell occupies ~30 px (CPSAM's training median). Omit to run at native resolution.

Cells in source image	Suggested
Roughly 20–60 px across	leave blank
Tiny (5–15 px)	≈ 10
Large (80+ px)	your visual estimate

Performance (M1 Max, Chrome 135+, WebGPU)

Step	Time	Notes
Model fetch (cold cache)	~5 s	588 MB from local proxy / CDN
Model fetch (warm IDB)	<100 ms	IndexedDB hit
ort.InferenceSession.create	~1.3 s	one-time per session
First inference (cold shader)	~2.3 s	one-time WebGPU shader compile
Steady-state per-tile inference	277 ms	256×256 FP16
Per-tile preprocessing	~14 ms amortized	normalize + tile copy
Full-image dynamics	74 ms (400×400)	average + Euler + cluster
Abort latency	<50 ms	next tile boundary

Architecture

input image → buildCpsamChannels → diameterResize → normalizePerChannel → makeTiles
                                                                              ⇣ (per tile, via worker)
                                                              ort.InferenceSession.run
                                                                              ⇣
                                                                       averageTiles
                                                                              ⇣
                                                                       computeMasks (Euler + cluster + renumber)
                                                                              ⇣
                                                                  (optional) nearest-neighbor unresize
                                                                              ⇣
                                                                        Uint32Array masks

See src/ for module-level documentation.

Testing

npm run test         # vitest: 14 parity tests against numpy fixtures
npm run typecheck    # tsc --noEmit
npm run build        # vite library build + tsc --emitDeclarationOnly
npm run demo         # vite serve examples/demo

The demo at examples/demo/ is a complete client that exercises the full pipeline. Point it at a local model file via examples/demo/public/cpsam_fp16.onnx (symlink), or change the URL in the Model field.

Troubleshooting

Symptom	Cause	Fix
e.getValue is not a function at session-create	Wrong ORT entry point	Import from onnxruntime-web/webgpu, not onnxruntime-web.
Failed to fetch dynamically imported module: …/ort-wasm-simd-threaded.asyncify.mjs	Cross-origin dynamic import blocked	Serve ORT WASM files same-origin (or proxy). See configureOrt({ wasmPaths }).
Float16Array is not defined	Browser too old	Chrome ≥135, Safari ≥17.4. No earlier polyfill is supported.
Operation aborted after AbortSignal fires	Working as intended	Worker terminates; next segment() call respawns from IDB cache (~150 ms).
Mask overlay has split cells at tile borders	Tile stitching off	Bug — file an issue. (M5 averaging should eliminate this.)

Credits

• Model and algorithm: Cellpose-SAM (Stringer et al., 2025). Original implementation: MouseLand/cellpose — BSD-3.
• Inference runtime: onnxruntime-web.

Security

See SECURITY.md for the threat model, reporting process, and recommended consumer practices.

License

MIT — see LICENSE.