diff --git a/README.md b/README.md
index 56e8008b5..3b5c9ceb3 100644
--- a/README.md
+++ b/README.md
@@ -20,11 +20,90 @@ a more modern dashboard, richer API workflows, sophisticated cache quantization,
 - Form a small cluster of Macs and split larger models across them.
 - Use a central model store so the cluster downloads once and stages locally.
 - Talk to the cluster through OpenAI Chat Completions, OpenAI Responses, Claude Messages, or Ollama-compatible APIs.
+- Push KV cache memory down hard with rotation-based 3-bit quantization (RotorQuant, OptiQ, TurboQuant) and pick a backend per workload from the dashboard.
+- Use a model-aware reasoning contract that handles toggleable and non-toggleable thinking models without baking assumptions into client code.
 - Experiment with advanced placement modes, RDMA, and KV cache backends when you are ready.
 - Run non-chat workloads such as embeddings and other specialized model flows.
 - Build TTS-oriented and other API-driven workflows on top of the cluster.
 - Actually use your cluster for real inference workloads instead of treating it as a demo.
 
+## Everything Different About Skulk
+
+This is the running list of where Skulk diverges from upstream [exo](https://github.com/exo-explore/exo). It is a living section — every meaningful change should land here when it ships, so anyone evaluating Skulk can see the surface area at a glance.
+
+### Inference and KV cache
+
+- **RotorQuant KV cache backend** — pure-MLX port of IsoQuant 3-bit (block-diagonal quaternion rotations + Lloyd-Max centroids) with **deferred prefill on Metal**, a contribution that does not exist in any upstream project (the llama.cpp fork ships it CUDA-only). GQA-native; no fallback for grouped-query models. See [docs/kv-cache-backends.md](docs/kv-cache-backends.md).
+- **TurboQuant native and adaptive backends** — randomized Hadamard rotation + Lloyd-Max centroids, with an adaptive variant that keeps edge attention layers in fp16 for accuracy.
+- **OptiQ KV cache integration** — wraps `mlx-optiq`'s rotated-space attention path so the rotation cost stays out of the per-token loop on supported (non-GQA) models.
+- **OptiQ mixed-precision weight quantization pipeline** — async wrapper around `mlx-optiq`'s sensitivity analysis and KL-divergence per-layer bit allocation, exposed as a model-store optimization job.
+- **KV prefix cache with snapshot/restore** — LRU-evicted prompt-prefix cache that snapshots SSM and rotating-window cache states so prefix matches are reusable across conversation turns even for hybrid Mamba/Transformer architectures.
+- **Pipeline-parallel prefill for short prompts** — pipelined models now route every prefill through the pipeline path, fixing prior warmup hangs on Gemma-class models.
+- **Force-sequential fallback** — quantized backends and Gemma 4 transparently fall back to a sequential generator when batch/history mode is incompatible with their cache layout.
+
+### Model capability system
+
+- **Two-layer capability model** — declarative `ModelCard` (with optional `reasoning`, `modalities`, `tooling`, and `runtime` sections) plus a normalized `ResolvedCapabilityProfile` derived from the card and conservative family defaults. This is the source of truth for prompt rendering, output parsing, tool-call handling, and the `/v1/models` `resolved_capabilities` field.
+- **Phase 2 thinking contract** — `enable_thinking`, `reasoning_effort`, and the dashboard thinking toggle are all driven by `supports_thinking_toggle`, so non-toggleable reasoning models behave correctly without leaking model-specific quirks into client code.
+- **Output parser selection** — model cards declare `output_parser` (`generic`, `gemma4`, `gpt_oss`, `deepseek_v32`, etc.), so reasoning markers are normalized into structured `reasoning_content` per family.
+- **Model store metadata pipeline** — capability resolution feeds `/v1/models` so dashboards and clients can discover thinking, multimodal, and tool support without hardcoding model lists.
+
+### API surface
+
+- **Claude Messages API** — `/v1/messages` adapter, including streaming, tool use, image inputs, and capability-aware thinking controls.
+- **Ollama compatibility** — both `/api/chat` and `/api/generate`, with adapter-side reasoning normalization.
+- **OpenAI Responses API** — `/v1/responses` adapter alongside chat completions.
+- **Embeddings endpoint** for non-chat workloads.
+- **Model store endpoints** — search, add, download, capability resolution, optimization jobs, and registry management, all exposed under stable URLs and documented in the OpenAPI spec.
+- **Cluster-wide config endpoints** — `GET`/`POST` config that gossipsubs to every node and writes back to `skulk.yaml`.
+- **Tracing, downloads, instance previews, and placement endpoints** — distributed-system observability and pre-launch placement inspection that upstream does not expose.
+
+### Dashboard
+
+- **React dashboard (default)** — replaces upstream's Svelte UI with a typed React + styled-components app that ships with the binary. The legacy Svelte dashboard is kept only as a fallback in the repo.
+- **Cluster topology view** with live device icons, GPU stats, network mesh visualization, and connection status banners.
+- **Placement preview / placement manager** for inspecting and choosing valid placements before launching.
+- **Model store browser** with HuggingFace search, family sidebar, model filters, capability badges, recent models, and per-model launch controls.
+- **Reasoning-aware chat UI** that splits inline `<think>` and Gemma `<|channel>` markers into a dedicated thinking panel and merges them with `reasoning_content` deltas from the API.
+- **Image attachments and multimodal chat affordances** for vision models.
+- **Cluster-wide settings panel** that writes to `skulk.yaml` and syncs across nodes via gossipsub.
+- **Light and dark themes** with first-class theme tokens, screenshots in both modes for documentation work.
+
+### Centralized logging and observability
+
+- **Structured JSON stdout** when `logging.enabled` is set, configurable from the dashboard Settings panel and synced cluster-wide.
+- **Vector + VictoriaLogs + Grafana stack** — local Vector log shipper on each node, central VictoriaLogs storage, ready-made Grafana dashboards. Stack definition lives in `deployment/logging/`.
+- **Distributed tracing** opt-in via `EXO_TRACING_ENABLED`.
+
+### Model store
+
+- **Centralized model store host** — one node downloads, the rest of the cluster stages over the LAN.
+- **Persistent registry** with capability resolution and download tracking.
+- **Custom model card support** — add your own model with `POST /models/add`.
+- **Image and embedding model cards** behind feature flags.
+- **Optimization job pipeline** for mlx-optiq mixed-precision weight quantization.
+
+### Cluster operation
+
+- **Cluster-wide settings sync** for KV cache backend, logging, model store host, HF token, and other inference toggles.
+- **Bootstrap peer config from `skulk.yaml`, env, or CLI** for fixed-topology clusters.
+- **Election (bully algorithm) + master/worker split** for indexing events and broadcasting state.
+- **`SKULK_*` environment variables** alongside the legacy `EXO_*` set, so new options can land without colliding with upstream.
+- **`skulk.yaml`** as the canonical config file, with `exo.yaml` kept for backwards compatibility.
+
+### Build, type system, and dev workflow
+
+- **Strict basedpyright** type checking — zero-error policy for new code.
+- **Ruff** linting and **`nix fmt`** formatting in CI.
+- **Nix flake** for reproducible toolchain setup.
+- **Docusaurus docs site** with auto-generated OpenAPI per-endpoint pages and TypeDoc HTML reference for the dashboard, both built from source.
+- **Pre-commit checklist** documented in `CLAUDE.md` and enforced in CI.
+
+### Hardware and platform
+
+- **Apple Silicon as the primary target**, including RDMA over Thunderbolt 5 on supported hardware and matched macOS versions.
+- **Linux supported** (CPU-oriented in this fork; GPU work happens on Apple Silicon).
+
 ## Prerequisites
 
 ### macOS
@@ -106,7 +185,9 @@ Important behavior:
 - **Placement previews**: inspect valid placements before launching a model.
 - **Thinking-aware chat UI**: chat with compatible models and surface reasoning content.
 - **Alternative API compatibility**: OpenAI Chat Completions, OpenAI Responses, Claude Messages, and Ollama.
-- **Experimental inference tuning**: OptiQ and other KV cache backends for long-context and memory experiments.
+- **Rotation-based KV cache backends**: RotorQuant (IsoQuant 3-bit + deferred prefill), OptiQ, TurboQuant, and TurboQuant Adaptive — pick per workload from the dashboard.
+- **Capability-driven thinking contract**: model cards declare reasoning support; the API and dashboard route accordingly.
+- **Experimental inference tuning**: long-context and memory experiments via the KV cache backends above.
 
 ## Dashboard
 
@@ -373,17 +454,27 @@ uv run exo --bootstrap-peers /ip4/192.168.1.20/tcp/5678/p2p/12D3KooW...
 | `EXO_NO_BATCH` | Force sequential generation | `false` |
 | `EXO_OPTIQ_BITS` | Bit width for `optiq` | `4` |
 | `EXO_OPTIQ_FP16_LAYERS` | Edge FP16 layers for `optiq` | `4` |
+| `SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT` | Enable experimental pure-MLX RotorQuant/IsoQuant cache backends | `false` |
+| `SKULK_ROTORQUANT_FP16_LAYERS` | Edge FP16 layers for `rotorquant_adaptive` | `4` |
+| `SKULK_ROTORQUANT_DEFER_PREFILL` | Set to `0` to disable deferred prefill (debugging only) | `1` |
 | `EXO_BOOTSTRAP_PEERS` | Comma-separated static peers to dial on startup | None |
 | `HF_TOKEN` | Hugging Face token | None |
 
 Examples:
 
 ```bash
-EXO_OFFLINE=true uv run exo
-EXO_ENABLE_IMAGE_MODELS=true uv run exo
-EXO_KV_CACHE_BACKEND=optiq EXO_OPTIQ_BITS=4 EXO_OPTIQ_FP16_LAYERS=4 uv run exo
+EXO_OFFLINE=true uv run skulk
+EXO_ENABLE_IMAGE_MODELS=true uv run skulk
+EXO_KV_CACHE_BACKEND=optiq EXO_OPTIQ_BITS=4 EXO_OPTIQ_FP16_LAYERS=4 uv run skulk
+SKULK_KV_CACHE_BACKEND=default SKULK_MLX_HANG_DEBUG=1 uv run skulk -vv
 ```
 
+The `rotorquant` and `rotorquant_adaptive` cache backends are experimental
+pure-MLX IsoQuant storage/dequant backends, not the fused RotorQuant+QJL
+implementation from the paper. They fall back to `default` unless
+`SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT=1` is also set, and should be used only
+for isolated cache experiments.
+
 ## RDMA on macOS
 
 RDMA is relevant only if you are building a multi-node Mac cluster on supported Thunderbolt 5 hardware.
diff --git a/dashboard-react/package-lock.json b/dashboard-react/package-lock.json
index a190905fa..c4cd8c2b8 100644
--- a/dashboard-react/package-lock.json
+++ b/dashboard-react/package-lock.json
@@ -48,7 +48,6 @@
         "redoc": "2.5.2",
         "storybook": "^10.3.3",
         "typedoc": "^0.28.14",
-        "typedoc-plugin-markdown": "^4.11.0",
         "typescript": "~5.9.3",
         "typescript-eslint": "^8.57.0",
         "vite": "^8.0.1",
@@ -7437,19 +7436,6 @@
         "typescript": "5.0.x || 5.1.x || 5.2.x || 5.3.x || 5.4.x || 5.5.x || 5.6.x || 5.7.x || 5.8.x || 5.9.x || 6.0.x"
       }
     },
-    "node_modules/typedoc-plugin-markdown": {
-      "version": "4.11.0",
-      "resolved": "https://registry.npmjs.org/typedoc-plugin-markdown/-/typedoc-plugin-markdown-4.11.0.tgz",
-      "integrity": "sha512-2iunh2ALyfyh204OF7h2u0kuQ84xB3jFZtFyUr01nThJkLvR8oGGSSDlyt2gyO4kXhvUxDcVbO0y43+qX+wFbw==",
-      "dev": true,
-      "license": "MIT",
-      "engines": {
-        "node": ">= 18"
-      },
-      "peerDependencies": {
-        "typedoc": "0.28.x"
-      }
-    },
     "node_modules/typedoc/node_modules/balanced-match": {
       "version": "4.0.4",
       "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz",
diff --git a/dashboard-react/src/components/layout/SettingsPanel.tsx b/dashboard-react/src/components/layout/SettingsPanel.tsx
index 6b080cf94..a66b93b23 100644
--- a/dashboard-react/src/components/layout/SettingsPanel.tsx
+++ b/dashboard-react/src/components/layout/SettingsPanel.tsx
@@ -449,16 +449,22 @@ export function SettingsPanel({ open, onClose }: SettingsPanelProps) {
                 filled
                 content={
                   `• Default — No cache quantization. Best baseline quality, highest memory use.\n` +
-                  `• OptiQ — Rotation-based quantization via mlx-optiq. Best long-context quality.\n` +
+                  `• OptiQ — Rotation-based quantization via mlx-optiq. Good long-context quality, no GQA support.\n` +
                   `• TurboQuant Adaptive — Quantizes middle KV layers, keeps edge layers in FP16. Proven stable.\n` +
-                  `• TurboQuant — Quantizes all KV layers. Most aggressive compression, higher quality risk.\n` +
+                  `• TurboQuant — Quantizes all KV layers. Most aggressive non-OptiQ compression.\n` +
                   `• MLX Quantized — MLX's built-in cache quantization.\n\n` +
-                  `Takes effect on next model launch. Incompatible models fall back to Default automatically.`
+                  `RotorQuant/IsoQuant is hidden from normal settings because it is experimental and must be enabled with SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT=1.`
                 }
               />
             </FieldLabel>
             <Select value={kvBackend} onChange={(e) => setKvBackend(e.target.value)} disabled={!!envOverride}>
               <option value="default">Default (no quantization)</option>
+              {kvBackend === 'rotorquant_adaptive' ? (
+                <option value="rotorquant_adaptive">RotorQuant Adaptive (experimental, env-gated)</option>
+              ) : null}
+              {kvBackend === 'rotorquant' ? (
+                <option value="rotorquant">RotorQuant (experimental, env-gated)</option>
+              ) : null}
               <option value="optiq">OptiQ (rotation-based)</option>
               <option value="turboquant_adaptive">TurboQuant Adaptive</option>
               <option value="turboquant">TurboQuant</option>
@@ -467,7 +473,7 @@ export function SettingsPanel({ open, onClose }: SettingsPanelProps) {
             {envOverride ? (
               <HintText>Overridden by SKULK_KV_CACHE_BACKEND environment variable. Remove the env var to configure here.</HintText>
             ) : (
-              <HintText>Changes take effect on the next model launch. Models with incompatible architectures (GQA, non-power-of-two head_dim) will automatically fall back to default.</HintText>
+              <HintText>Changes take effect on the next model launch. RotorQuant is experimental and intentionally unavailable from normal settings.</HintText>
             )}
           </Fieldset>
 
diff --git a/docs/kv-cache-backends.md b/docs/kv-cache-backends.md
index 1c21b8644..d2fc3520d 100644
--- a/docs/kv-cache-backends.md
+++ b/docs/kv-cache-backends.md
@@ -11,6 +11,8 @@ Skulk includes several opt-in KV cache backends for MLX text generation. These b
 - `turboquant`: correctness-first TurboQuant-inspired KV cache for standard `KVCache` layers
 - `turboquant_adaptive`: keeps outer KV layers in FP16 and applies TurboQuant to middle KV layers
 - `optiq`: **[NEW]** rotation-based KV cache via [mlx-optiq](https://mlx-optiq.pages.dev/) — uses randomized orthogonal rotations with Lloyd-Max quantization and rotated-space attention for superior long-context quality
+- `rotorquant`: **experimental, gated** pure-MLX IsoQuant-style storage/dequant cache. This is not the fused RotorQuant+QJL implementation from the RotorQuant paper.
+- `rotorquant_adaptive`: **experimental, gated** as above with FP16 protection on the first/last N attention layers.
 
 If `SKULK_KV_CACHE_BACKEND` is unset, or is set to `default`, Skulk behaves as before.
 
@@ -39,6 +41,23 @@ uv run skulk
 
 This mode keeps the first and last 4 KV layers in normal FP16-style cache and applies TurboQuant only to the middle KV layers. Proven stable across most models.
 
+### RotorQuant / IsoQuant (experimental only)
+
+The `rotorquant` names are intentionally gated behind an explicit opt-in:
+
+```bash
+SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT=1 \
+SKULK_KV_CACHE_BACKEND=rotorquant_adaptive \
+SKULK_ROTORQUANT_FP16_LAYERS=4 \
+uv run skulk
+```
+
+This backend is a pure-MLX IsoQuant-style cache that compresses storage and
+then returns fully dequantized fp16 K/V to normal MLX attention. It does not
+implement RotorQuant's fused Metal/CUDA attention path or QJL residual
+correction, so it should not be used as the default distributed inference
+baseline.
+
 ## Available Environment Variables
 
 | Variable | Backends | Default | Description |
@@ -50,6 +69,9 @@ This mode keeps the first and last 4 KV layers in normal FP16-style cache and ap
 | `SKULK_TQ_K_BITS` | `turboquant`, `turboquant_adaptive` | `3` | Key quantization bits |
 | `SKULK_TQ_V_BITS` | `turboquant`, `turboquant_adaptive` | `4` | Value quantization bits |
 | `SKULK_TQ_FP16_LAYERS` | `turboquant_adaptive` | `4` | Edge layers kept in FP16 |
+| `SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT` | `rotorquant`, `rotorquant_adaptive` | `0` | Required opt-in for experimental RotorQuant/IsoQuant backends |
+| `SKULK_ROTORQUANT_FP16_LAYERS` | `rotorquant_adaptive` | `4` | Edge layers kept in FP16 |
+| `SKULK_ROTORQUANT_DEFER_PREFILL` | `rotorquant`, `rotorquant_adaptive` | `1` | Set to `0` to disable deferred prefill while debugging |
 
 ## Invocation Examples
 
@@ -86,10 +108,12 @@ SKULK_KV_CACHE_BACKEND=turboquant_adaptive SKULK_TQ_K_BITS=3 SKULK_TQ_V_BITS=4 S
 | `turboquant_adaptive` | Low | Good | Moderate | Proven stable, Hadamard-based |
 | `turboquant` | Lowest | Variable | Moderate | Most aggressive compression |
 | `mlx_quantized` | Low | Good | Moderate | MLX built-in quantization |
+| `rotorquant_adaptive` | Low | Experimental | Experimental | Gated pure-MLX IsoQuant storage/dequant cache |
+| `rotorquant` | Lowest | Experimental | Experimental | Gated pure-MLX IsoQuant storage/dequant cache |
 
 ## Supported Cache Layouts
 
-All quantized backends (optiq, turboquant, mlx_quantized) compress only standard `KVCache` entries and preserve these cache types unchanged:
+All quantized backends (optiq, turboquant, mlx_quantized, and the gated rotorquant variants) compress only standard `KVCache` entries and preserve these cache types unchanged:
 
 - `ArraysCache`
 - `RotatingKVCache`
@@ -103,8 +127,10 @@ Mixed cache layouts are supported:
 ## Current Limitations
 
 - All quantized KV cache backends force sequential generation (no batch/history mode)
+- Gemma 4 text generation also forces sequential generation for now because distributed BatchGenerator mode can produce degenerate repetition with its sliding-window cache layout
 - The optiq backend requires `mlx-optiq` to be installed (`pip install mlx-optiq`)
 - The optiq backend's `patch_attention()` monkey-patches MLX's SDPA — avoid switching between optiq and other backends within the same process lifetime without a restart
+- The rotorquant backends are disabled unless `SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT=1` is set. If selected without the gate, Skulk falls back to `default`.
 
 ## About mlx-optiq
 
diff --git a/src/exo/api/main.py b/src/exo/api/main.py
index 7e68b1f7f..af74fcb9a 100644
--- a/src/exo/api/main.py
+++ b/src/exo/api/main.py
@@ -204,6 +204,7 @@
 from exo.worker.engines.mlx.constants import (
     DEFAULT_KV_CACHE_BACKEND,
     VALID_KV_CACHE_BACKENDS,
+    resolve_kv_cache_backend,
 )
 
 if TYPE_CHECKING:
@@ -1197,8 +1198,7 @@ async def _send_text_generation_with_images(
             )
         for idx, h in cached_hashes.items():
             _log_image_transport(
-                f"TextGeneration cached image {idx}: "
-                f"b64_sha256={h[:12]}..."
+                f"TextGeneration cached image {idx}: b64_sha256={h[:12]}..."
             )
 
         if not new_images:
@@ -2214,9 +2214,7 @@ async def get_models(self, status: str | None = Query(default=None)) -> ModelLis
                         downloaded_model_ids.add(dl.shard_metadata.model_card.model_id)
             cards = [c for c in cards if c.model_id in downloaded_model_ids]
 
-        return ModelList(
-            data=[self._model_list_entry(card) for card in cards]
-        )
+        return ModelList(data=[self._model_list_entry(card) for card in cards])
 
     async def add_custom_model(self, payload: AddCustomModelParams) -> ModelListModel:
         """Fetch a model from HuggingFace and save as a custom model card, then sync across the cluster."""
@@ -2580,7 +2578,7 @@ def _effective_kv_cache_backend(self) -> str:
 
         if configured_backend not in VALID_KV_CACHE_BACKENDS:
             return DEFAULT_KV_CACHE_BACKEND
-        return configured_backend
+        return resolve_kv_cache_backend(configured_backend)
 
     async def get_config(self) -> JSONResponse:
         if not self._config_path.exists():
@@ -2678,7 +2676,9 @@ async def update_config(self, request: Request) -> JSONResponse:
             log_on = bool(logging_cfg_update.get("enabled", False)) and bool(
                 logging_cfg_update.get("ingest_url")
             )
-            set_structured_stdout(log_on, ingest_url=str(logging_cfg_update.get("ingest_url", "")))
+            set_structured_stdout(
+                log_on, ingest_url=str(logging_cfg_update.get("ingest_url", ""))
+            )
         # model_store changes still require restart; inference-only changes don't
         has_store_changes = "model_store" in config_data
         return JSONResponse(
diff --git a/src/exo/api/tests/test_config_api.py b/src/exo/api/tests/test_config_api.py
index 161ac5208..34e177c11 100644
--- a/src/exo/api/tests/test_config_api.py
+++ b/src/exo/api/tests/test_config_api.py
@@ -60,7 +60,9 @@ def test_get_config_treats_blank_skulk_kv_backend_as_default(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> None:
     config_path = tmp_path / "exo.yaml"
-    config_path.write_text("inference:\n  kv_cache_backend: default\n", encoding="utf-8")
+    config_path.write_text(
+        "inference:\n  kv_cache_backend: default\n", encoding="utf-8"
+    )
     monkeypatch.setenv("SKULK_KV_CACHE_BACKEND", "")
     monkeypatch.setenv("EXO_KV_CACHE_BACKEND", "optiq")
 
@@ -79,7 +81,9 @@ def test_get_config_treats_invalid_skulk_kv_backend_as_default(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> None:
     config_path = tmp_path / "exo.yaml"
-    config_path.write_text("inference:\n  kv_cache_backend: default\n", encoding="utf-8")
+    config_path.write_text(
+        "inference:\n  kv_cache_backend: default\n", encoding="utf-8"
+    )
     monkeypatch.setenv("SKULK_KV_CACHE_BACKEND", "typo-backend")
 
     api = _build_api()
@@ -91,3 +95,24 @@ def test_get_config_treats_invalid_skulk_kv_backend_as_default(
     assert response.status_code == 200
     data: dict[str, Any] = response.json()
     assert data["effective"]["kv_cache_backend"] == "default"
+
+
+def test_get_config_treats_ungated_rotorquant_as_default(
+    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    config_path = tmp_path / "exo.yaml"
+    config_path.write_text(
+        "inference:\n  kv_cache_backend: default\n", encoding="utf-8"
+    )
+    monkeypatch.setenv("SKULK_KV_CACHE_BACKEND", "rotorquant_adaptive")
+    monkeypatch.setenv("SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT", "")
+
+    api = _build_api()
+    api._config_path = config_path  # pyright: ignore[reportPrivateUsage]
+    client = TestClient(api.app)
+
+    response = client.get("/config")
+
+    assert response.status_code == 200
+    data: dict[str, Any] = response.json()
+    assert data["effective"]["kv_cache_backend"] == "default"
diff --git a/src/exo/download/coordinator.py b/src/exo/download/coordinator.py
index 5e0b7e1f2..b9dadd546 100644
--- a/src/exo/download/coordinator.py
+++ b/src/exo/download/coordinator.py
@@ -181,24 +181,48 @@ async def _restart_node(self) -> None:
 
     async def _sync_config(self, config_yaml: str) -> None:
         """Write received config YAML to the local config file and
-        apply runtime-effective settings (e.g., KV cache backend)."""
+        apply runtime-effective settings (e.g., KV cache backend).
+
+        When the user set ``SKULK_KV_CACHE_BACKEND`` at launch, their
+        value is the source of truth. The cluster sync must not
+        overwrite it — neither in ``os.environ`` nor in the config
+        file.
+        """
         config_path = resolve_config_path()
         try:
+            import yaml
+
+            raw = yaml.safe_load(config_yaml)
+            user_set_kv = bool(
+                os.environ.get("_SKULK_KV_BACKEND_USER_SET")
+                or os.environ.get("_EXO_KV_BACKEND_USER_SET")
+            )
+
+            # Preserve the user's KV backend in the config file when
+            # the env var was set at launch — don't let cluster sync
+            # clobber it.
+            if user_set_kv and raw and isinstance(raw, dict):
+                local_backend = os.environ.get("SKULK_KV_CACHE_BACKEND")
+                if local_backend:
+                    inference = raw.get("inference")
+                    if isinstance(inference, dict):
+                        inference["kv_cache_backend"] = local_backend
+                    else:
+                        raw["inference"] = {"kv_cache_backend": local_backend}
+                    config_yaml = yaml.safe_dump(
+                        raw, default_flow_style=False, sort_keys=False
+                    )
+
             config_path.write_text(config_yaml)
             logger.info(
                 f"DownloadCoordinator: synced {config_path.name} from cluster ({len(config_yaml)} bytes)"
             )
-            # Apply inference config to env var so next runner spawn picks it up
-            import yaml
 
-            raw = yaml.safe_load(config_yaml)
+            # Apply inference config to env var so next runner spawn picks it up
             if raw and isinstance(raw, dict):
                 inference = raw.get("inference")
                 if isinstance(inference, dict) and "kv_cache_backend" in inference:
-                    # Don't overwrite if user provided the env var at launch
-                    if not os.environ.get(
-                        "_SKULK_KV_BACKEND_USER_SET"
-                    ) and not os.environ.get("_EXO_KV_BACKEND_USER_SET"):
+                    if not user_set_kv:
                         os.environ["SKULK_KV_CACHE_BACKEND"] = str(
                             inference["kv_cache_backend"]
                         )
@@ -210,7 +234,8 @@ async def _sync_config(self, config_yaml: str) -> None:
                         )
                     else:
                         logger.info(
-                            "DownloadCoordinator: skipping KV backend update (user env var override active)"
+                            "DownloadCoordinator: keeping user-set KV_CACHE_BACKEND="
+                            f"{os.environ.get('SKULK_KV_CACHE_BACKEND', '?')}"
                         )
                 # Apply HF token if not user-set
                 hf_token = raw.get("hf_token")
diff --git a/src/exo/main.py b/src/exo/main.py
index ddaf078bc..7b2aceb24 100644
--- a/src/exo/main.py
+++ b/src/exo/main.py
@@ -29,6 +29,7 @@
     load_exo_config,
     resolve_config_path,
     resolve_node_staging,
+    update_config_field,
 )
 from exo.store.model_store import ModelStore
 from exo.store.model_store_client import ModelStoreClient, ModelStoreDownloader
@@ -98,12 +99,26 @@ async def create(cls, args: "Args") -> Self:
             "1" if _user_set_kv_backend else ""
         )  # legacy compat
 
-        # Apply inference config to env var so runner subprocesses inherit it.
-        # Env var takes precedence if user set it at launch.
-        if (
+        # Env var is the source of truth for KV backend. When set at launch,
+        # write it back to the config file so they stay in sync. When not
+        # set, apply the config file value to the env var so runner
+        # subprocesses inherit it.
+        if _user_set_kv_backend:
+            launch_backend = os.environ.get(
+                "SKULK_KV_CACHE_BACKEND",
+                os.environ.get("EXO_KV_CACHE_BACKEND", ""),
+            )
+            if launch_backend:
+                if update_config_field("inference", "kv_cache_backend", launch_backend):
+                    logger.info(
+                        f"Synced launch env KV backend to config: kv_cache_backend={launch_backend}"
+                    )
+                # Ensure both env vars are in sync
+                os.environ["SKULK_KV_CACHE_BACKEND"] = launch_backend
+                os.environ["EXO_KV_CACHE_BACKEND"] = launch_backend
+        elif (
             exo_config is not None
             and exo_config.inference is not None
-            and not _user_set_kv_backend
         ):
             os.environ["SKULK_KV_CACHE_BACKEND"] = exo_config.inference.kv_cache_backend
             os.environ["EXO_KV_CACHE_BACKEND"] = (
diff --git a/src/exo/store/config.py b/src/exo/store/config.py
index 54a778f78..7c0b617a2 100644
--- a/src/exo/store/config.py
+++ b/src/exo/store/config.py
@@ -235,10 +235,38 @@ class InferenceConfig(FrozenModel):
     """
 
     kv_cache_backend: Literal[
-        "default", "mlx_quantized", "turboquant", "turboquant_adaptive", "optiq"
+        "default",
+        "mlx_quantized",
+        "turboquant",
+        "turboquant_adaptive",
+        "optiq",
+        "rotorquant",
+        "rotorquant_adaptive",
     ] = "default"
 
 
+def update_config_field(section: str, key: str, value: object) -> bool:
+    """Update a single field in the config file, preserving all other content.
+
+    Reads the raw YAML, patches ``raw[section][key] = value``, and writes
+    it back.  Returns ``True`` if the file was updated, ``False`` if no
+    config file exists.
+    """
+    path = resolve_config_path()
+    if not path.exists():
+        return False
+    with path.open() as f:
+        raw: dict[str, object] = yaml.safe_load(f) or {}
+    sec = raw.get(section)
+    if not isinstance(sec, dict):
+        sec = {}
+        raw[section] = sec
+    sec[key] = value
+    with path.open("w") as f:
+        yaml.dump(raw, f, default_flow_style=False, sort_keys=False)
+    return True
+
+
 def resolve_config_path() -> Path:
     """Find the config file, preferring ``skulk.yaml`` over legacy ``exo.yaml``."""
     skulk = Path("skulk.yaml")
diff --git a/src/exo/worker/engines/mlx/auto_parallel.py b/src/exo/worker/engines/mlx/auto_parallel.py
index 6094bb2d8..fc4833c7a 100644
--- a/src/exo/worker/engines/mlx/auto_parallel.py
+++ b/src/exo/worker/engines/mlx/auto_parallel.py
@@ -192,6 +192,46 @@ def watchdog() -> None:
         completed.set()
 
 
+def _cache_dep_anchor(cache: object) -> mx.array | None:
+    """Return an array from ``cache`` suitable for ``mx.depends`` ordering.
+
+    Standard ``KVCache`` exposes ``.keys``; quantized caches like
+    ``RotorQuantKVCache`` do not.  Fall back to the first leaf of
+    ``.state`` so we can order cache evaluation relative to distributed
+    sends on *any* cache backend.
+    """
+    # Fast path: mlx-lm KVCache and friends
+    keys = getattr(cache, "keys", None)
+    if isinstance(keys, mx.array):
+        return keys
+
+    # Generic path: first array in cache.state
+    state: object = getattr(cache, "state", None)
+    if isinstance(state, (list, tuple)):
+        for leaf in list(cast(list[object], state)):
+            if isinstance(leaf, mx.array):
+                return leaf
+    return None
+
+
+def _set_cache_dep_anchor(cache: object, value: mx.array) -> None:
+    """Write back a dependency-updated anchor array to ``cache``.
+
+    Mirrors :func:`_cache_dep_anchor`: if the cache has ``.keys`` we set
+    it directly; otherwise we update the first matching private attribute
+    from the state pytree.
+    """
+    if hasattr(cache, "keys") and isinstance(getattr(cache, "keys", None), mx.array):
+        cache.keys = value  # type: ignore[attr-defined]
+        return
+
+    # Walk common private attributes that back the state pytree.
+    for attr in ("_key_indices", "_pending_keys"):
+        if hasattr(cache, attr) and isinstance(getattr(cache, attr, None), mx.array):
+            setattr(cache, attr, value)
+            return
+
+
 class _LayerCallable(Protocol):
     """Structural type that any compatible layer must satisfy.
 
@@ -302,31 +342,51 @@ def __call__(self, x: mx.array, *args: object, **kwargs: object) -> mx.array:
                     output = mx.distributed.send(
                         output, (self.r + 1) % self.s, group=self.group
                     )
+            _dep_array: mx.array | None = None
             if cache is not None:
                 # CacheList (used by MLA models like DeepSeekV32, GLM MoE DSA)
                 # doesn't have .keys directly; access via first sub-cache.
-                _cache = cache[0] if hasattr(cache, "caches") else cache  # type: ignore
-                if hasattr(_cache, "keys"):  # pyright: ignore[reportAny]
-                    _cache.keys = mx.depends(_cache.keys, output)  # type: ignore
+                _cache: object = cache[0] if hasattr(cache, "caches") else cache  # type: ignore[index]
+                _dep_array = _cache_dep_anchor(_cache)
+                if _dep_array is not None:
+                    _set_cache_dep_anchor(_cache, mx.depends(_dep_array, output))
             with _hang_debug_watch(
                 f"pipeline_last eval_send rank={self.r} dst={(self.r + 1) % self.s} prefill={self.is_prefill}"
             ):
                 mx.eval(output)
-            if cache is not None and hasattr(_cache, "keys"):  # type: ignore
+            if _dep_array is not None:
                 with _hang_debug_watch(
                     f"pipeline_last eval_cache_dep rank={self.r} dst={(self.r + 1) % self.s} prefill={self.is_prefill}"
                 ):
-                    mx.eval(_cache.keys)  # type: ignore
+                    mx.eval(_cache_dep_anchor(_cache))  # type: ignore[possibly-undefined]
 
         if not self.is_prefill:
+            # Broadcast the last rank's output to all other ranks so every
+            # rank can apply lm_head and sample the same token.
+            #
+            # Uses all_sum instead of point-to-point send/recv: the last rank
+            # contributes its real output while other ranks contribute zeros.
+            # This avoids JACCL send/recv corruption (mlx#3149) that caused
+            # subsequent all_gather collectives to return NaN/garbage after
+            # warmup — and unlike all_gather, orphaned prefetch steps from
+            # generate_step's one-step-ahead prefetch can't deadlock because
+            # all_sum is commutative and tolerates collective reordering.
+            #
+            # The all_sum runs on the CPU stream rather than the default GPU
+            # stream to avoid a JACCL hang observed on 3-node pipelines when
+            # send/recv and all_sum share the same GPU stream.  CPU-stream
+            # collectives (used by mx_barrier) are proven reliable; this
+            # isolates the broadcast from pipeline point-to-point traffic.
+            contribution = output if self.r == self.s - 1 else mx.zeros_like(output)
+            cpu_stream = mx.default_stream(mx.Device(mx.cpu))
             with _hang_debug_watch(
-                f"pipeline_last all_gather rank={self.r} world={self.s}"
+                f"pipeline_last all_sum_broadcast rank={self.r} world={self.s}"
             ):
-                output = mx.distributed.all_gather(output, group=self.group)[
-                    -output.shape[0] :
-                ]
+                output = mx.distributed.all_sum(
+                    contribution, group=self.group, stream=cpu_stream
+                )
             with _hang_debug_watch(
-                f"pipeline_last eval_all_gather rank={self.r} world={self.s}"
+                f"pipeline_last eval_all_sum_broadcast rank={self.r} world={self.s}"
             ):
                 mx.eval(output)
 
@@ -564,11 +624,16 @@ def patched_call(
             # directly, so pipeline-parallel Gemma 4 needs this patch on both
             # the top-level model and the nested language model.
             if cache is not None:
-                logits_array = logits.logits if hasattr(logits, "logits") else logits  # type: ignore[attr-defined]
-                last = cache[-1]  # type: ignore[index]
-                dep_cache = last[0] if hasattr(last, "caches") else last  # type: ignore[index]
-                if hasattr(dep_cache, "keys") and dep_cache.keys is not None:  # type: ignore[attr-defined]
-                    dep_cache.keys = mx.depends(dep_cache.keys, logits_array)  # type: ignore[attr-defined]
+                logits_array: mx.array = (
+                    logits.logits if hasattr(logits, "logits") else logits
+                )  # type: ignore[attr-defined]
+                last = cast(object, cache[-1])
+                dep_cache = cast(object, last[0] if hasattr(last, "caches") else last)  # pyright: ignore[reportIndexIssue]
+                anchor = _cache_dep_anchor(dep_cache)
+                if anchor is not None:
+                    _set_cache_dep_anchor(
+                        dep_cache, mx.depends(anchor, cast(mx.array, logits_array))
+                    )
 
             return logits
 
diff --git a/src/exo/worker/engines/mlx/cache.py b/src/exo/worker/engines/mlx/cache.py
index 2e4c4859d..8fc68c44c 100644
--- a/src/exo/worker/engines/mlx/cache.py
+++ b/src/exo/worker/engines/mlx/cache.py
@@ -12,6 +12,7 @@
 )
 from mlx_lm.tokenizer_utils import TokenizerWrapper
 
+from exo.shared.constants import preferred_env_value
 from exo.shared.types.memory import Memory
 from exo.shared.types.mlx import KVCacheType, Model
 from exo.worker.engines.mlx.constants import (
@@ -19,16 +20,25 @@
     DEFAULT_KV_CACHE_BACKEND,
     DEFAULT_TURBOQUANT_K_BITS,
     DEFAULT_TURBOQUANT_V_BITS,
-    KV_CACHE_BACKEND,
+    EXPERIMENTAL_ROTORQUANT_BACKENDS,
     KV_CACHE_BITS,
     OPTIQ_BITS,
     OPTIQ_FP16_LAYERS,
+    ROTORQUANT_DEFER_PREFILL,
+    ROTORQUANT_FP16_LAYERS,
     TURBOQUANT_FP16_LAYERS,
     TURBOQUANT_K_BITS,
     TURBOQUANT_V_BITS,
     VALID_KV_CACHE_BACKENDS,
     KVCacheBackend,
+    experimental_rotorquant_enabled,
+    resolve_kv_cache_backend,
 )
+from exo.worker.engines.mlx.rotorquant import (
+    make_rotorquant_adaptive_cache,
+    make_rotorquant_cache_from_template,
+)
+from exo.worker.engines.mlx.rotorquant.tables import ISO3_BLOCK_SIZE
 from exo.worker.engines.mlx.turboquant import (
     make_turboquant_adaptive_cache,
     make_turboquant_cache_from_template,
@@ -337,17 +347,28 @@ def _evict_if_needed(self):
             )
 
     def get_memory_used_percentage(self) -> float:
+        """Return the maximum memory pressure across all ranks.
+
+        Uses all_sum with slotted contributions instead of all_gather.
+        JACCL all_gather deadlocks intermittently on 3-node pipelines;
+        all_sum is proven reliable.
+        """
         local_pressure: float = get_memory_used_percentage()
 
         if self._group is None:
             return local_pressure
 
-        all_pressure = mx.distributed.all_gather(
-            mx.array([local_pressure], dtype=mx.float32),
+        world_size = self._group.size()
+        rank = self._group.rank()
+        slots = [0.0] * world_size
+        slots[rank] = local_pressure
+        cpu_stream = mx.default_stream(mx.Device(mx.cpu))
+        merged = mx.distributed.all_sum(
+            mx.array(slots, dtype=mx.float32),
             group=self._group,
+            stream=cpu_stream,
         )
-        # .item() evals.
-        max_pressure = float(mx.max(all_pressure).item())
+        max_pressure = float(mx.max(merged).item())
         return max_pressure
 
 
@@ -417,6 +438,13 @@ def get_memory_used_percentage() -> float:
     return float(mem.percent / 100)
 
 
+def _make_default_cache(model: Model) -> KVCacheType:
+    """Build the default (unquantized) KV cache for a model."""
+    if hasattr(model, "make_cache"):
+        return model.make_cache()  # type: ignore
+    return [KVCache() for _ in model.layers]
+
+
 def make_kv_cache(
     model: Model, max_kv_size: int | None = None, keep: int = 0
 ) -> KVCacheType:
@@ -579,19 +607,85 @@ def _is_power_of_two(n: int) -> bool:
             for i, _ in enumerate(model.layers)
         ]
 
-    if hasattr(model, "make_cache"):
-        logger.info("Using MLX LM's make cache")
-        return model.make_cache()  # type: ignore
+    if backend in ("rotorquant", "rotorquant_adaptive"):
+        # Preflight: RotorQuant requires head_dim divisible by the
+        # IsoQuant block size (128). Models with smaller heads (e.g. 64-d)
+        # would crash at first token; fall back to default instead.
+        if len(model.layers) > 0:
+            first_layer = model.layers[0]
+            attn = getattr(first_layer, "self_attn", first_layer)
+            model_head_dim: int = getattr(attn, "head_dim", ISO3_BLOCK_SIZE)
+        else:
+            model_head_dim = ISO3_BLOCK_SIZE
+        if model_head_dim % ISO3_BLOCK_SIZE != 0:
+            logger.warning(
+                f"RotorQuant requires head_dim divisible by {ISO3_BLOCK_SIZE}, "
+                f"but this model has head_dim={model_head_dim}; "
+                f"falling back to default KV cache"
+            )
+            return _make_default_cache(model)
+
+        if backend == "rotorquant":
+            logger.info(
+                f"Using rotorquant KV cache (defer_prefill={ROTORQUANT_DEFER_PREFILL})"
+            )
+            return make_rotorquant_cache_from_template(
+                model,
+                defer_prefill=ROTORQUANT_DEFER_PREFILL,
+            )
+
+        logger.info(
+            f"Using rotorquant adaptive KV cache "
+            f"(fp16_layers={ROTORQUANT_FP16_LAYERS}, "
+            f"defer_prefill={ROTORQUANT_DEFER_PREFILL})"
+        )
+        return make_rotorquant_adaptive_cache(
+            model,
+            fp16_layers=ROTORQUANT_FP16_LAYERS,
+            defer_prefill=ROTORQUANT_DEFER_PREFILL,
+        )
 
     logger.info("Using default KV cache")
-    return [KVCache() for _ in model.layers]
+    return _make_default_cache(model)
 
 
 def get_kv_cache_backend() -> KVCacheBackend:
-    backend = KV_CACHE_BACKEND
+    """Return the active KV cache backend, reading ``os.environ`` at call time.
+
+    The env var is the source of truth — it can be set at launch, updated
+    by config sync via gossipsub, or changed from the dashboard.  Reading
+    it dynamically (instead of the frozen import-time constant) ensures
+    that runtime updates are always visible to the runner.
+    """
+    configured_backend = (
+        preferred_env_value(
+            "SKULK_KV_CACHE_BACKEND", "EXO_KV_CACHE_BACKEND", DEFAULT_KV_CACHE_BACKEND
+        )
+        or DEFAULT_KV_CACHE_BACKEND
+    )
+    backend = resolve_kv_cache_backend(configured_backend)
+
+    if configured_backend not in VALID_KV_CACHE_BACKENDS:
+        logger.warning(
+            f"Unknown KV_CACHE_BACKEND={configured_backend!r}; "
+            f"falling back to {DEFAULT_KV_CACHE_BACKEND!r}"
+        )
+        return backend
+
     if backend not in VALID_KV_CACHE_BACKENDS:
         logger.warning(
-            f"Unknown EXO_KV_CACHE_BACKEND={backend!r}; falling back to {DEFAULT_KV_CACHE_BACKEND!r}"
+            f"Unknown KV_CACHE_BACKEND={backend!r}; falling back to {DEFAULT_KV_CACHE_BACKEND!r}"
         )
         return DEFAULT_KV_CACHE_BACKEND
+
+    if (
+        configured_backend in EXPERIMENTAL_ROTORQUANT_BACKENDS
+        and not experimental_rotorquant_enabled()
+    ):
+        logger.warning(
+            f"KV_CACHE_BACKEND={configured_backend!r} requested, but RotorQuant is "
+            "an experimental pure-MLX IsoQuant storage/dequant backend. Falling "
+            "back to 'default'. Set SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT=1 to "
+            "force this backend for isolated testing."
+        )
     return backend
diff --git a/src/exo/worker/engines/mlx/constants.py b/src/exo/worker/engines/mlx/constants.py
index 87d039542..ee80c4621 100644
--- a/src/exo/worker/engines/mlx/constants.py
+++ b/src/exo/worker/engines/mlx/constants.py
@@ -1,5 +1,5 @@
 import os
-from typing import Literal, cast
+from typing import Literal
 
 from exo.shared.constants import preferred_env_value
 
@@ -23,6 +23,8 @@
     "turboquant",
     "turboquant_adaptive",
     "optiq",
+    "rotorquant",
+    "rotorquant_adaptive",
 ]
 DEFAULT_KV_CACHE_BACKEND: KVCacheBackend = "default"
 VALID_KV_CACHE_BACKENDS: tuple[KVCacheBackend, ...] = (
@@ -31,16 +33,54 @@
     "turboquant",
     "turboquant_adaptive",
     "optiq",
+    "rotorquant",
+    "rotorquant_adaptive",
 )
+EXPERIMENTAL_ROTORQUANT_BACKENDS: tuple[KVCacheBackend, ...] = (
+    "rotorquant",
+    "rotorquant_adaptive",
+)
+
+
+def experimental_rotorquant_enabled() -> bool:
+    """Return whether the pure-MLX RotorQuant/IsoQuant backend is enabled.
+
+    The backend is intentionally gated because it is an experimental
+    storage/dequant cache, not the fused RotorQuant implementation described
+    in the paper. Keeping this behind an explicit flag prevents model-card or
+    dashboard config from putting unstable Metal cache code on the normal
+    inference path.
+    """
+    value = preferred_env_value(
+        "SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT",
+        "EXO_ENABLE_EXPERIMENTAL_ROTORQUANT",
+        "",
+    )
+    return str(value).strip().lower() in {"1", "true", "yes", "on"}
+
+
+def resolve_kv_cache_backend(raw_backend: str | None) -> KVCacheBackend:
+    """Normalize a configured KV cache backend into a safe runtime backend."""
+    backend = raw_backend or DEFAULT_KV_CACHE_BACKEND
+    if backend not in VALID_KV_CACHE_BACKENDS:
+        return DEFAULT_KV_CACHE_BACKEND
+
+    resolved: KVCacheBackend = backend
+    if (
+        resolved in EXPERIMENTAL_ROTORQUANT_BACKENDS
+        and not experimental_rotorquant_enabled()
+    ):
+        return DEFAULT_KV_CACHE_BACKEND
+
+    return resolved
+
+
 _kv_cache_backend_value = preferred_env_value(
     "SKULK_KV_CACHE_BACKEND",
     "EXO_KV_CACHE_BACKEND",
     DEFAULT_KV_CACHE_BACKEND,
 )
-KV_CACHE_BACKEND: KVCacheBackend = cast(
-    KVCacheBackend,
-    _kv_cache_backend_value if _kv_cache_backend_value else DEFAULT_KV_CACHE_BACKEND,
-)
+KV_CACHE_BACKEND: KVCacheBackend = resolve_kv_cache_backend(_kv_cache_backend_value)
 TURBOQUANT_K_BITS: int | None = (
     int(os.environ.get("SKULK_TQ_K_BITS", os.environ.get("EXO_TQ_K_BITS", "")))
     if os.environ.get("SKULK_TQ_K_BITS", os.environ.get("EXO_TQ_K_BITS"))
@@ -56,6 +96,20 @@
 DEFAULT_TURBOQUANT_V_BITS: int = 4
 OPTIQ_BITS: int = int(os.environ.get("EXO_OPTIQ_BITS", "4"))
 OPTIQ_FP16_LAYERS: int = int(os.environ.get("EXO_OPTIQ_FP16_LAYERS", "4"))
+ROTORQUANT_FP16_LAYERS: int = int(
+    os.environ.get(
+        "SKULK_ROTORQUANT_FP16_LAYERS",
+        os.environ.get("EXO_ROTORQUANT_FP16_LAYERS", "4"),
+    )
+)
+# Deferred prefill keeps K/V in fp16 during prompt processing and flushes
+# the buffer to compressed storage on the first decode token. It is the
+# load-bearing accuracy improvement of this backend; only disable for
+# debugging.
+ROTORQUANT_DEFER_PREFILL: bool = os.environ.get(
+    "SKULK_ROTORQUANT_DEFER_PREFILL",
+    os.environ.get("EXO_ROTORQUANT_DEFER_PREFILL", "1"),
+) not in ("0", "false", "False", "")
 
 DEFAULT_TOP_LOGPROBS: int = 5
 
diff --git a/src/exo/worker/engines/mlx/gemma4_prompt.py b/src/exo/worker/engines/mlx/gemma4_prompt.py
index 1cf44ce41..20e4270d3 100644
--- a/src/exo/worker/engines/mlx/gemma4_prompt.py
+++ b/src/exo/worker/engines/mlx/gemma4_prompt.py
@@ -2,13 +2,18 @@
 
 This module follows the Gemma 4 chat structure used by the reference Hugging
 Face template and Ollama's dedicated Gemma 4 renderer, while preserving one
-intentional divergence: when thinking is disabled we omit the empty synthetic
-thought-channel suffix because that shape wedges our distributed MLX warmup
-path. We keep a dedicated renderer so multimodal prompts stay under explicit
-repo control instead of relying on generic tokenizer chat templating.
+intentional warmup-only divergence: distributed warmup can suppress the empty
+synthetic thought-channel suffix because that shape has wedged our distributed
+MLX warmup path. Real inference keeps the reference suffix so generation begins
+at the expected visible assistant-content boundary. We keep a dedicated renderer
+so multimodal prompts stay under explicit repo control instead of relying on
+generic tokenizer chat templating.
 """
 
-from typing import Any
+from collections.abc import Mapping, Sequence
+from typing import TypeAlias, cast
+
+Gemma4Message: TypeAlias = Mapping[str, object]
 
 
 def strip_gemma4_thinking(text: str) -> str:
@@ -28,7 +33,7 @@ def strip_gemma4_thinking(text: str) -> str:
     return "".join(result).strip()
 
 
-def _render_gemma4_content(content: Any, role: str) -> str:
+def _render_gemma4_content(content: object, role: str) -> str:
     """Render one Gemma 4 message body."""
     if isinstance(content, str):
         return strip_gemma4_thinking(content) if role == "model" else content.strip()
@@ -36,14 +41,18 @@ def _render_gemma4_content(content: Any, role: str) -> str:
     if not isinstance(content, list):
         return str(content).strip()
 
+    content_items = cast(Sequence[object], content)
     parts: list[str] = []
-    for item in content:
-        if not isinstance(item, dict):
+    for item in content_items:
+        if not isinstance(item, Mapping):
             continue
-        item_type = str(item.get("type", ""))
+        item_mapping = cast(Mapping[str, object], item)
+        item_type = str(item_mapping.get("type", ""))
         if item_type == "text":
-            text = str(item.get("text", ""))
-            parts.append(strip_gemma4_thinking(text) if role == "model" else text.strip())
+            text = str(item_mapping.get("text", ""))
+            parts.append(
+                strip_gemma4_thinking(text) if role == "model" else text.strip()
+            )
         elif item_type == "image":
             parts.append("\n\n<|image|>\n\n")
         elif item_type == "audio":
@@ -54,10 +63,11 @@ def _render_gemma4_content(content: Any, role: str) -> str:
 
 
 def render_gemma4_prompt(
-    messages: list[dict[str, Any]],
+    messages: Sequence[Gemma4Message],
     *,
     add_generation_prompt: bool,
     enable_thinking: bool | None = None,
+    suppress_empty_thought_channel: bool = False,
 ) -> str:
     """Render a Gemma 4 prompt matching the reference chat template.
 
@@ -78,24 +88,32 @@ def render_gemma4_prompt(
         if enable_thinking:
             prompt_parts.append("<|think|>")
         if has_system_message:
-            prompt_parts.append(_render_gemma4_content(messages[0].get("content", ""), "system"))
+            prompt_parts.append(
+                _render_gemma4_content(messages[0].get("content", ""), "system")
+            )
             loop_messages = messages[1:]
         prompt_parts.append("<turn|>\n")
 
     for message in loop_messages:
-        role = "model" if str(message.get("role", "user")) == "assistant" else str(
-            message.get("role", "user")
+        role = (
+            "model"
+            if str(message.get("role", "user")) == "assistant"
+            else str(message.get("role", "user"))
         )
         prompt_parts.append(f"<|turn>{role}\n")
         prompt_parts.append(_render_gemma4_content(message.get("content", ""), role))
         prompt_parts.append("<turn|>\n")
 
     if add_generation_prompt:
-        # Gemma's published chat template appends an empty thought channel even
-        # when thinking is disabled. We intentionally omit that suffix here
-        # because it appears to trigger a pipeline warmup wedge in our MLX
-        # distributed path, and real traffic succeeds without this synthetic
-        # prefix.
         prompt_parts.append("<|turn>model\n")
+        if not enable_thinking and not suppress_empty_thought_channel:
+            # Gemma 4 expects the assistant turn to pass through the channel
+            # marker state. For normal inference we match the official
+            # template by adding an empty thought channel so generation begins
+            # in visible assistant content instead of at a raw ``<|channel>``
+            # boundary. Distributed warmup can suppress this suffix because it
+            # only validates prefill plus one decode step and historically
+            # wedged on richer synthetic prompt shapes.
+            prompt_parts.append("<|channel>thought\n<channel|>")
 
     return "".join(prompt_parts)
diff --git a/src/exo/worker/engines/mlx/generator/generate.py b/src/exo/worker/engines/mlx/generator/generate.py
index 5e40a7124..ea1930d4c 100644
--- a/src/exo/worker/engines/mlx/generator/generate.py
+++ b/src/exo/worker/engines/mlx/generator/generate.py
@@ -233,7 +233,9 @@ def _slice_native_pixel_values_for_uncached_suffix(
         return pixel_values
 
     available_images = (
-        len(pixel_values) if isinstance(pixel_values, list) else int(pixel_values.shape[0])
+        len(pixel_values)
+        if isinstance(pixel_values, list)
+        else int(pixel_values.shape[0])
     )
     remaining_indices = [
         idx
@@ -507,12 +509,12 @@ def combined_progress_callback(processed: int, total: int) -> None:
         prefill_step_size // min(4, group_size) if is_pipeline else prefill_step_size
     )
 
-    # Pipeline-parallel prefill can wedge when the prompt fits inside a single
-    # effective per-rank chunk (i.e. ``n_real == 1`` in the prefill loop).
     # Mirror pipeline_parallel_prefill's chunking math here by reserving the
     # final token for the post-loop pass before computing the real chunk count.
-    # That keeps multi-chunk prompts on pipeline_parallel_prefill while routing
-    # single-chunk short / warmup prompts through stream_generate.
+    # This value is primarily diagnostic now: all pipeline prompts use the
+    # explicit pipeline prefill path, but short one-chunk prompts intentionally
+    # suppress the distributed progress callback because there is no useful
+    # cancellation window inside a millisecond-scale prefill.
     real_prefill_tokens = max(num_tokens - 1, 0)
     pipeline_chunks = (
         (real_prefill_tokens + effective_prefill_step_size - 1)
@@ -520,7 +522,7 @@ def combined_progress_callback(processed: int, total: int) -> None:
         if effective_prefill_step_size > 0
         else 0
     )
-    use_pipeline_prefill = is_pipeline and pipeline_chunks >= 2
+    use_pipeline_prefill = is_pipeline
     logger.info(
         "Prefill path selected: "
         f"{'pipeline_parallel_prefill' if use_pipeline_prefill else 'stream_generate'} "
@@ -529,12 +531,23 @@ def combined_progress_callback(processed: int, total: int) -> None:
         f"prefill_step_size_effective={effective_prefill_step_size}, "
         f"pipeline_chunks={pipeline_chunks})"
     )
-    # Only enable pipeline-prefill mode for the true pipeline prefill path.
-    # Short prompts routed through stream_generate must behave like normal
-    # generation; leaving pipeline wrappers in prefill mode there can strand
-    # non-zero ranks in the pipeline send/recv path without the coordinated
-    # queue/flush behavior used by pipeline_parallel_prefill.
-    set_pipeline_prefill(model, is_prefill=use_pipeline_prefill)
+    # Pipeline models must run in prefill mode during any prefill forward
+    # pass — even the stream_generate fallback path for short prompts.
+    # With is_prefill=False, PipelineLastLayer performs all_gather on
+    # every forward pass. generate_step discards prefill logits and only
+    # evaluates cache states, so the all_gather is never consumed but
+    # remains queued. When a subsequent mx.eval triggers it, the ranks
+    # are desynchronized and the collective deadlocks (observed as a
+    # warmup hang on 3-rank pipeline with Gemma 4).
+    #
+    # With is_prefill=True, PipelineLastLayer uses point-to-point
+    # send/recv only — no collective required. The tradeoff: the first
+    # decode token sampled by generate_step is wrong on non-last ranks
+    # (they see rank-local activations, not full logits). This is
+    # acceptable because prefill() trims those decode tokens from the
+    # cache before real generation begins, and the finally block below
+    # resets is_prefill=False for subsequent decode.
+    set_pipeline_prefill(model, is_prefill=is_pipeline)
 
     with _hang_debug_watch(f"prefill barrier rank={rank} group_size={group_size}"):
         mx_barrier(group)
@@ -546,6 +559,9 @@ def combined_progress_callback(processed: int, total: int) -> None:
         if use_pipeline_prefill:
             set_pipeline_queue_sends(model, queue_sends=True)
             assert group is not None, "Pipeline prefill requires a distributed group"
+            pipeline_distributed_callback = (
+                distributed_prompt_progress_callback if pipeline_chunks >= 2 else None
+            )
             with _hang_debug_watch(
                 f"pipeline_parallel_prefill rank={rank} group_size={group_size}"
             ):
@@ -557,10 +573,16 @@ def combined_progress_callback(processed: int, total: int) -> None:
                     kv_group_size=KV_GROUP_SIZE,
                     kv_bits=KV_BITS,
                     prompt_progress_callback=progress_callback,
-                    distributed_prompt_progress_callback=distributed_prompt_progress_callback,
+                    distributed_prompt_progress_callback=pipeline_distributed_callback,
                     group=group,
                 )
         else:
+            # Non-pipeline models can safely use upstream stream_generate for
+            # prefill. Pipeline models avoid it entirely: mlx-lm's generate_step
+            # prefetches one decode step before yielding even with max_tokens=1,
+            # which can leave hidden pipeline sends/recvs in flight and wedge
+            # the next collective.
+            #
             # Use max_tokens=1 because max_tokens=0 does not work.
             # We just throw away the generated token - we only care about filling the cache
             with _hang_debug_watch(
@@ -576,7 +598,7 @@ def combined_progress_callback(processed: int, total: int) -> None:
                     prefill_step_size=prefill_step_size,
                     kv_group_size=KV_GROUP_SIZE,
                     kv_bits=KV_BITS,
-                    prompt_progress_callback=combined_progress_callback,
+                    prompt_progress_callback=progress_callback,
                 ):
                     logger.info(
                         f"Prefill stream_generate yielded first token (rank={rank})"
@@ -636,6 +658,12 @@ def warmup_inference(
             "prompt and ignoring warmup shaping overrides"
         )
 
+    # Distributed pipeline warmup MUST use greedy sampling so every rank
+    # samples the same token from the shared logits.  Stochastic sampling
+    # with per-rank random states can diverge, causing ranks to follow
+    # different code paths inside generate_step and issue mismatched
+    # collective operations.
+    is_distributed = _is_distributed_warmup(group)
     warmup_task_params = TextGenerationTaskParams(
         model=model_id,
         # Distributed pipeline warmup always uses the minimal sanity-check
@@ -647,11 +675,14 @@ def warmup_inference(
                 content=_warmup_user_content(group),
             )
         ],
-        max_output_tokens=1024,
+        # Warmup is a prefill + one-token decode sanity check. Asking the
+        # generator for exactly one token lets it take its normal terminal
+        # path instead of aborting a larger generation mid-stream.
+        max_output_tokens=1,
         enable_thinking=False,
-        temperature=1.0,
-        top_p=0.95,
-        top_k=64,
+        temperature=0.0 if is_distributed else 1.0,
+        top_p=1.0 if is_distributed else 0.95,
+        top_k=0 if is_distributed else 64,
     )
 
     with _hang_debug_watch(f"warmup apply_chat_template model={model_id}"):
@@ -659,6 +690,7 @@ def warmup_inference(
             tokenizer=tokenizer,
             task_params=warmup_task_params,
             model_card=model_card,
+            suppress_empty_gemma4_thought_channel=True,
         )
     logger.info(
         "Warmup prompt prepared "
@@ -693,10 +725,6 @@ def warmup_inference(
             group=group,
         ):
             tokens_generated += 1
-            # Warmup is only meant to validate prefill plus the first decode
-            # step. Stopping after the first token keeps runner readiness
-            # bounded even if the model would otherwise continue sampling.
-            break
 
     # The single-token warmup path intentionally samples only the first decode
     # step, so cold-start compile/prefill latency can dominate the elapsed
@@ -718,14 +746,19 @@ def warmup_inference(
 
     logger.info(f"warmed up by generating {tokens_generated} tokens")
     if group is not None:
-        check_for_cancel_every = int(
-            mx.max(
-                mx.distributed.all_gather(
-                    mx.array([check_for_cancel_every]),
-                    group=group,
-                )
-            ).item()
+        # Use all_sum with slotted contribution instead of all_gather.
+        # JACCL all_gather deadlocks intermittently on 3-node pipelines;
+        # all_sum is proven reliable.  Each rank puts its value in its
+        # own slot; we take the max across all slots.
+        world_size = group.size()
+        rank = group.rank()
+        slots = [0] * world_size
+        slots[rank] = check_for_cancel_every
+        cpu_stream = mx.default_stream(mx.Device(mx.cpu))
+        merged = mx.distributed.all_sum(
+            mx.array(slots, dtype=mx.int32), group=group, stream=cpu_stream
         )
+        check_for_cancel_every = int(mx.max(merged).item())
 
     logger.info(
         f"runner checking for cancellation every {check_for_cancel_every} tokens"
@@ -946,7 +979,9 @@ def _mlx_generate_native_vision(
         accumulated_text += final_text
 
     generation_elapsed = time.perf_counter() - generation_start_time
-    generation_tps = completion_tokens / generation_elapsed if generation_elapsed > 0 else 0.0
+    generation_tps = (
+        completion_tokens / generation_elapsed if generation_elapsed > 0 else 0.0
+    )
 
     final_stats = GenerationStats(
         prompt_tps=float(prompt_tps),
diff --git a/src/exo/worker/engines/mlx/rotorquant/__init__.py b/src/exo/worker/engines/mlx/rotorquant/__init__.py
new file mode 100644
index 000000000..608c8c48a
--- /dev/null
+++ b/src/exo/worker/engines/mlx/rotorquant/__init__.py
@@ -0,0 +1,33 @@
+"""Experimental RotorQuant-named KV cache backend (IsoQuant variant).
+
+Pure-MLX port of the IsoQuant 3-bit KV cache compression from
+johndpope/llama-cpp-turboquant (MIT) and scrya-com/rotorquant (MIT).
+This is not the fused RotorQuant+QJL attention implementation described in
+the RotorQuant paper; it is a storage/dequant cache used for isolated MLX
+experiments.
+
+Key properties vs the older TurboQuant native backend:
+- Block-diagonal 4D quaternion rotations instead of randomized Hadamard
+- Norm-correction trick for unbiased magnitudes after centroid quantization
+- Optional deferred prefill: K/V stay in fp16 during prompt processing,
+  flushed to compressed storage on the first decode-shaped call
+- GQA-native (compression is per-(kv_head, token), Q heads fan out at SDPA)
+
+The backend stores indices and norms; ``update_and_fetch`` returns fully
+dequantized fp16 K/V to standard ``mx.fast.scaled_dot_product_attention``.
+The centroid-space attention optimization from OptiQ is intentionally
+deferred to a follow-up; v1 prioritizes correctness and the deferred-prefill
+quality win over the rotated-space SDPA perf win.
+"""
+
+from exo.worker.engines.mlx.rotorquant.cache import (
+    RotorQuantKVCache,
+    make_rotorquant_adaptive_cache,
+    make_rotorquant_cache_from_template,
+)
+
+__all__ = [
+    "RotorQuantKVCache",
+    "make_rotorquant_adaptive_cache",
+    "make_rotorquant_cache_from_template",
+]
diff --git a/src/exo/worker/engines/mlx/rotorquant/cache.py b/src/exo/worker/engines/mlx/rotorquant/cache.py
new file mode 100644
index 000000000..cc4a09d3f
--- /dev/null
+++ b/src/exo/worker/engines/mlx/rotorquant/cache.py
@@ -0,0 +1,585 @@
+"""Experimental pure-MLX IsoQuant KV cache with optional deferred prefill.
+
+This module intentionally does not implement the fused RotorQuant+QJL
+attention path from the RotorQuant paper. It stores compressed K/V indices
+and norms, then materializes dequantized fp16 K/V for normal MLX attention.
+That makes it useful for isolated cache experiments, but not a production
+substitute for the upstream fused llama.cpp path.
+
+The idea: while the engine is processing a prompt (``update_and_fetch``
+called with ``num_steps > 1``), keep K and V in fp16. Quantization only
+happens once, on the first decode-shaped call (``num_steps == 1``).
+This avoids compounding centroid-roundtrip errors through every prefill
+attention chain.
+"""
+
+from collections.abc import Sequence
+from typing import cast
+
+import mlx.core as mx
+from mlx_lm.models.cache import (
+    ArraysCache,
+    KVCache,
+    RotatingKVCache,
+    create_attention_mask,
+)
+
+from exo.shared.types.mlx import KVCacheType, Model
+from exo.worker.engines.mlx.rotorquant.quantizer import IsoQuantizer
+from exo.worker.engines.mlx.rotorquant.tables import ISO3_BLOCK_SIZE
+
+
+class RotorQuantKVCache:
+    """Per-layer KV cache backed by IsoQuant 3-bit compression.
+
+    Storage layout (one slot per token per kv-head):
+      - ``key_indices``:   ``(B, kv_heads, T, head_dim)`` uint8
+      - ``key_norms``:     ``(B, kv_heads, T, head_dim // 128)`` fp16
+      - ``value_indices``: same shape as keys
+      - ``value_norms``:   same shape as key_norms
+
+    During the deferred-prefill window the class additionally holds
+    ``_pending_keys`` and ``_pending_values`` as fp16 buffers, and
+    ``update_and_fetch`` returns the concatenation of those buffers
+    directly without ever quantizing. The first call with
+    ``num_steps == 1`` flushes the pending buffers through the
+    quantizer in a single batched operation.
+    """
+
+    step = 256
+
+    def __init__(self, *, defer_prefill: bool = True, seed: int = 42) -> None:
+        # Deferred prefill is on by default because it is both an
+        # accuracy and a peak-memory-amortization win; tests can flip
+        # it off to exercise the always-quantized path.
+        self.defer_prefill = defer_prefill
+        self.seed = seed
+        self.offset = 0
+
+        self._quantizer: IsoQuantizer | None = None
+        self.head_dim: int | None = None
+
+        self._key_indices: mx.array | None = None
+        self._key_norms: mx.array | None = None
+        self._value_indices: mx.array | None = None
+        self._value_norms: mx.array | None = None
+
+        # Deferred prefill scratch buffers — kept in fp16 until the
+        # first decode-shaped call triggers ``_flush_deferred``.
+        self._pending_keys: mx.array | None = None
+        self._pending_values: mx.array | None = None
+        self._pending_active: bool = defer_prefill
+
+    # ------------------------------------------------------------------
+    # Initialization helpers
+    # ------------------------------------------------------------------
+
+    def _ensure_quantizer(self, head_dim: int) -> None:
+        if head_dim % ISO3_BLOCK_SIZE != 0:
+            raise ValueError(
+                f"RotorQuantKVCache requires head_dim divisible by "
+                f"{ISO3_BLOCK_SIZE}, got {head_dim}"
+            )
+        if self._quantizer is None:
+            self._quantizer = IsoQuantizer()
+            self.head_dim = head_dim
+
+    def _expand_storage(
+        self,
+        batch_size: int,
+        kv_heads: int,
+        steps: int,
+        head_dim: int,
+    ) -> None:
+        n_blocks = head_dim // ISO3_BLOCK_SIZE
+        needed = self.offset + steps
+        if self._key_indices is not None and needed <= self._key_indices.shape[2]:
+            return
+
+        new_steps = ((steps + self.step - 1) // self.step) * self.step
+        if self._key_indices is None:
+            shape_idx = (batch_size, kv_heads, new_steps, head_dim)
+            shape_norm = (batch_size, kv_heads, new_steps, n_blocks)
+            self._key_indices = mx.zeros(shape_idx, dtype=mx.uint8)
+            self._key_norms = mx.zeros(shape_norm, dtype=mx.float16)
+            self._value_indices = mx.zeros(shape_idx, dtype=mx.uint8)
+            self._value_norms = mx.zeros(shape_norm, dtype=mx.float16)
+            return
+
+        # Concatenate a fresh slab of zeros to the right of the live
+        # region. The live region ends at ``self.offset``; everything
+        # past that is stale capacity from a previous expansion.
+        assert self._key_norms is not None
+        assert self._value_indices is not None
+        assert self._value_norms is not None
+        zeros_idx = mx.zeros(
+            (batch_size, kv_heads, new_steps, head_dim),
+            dtype=mx.uint8,
+        )
+        zeros_norm = mx.zeros(
+            (batch_size, kv_heads, new_steps, n_blocks),
+            dtype=mx.float16,
+        )
+        self._key_indices = mx.concatenate(
+            [self._key_indices[..., : self.offset, :], zeros_idx],
+            axis=2,
+        )
+        self._key_norms = mx.concatenate(
+            [self._key_norms[..., : self.offset, :], zeros_norm],
+            axis=2,
+        )
+        self._value_indices = mx.concatenate(
+            [self._value_indices[..., : self.offset, :], zeros_idx],
+            axis=2,
+        )
+        self._value_norms = mx.concatenate(
+            [self._value_norms[..., : self.offset, :], zeros_norm],
+            axis=2,
+        )
+
+    # ------------------------------------------------------------------
+    # Update / fetch
+    # ------------------------------------------------------------------
+
+    def update_and_fetch(
+        self,
+        keys: mx.array,
+        values: mx.array,
+    ) -> tuple[mx.array, mx.array]:
+        """Insert ``keys``/``values`` and return the full live KV cache.
+
+        While deferred prefill is active, the call appends to the fp16
+        scratch buffers and returns the concatenated fp16 view directly.
+        Once the first decode-shaped call (``num_steps == 1``) arrives,
+        the scratch buffers are flushed through the quantizer in one
+        batched operation, after which all subsequent calls quantize on
+        insert and return a freshly dequantized view.
+        """
+        _, _, num_steps, head_dim = keys.shape
+        self._ensure_quantizer(head_dim)
+
+        # Deferred-prefill path: keep raw fp16 until the first decode token.
+        # On a multi-token call we append to the scratch and bail out
+        # immediately. On a single-token call we flush whatever we have
+        # accumulated, then fall through to the normal quantized insert
+        # path so the new decode token is stored alongside the rest.
+        if self._pending_active:
+            if num_steps > 1:
+                self._append_pending(keys, values)
+                assert self._pending_keys is not None
+                assert self._pending_values is not None
+                # Advance the logical offset so downstream attention
+                # (RoPE position via ``cache.offset``) sees the correct
+                # token count when prefill is split into multiple
+                # chunks. Without this, every chunk after the first is
+                # positioned as if it started at token 0.
+                self.offset += num_steps
+                return self._pending_keys, self._pending_values
+            self._flush_deferred(keys.dtype)
+
+        if num_steps > 0:
+            self._quantize_and_store(keys, values)
+
+        return self._dequantize_live(keys.dtype)
+
+    def _append_pending(self, keys: mx.array, values: mx.array) -> None:
+        """Append new K/V to the deferred-prefill scratch buffers."""
+        if self._pending_keys is None:
+            self._pending_keys = keys.astype(mx.float16)
+            self._pending_values = values.astype(mx.float16)
+            return
+        assert self._pending_values is not None
+        self._pending_keys = mx.concatenate(
+            [self._pending_keys, keys.astype(mx.float16)],
+            axis=2,
+        )
+        self._pending_values = mx.concatenate(
+            [self._pending_values, values.astype(mx.float16)],
+            axis=2,
+        )
+
+    def _flush_deferred(self, dtype: mx.Dtype) -> None:
+        """Quantize the deferred prefill buffer and free the fp16 scratch.
+
+        If the first call into ``update_and_fetch`` is already decode-shaped
+        (e.g., a 1-token prompt) the deferred buffers will be empty. In that
+        case there is nothing to flush — we just exit the deferred phase and
+        let the caller's normal quantize-on-insert path handle the new
+        token.
+        """
+        del dtype  # the live cache stores indices regardless of caller dtype
+        assert self._quantizer is not None
+
+        if self._pending_keys is None or self._pending_values is None:
+            self._pending_keys = None
+            self._pending_values = None
+            self._pending_active = False
+            return
+
+        pending_keys = self._pending_keys
+        pending_values = self._pending_values
+        batch_size, kv_heads, num_pending, head_dim = pending_keys.shape
+
+        # ``offset`` was advanced as tokens were appended to the pending
+        # scratch, so the live region we are about to populate is
+        # ``[offset - num_pending, offset)``. ``_expand_storage`` sizes
+        # capacity from ``offset``, so temporarily rewind it to the
+        # pre-pending position before expanding so we don't allocate
+        # ``num_pending`` extra slots.
+        live_end = self.offset
+        self.offset = live_end - num_pending
+        self._expand_storage(batch_size, kv_heads, num_pending, head_dim)
+        assert self._key_indices is not None
+        assert self._key_norms is not None
+        assert self._value_indices is not None
+        assert self._value_norms is not None
+
+        key_indices, key_norms = self._quantizer.quantize(pending_keys)
+        value_indices, value_norms = self._quantizer.quantize(pending_values)
+
+        self._key_indices[..., self.offset : live_end, :] = key_indices
+        self._key_norms[..., self.offset : live_end, :] = key_norms
+        self._value_indices[..., self.offset : live_end, :] = value_indices
+        self._value_norms[..., self.offset : live_end, :] = value_norms
+        self.offset = live_end
+
+        # Free the scratch — from now on we are in the steady-state
+        # decode path and quantize on insert.
+        self._pending_keys = None
+        self._pending_values = None
+        self._pending_active = False
+
+    def _quantize_and_store(self, keys: mx.array, values: mx.array) -> None:
+        """Quantize freshly arrived tokens and append to the live cache."""
+        assert self._quantizer is not None
+        batch_size, kv_heads, num_steps, head_dim = keys.shape
+
+        self._expand_storage(batch_size, kv_heads, num_steps, head_dim)
+        assert self._key_indices is not None
+        assert self._key_norms is not None
+        assert self._value_indices is not None
+        assert self._value_norms is not None
+
+        key_indices, key_norms = self._quantizer.quantize(keys)
+        value_indices, value_norms = self._quantizer.quantize(values)
+
+        end = self.offset + num_steps
+        self._key_indices[..., self.offset : end, :] = key_indices
+        self._key_norms[..., self.offset : end, :] = key_norms
+        self._value_indices[..., self.offset : end, :] = value_indices
+        self._value_norms[..., self.offset : end, :] = value_norms
+        self.offset = end
+
+    def _dequantize_live(
+        self, output_dtype: mx.Dtype
+    ) -> tuple[mx.array, mx.array]:
+        """Materialize the full live cache as fp16 K/V for SDPA."""
+        assert self._quantizer is not None
+        assert self._key_indices is not None
+        assert self._key_norms is not None
+        assert self._value_indices is not None
+        assert self._value_norms is not None
+
+        keys_out = self._quantizer.dequantize(
+            self._key_indices[..., : self.offset, :],
+            self._key_norms[..., : self.offset, :],
+            output_dtype,
+        )
+        values_out = self._quantizer.dequantize(
+            self._value_indices[..., : self.offset, :],
+            self._value_norms[..., : self.offset, :],
+            output_dtype,
+        )
+        return keys_out, values_out
+
+    # ------------------------------------------------------------------
+    # mlx-lm cache protocol
+    # ------------------------------------------------------------------
+
+    def size(self) -> int:
+        # ``offset`` is the logical token count in both the deferred and
+        # steady-state phases (it is advanced on append in either path).
+        return self.offset
+
+    @property
+    def state(self) -> object:
+        """Pytree of array leaves for snapshot/restore and ``mx.eval``.
+
+        The shape of the tuple disambiguates phases for the setter:
+          - 2 elements → deferred-prefill scratch ``(pending_keys, pending_values)``
+          - 4 elements → live quantized cache ``(key_indices, key_norms, value_indices, value_norms)``
+          - empty list → uninitialized cache
+
+        We deliberately do **not** include a Python ``str`` tag here:
+        ``mx.eval`` and ``mx.save_safetensors`` expect every leaf in the
+        cache pytree to be an ``mx.array``, and a ``str`` leaf raises at
+        runtime. The phase is also tracked in ``meta_state`` via
+        ``_pending_active`` for human inspection.
+        """
+        if self._pending_active:
+            if self._pending_keys is None:
+                return []
+            assert self._pending_values is not None
+            return (self._pending_keys, self._pending_values)
+        if self._key_indices is None:
+            return []
+        assert self._key_norms is not None
+        assert self._value_indices is not None
+        assert self._value_norms is not None
+        return (
+            self._key_indices[..., : self.offset, :],
+            self._key_norms[..., : self.offset, :],
+            self._value_indices[..., : self.offset, :],
+            self._value_norms[..., : self.offset, :],
+        )
+
+    @state.setter
+    def state(self, value: object) -> None:
+        seq = cast(Sequence[mx.array], value)
+        if len(seq) == 0:
+            return
+        if len(seq) == 2:
+            self._pending_keys = seq[0]
+            self._pending_values = seq[1]
+            self._pending_active = True
+            # Keep ``offset`` consistent with the logical token count
+            # restored from the pending scratch.
+            self.offset = int(self._pending_keys.shape[2])
+            return
+        if len(seq) == 4:
+            self._key_indices = seq[0]
+            self._key_norms = seq[1]
+            self._value_indices = seq[2]
+            self._value_norms = seq[3]
+            self.offset = int(self._key_indices.shape[2])
+            self._pending_active = False
+            return
+        raise ValueError(
+            f"RotorQuantKVCache.state expects 0, 2, or 4 array leaves, got {len(seq)}"
+        )
+
+    @property
+    def meta_state(self) -> object:
+        head_dim = -1 if self.head_dim is None else self.head_dim
+        return (
+            str(self.offset),
+            str(self.seed),
+            str(int(self.defer_prefill)),
+            str(int(self._pending_active)),
+            str(head_dim),
+        )
+
+    @meta_state.setter
+    def meta_state(self, value: object) -> None:
+        offset, seed, defer, pending, head_dim = (
+            int(part) for part in cast(Sequence[str], value)
+        )
+        self.offset = offset
+        self.seed = seed
+        self.defer_prefill = bool(defer)
+        self._pending_active = bool(pending)
+        self.head_dim = None if head_dim < 0 else head_dim
+        if self.head_dim is not None and self._quantizer is None:
+            self._quantizer = IsoQuantizer()
+
+    def is_trimmable(self) -> bool:
+        return True
+
+    def trim(self, n: int) -> int:
+        if self._pending_active and self._pending_keys is not None:
+            assert self._pending_values is not None
+            available = int(self._pending_keys.shape[2])
+            trimmed = min(available, n)
+            new_len = available - trimmed
+            self._pending_keys = self._pending_keys[..., :new_len, :]
+            self._pending_values = self._pending_values[..., :new_len, :]
+            self.offset -= trimmed
+            return trimmed
+        trimmed = min(self.offset, n)
+        self.offset -= trimmed
+        return trimmed
+
+    def make_mask(self, *args: object, **kwargs: object) -> object:
+        if len(args) == 0 or not isinstance(args[0], int):
+            return None
+        window_size = kwargs.get("window_size")
+        return_array = kwargs.get("return_array", False)
+        if not isinstance(window_size, (int, type(None))):
+            return None
+        if not isinstance(return_array, bool):
+            return None
+        return create_attention_mask(
+            args[0],
+            offset=self.size(),
+            window_size=window_size,
+            return_array=return_array,
+        )
+
+    @property
+    def is_deferred(self) -> bool:
+        """Whether the cache is still in the deferred-prefill phase.
+
+        Public so tests and observability code can ask without poking
+        at private attributes.
+        """
+        return self._pending_active
+
+    @property
+    def has_live_storage(self) -> bool:
+        """Whether the quantized live cache has been allocated yet."""
+        return self._key_indices is not None
+
+    @property
+    def has_pending_storage(self) -> bool:
+        """Whether the deferred-prefill scratch buffer is currently live."""
+        return self._pending_keys is not None
+
+    def empty(self) -> bool:
+        if self._pending_active:
+            return self._pending_keys is None
+        return self._key_indices is None
+
+    @property
+    def nbytes(self) -> int:
+        total = 0
+        if self._key_indices is not None:
+            assert self._key_norms is not None
+            assert self._value_indices is not None
+            assert self._value_norms is not None
+            total += int(self._key_indices.nbytes)
+            total += int(self._key_norms.nbytes)
+            total += int(self._value_indices.nbytes)
+            total += int(self._value_norms.nbytes)
+        if self._pending_keys is not None:
+            assert self._pending_values is not None
+            total += int(self._pending_keys.nbytes)
+            total += int(self._pending_values.nbytes)
+        return total
+
+
+# ----------------------------------------------------------------------
+# Compatibility check + factory functions (mirrors turboquant/cache.py)
+# ----------------------------------------------------------------------
+
+
+def ensure_rotorquant_compatible(model: Model) -> None:
+    """Validate that the model uses cache types RotorQuant can wrap.
+
+    RotorQuant can only replace plain ``KVCache`` entries; ``ArraysCache``
+    (SSM) and ``RotatingKVCache`` (sliding window) entries are passed
+    through to mlx-lm unchanged. Anything else is rejected so we never
+    silently corrupt a model with custom cache layouts.
+    """
+    if not hasattr(model, "make_cache"):
+        return
+
+    sample_cache = cast(
+        list[object],
+        model.make_cache(),  # type: ignore[reportUnknownMemberType]
+    )
+    if len(sample_cache) == 0:
+        return
+    if not all(
+        isinstance(entry, (KVCache, ArraysCache, RotatingKVCache))
+        for entry in sample_cache
+    ):
+        cache_types = ", ".join(
+            sorted({type(entry).__name__ for entry in sample_cache})
+        )
+        raise ValueError(
+            "RotorQuant backend currently supports KVCache entries plus optional "
+            "ArraysCache and RotatingKVCache passthrough; "
+            f"found cache type(s): {cache_types}"
+        )
+
+
+def make_rotorquant_cache_from_template(
+    model: Model,
+    *,
+    defer_prefill: bool = True,
+    seed: int = 42,
+) -> KVCacheType:
+    """Build one RotorQuantKVCache per attention layer.
+
+    Mirrors :func:`make_turboquant_cache_from_template`. Non-KV entries
+    such as Mamba SSM caches are preserved unchanged.
+    """
+    ensure_rotorquant_compatible(model)
+    if not hasattr(model, "make_cache"):
+        return [
+            RotorQuantKVCache(defer_prefill=defer_prefill, seed=seed + index)
+            for index, _layer in enumerate(model.layers)
+        ]
+
+    template_cache = cast(
+        list[object],
+        model.make_cache(),  # type: ignore[reportUnknownMemberType]
+    )
+    caches: list[object] = []
+    kv_index = 0
+    for entry in template_cache:
+        if isinstance(entry, KVCache):
+            caches.append(
+                RotorQuantKVCache(defer_prefill=defer_prefill, seed=seed + kv_index)
+            )
+            kv_index += 1
+        else:
+            caches.append(entry)
+    return caches
+
+
+def make_rotorquant_adaptive_cache(
+    model: Model,
+    *,
+    fp16_layers: int,
+    defer_prefill: bool = True,
+    seed: int = 42,
+) -> KVCacheType:
+    """Build a RotorQuant cache with FP16 protection on edge layers.
+
+    The first ``fp16_layers`` and last ``fp16_layers`` attention layers
+    use plain mlx-lm ``KVCache`` instances; everything in between uses
+    :class:`RotorQuantKVCache`. Mirrors
+    :func:`make_turboquant_adaptive_cache` exactly.
+    """
+    ensure_rotorquant_compatible(model)
+    if not hasattr(model, "make_cache"):
+        caches: list[object] = []
+        for index, _layer in enumerate(model.layers):
+            if index < fp16_layers or index >= len(model.layers) - fp16_layers:
+                caches.append(KVCache())
+            else:
+                caches.append(
+                    RotorQuantKVCache(
+                        defer_prefill=defer_prefill,
+                        seed=seed + index,
+                    )
+                )
+        return caches
+
+    template_cache = cast(
+        list[object],
+        model.make_cache(),  # type: ignore[reportUnknownMemberType]
+    )
+    kv_positions = [
+        index
+        for index, entry in enumerate(template_cache)
+        if isinstance(entry, KVCache)
+    ]
+    caches = []
+    for index, entry in enumerate(template_cache):
+        if not isinstance(entry, KVCache):
+            caches.append(entry)
+            continue
+
+        kv_order = kv_positions.index(index)
+        if kv_order < fp16_layers or kv_order >= len(kv_positions) - fp16_layers:
+            caches.append(KVCache())
+        else:
+            caches.append(
+                RotorQuantKVCache(
+                    defer_prefill=defer_prefill,
+                    seed=seed + kv_order,
+                )
+            )
+    return caches
diff --git a/src/exo/worker/engines/mlx/rotorquant/quantizer.py b/src/exo/worker/engines/mlx/rotorquant/quantizer.py
new file mode 100644
index 000000000..5a22310ef
--- /dev/null
+++ b/src/exo/worker/engines/mlx/rotorquant/quantizer.py
@@ -0,0 +1,146 @@
+"""3-bit IsoQuant quantizer with norm correction.
+
+Mirrors the quantize/dequantize logic in
+``johndpope/llama-cpp-turboquant/ggml/src/ggml-iso-quant.c`` but in pure
+MLX. Storage is one fp16 norm and one uint8 index per element per
+``(B, H, T)`` block — bit-packing into the 50-byte ``block_iso3_0``
+layout is a future-work optimization that does not affect correctness.
+
+The norm-correction trick (stored norm = ``||x|| / sqrt(Σ centroid²)``)
+is critical and not mentioned in the README of either upstream project.
+It absorbs the L2 shrinkage induced by centroid quantization so that
+attention's inner products remain unbiased.
+"""
+
+import mlx.core as mx
+
+from exo.worker.engines.mlx.rotorquant.rotation import (
+    iso3_rotate_forward,
+    iso3_rotate_inverse,
+)
+from exo.worker.engines.mlx.rotorquant.tables import (
+    ISO3_BLOCK_SIZE,
+    centroid_table,
+)
+
+_NORM_EPS = 1e-10
+
+
+class IsoQuantizer:
+    """Stateless 3-bit quantizer for ``head_dim``-multiple-of-128 vectors.
+
+    Holds the precomputed centroid table and centroid-boundary midpoints
+    on the active MLX device. The quantizer itself carries no per-token
+    state; the cache class owns the actual storage tensors.
+    """
+
+    def __init__(self) -> None:
+        self._centroids = centroid_table()
+        # Boundary midpoints between adjacent centroids: a value falls
+        # into bin ``i`` iff it lies between boundary i-1 and i. With 8
+        # centroids there are 7 internal boundaries.
+        boundaries = (self._centroids[:-1] + self._centroids[1:]) / 2.0
+        self._boundary_values: list[float] = boundaries.tolist()  # type: ignore[reportAny]
+
+    @property
+    def centroids(self) -> mx.array:
+        """Return the centroid table as an fp32 array of shape ``(8,)``."""
+        return self._centroids
+
+    def quantize(self, x: mx.array) -> tuple[mx.array, mx.array]:
+        """Quantize a tensor whose last dim is a multiple of the block size.
+
+        Args:
+            x: ``(..., d)`` tensor where ``d`` is a multiple of 128.
+
+        Returns:
+            A pair ``(indices, norms)``:
+              - ``indices``: ``(..., d)`` uint8 in ``[0, 7]``
+              - ``norms``: ``(..., d // 128)`` fp16 corrected norms,
+                one per iso3 block.
+        """
+        if x.shape[-1] % ISO3_BLOCK_SIZE != 0:
+            raise ValueError(
+                f"IsoQuantizer requires last dim divisible by {ISO3_BLOCK_SIZE}, "
+                f"got {x.shape[-1]}"
+            )
+
+        x32 = x.astype(mx.float32)
+        leading = x32.shape[:-1]
+        n_blocks = x32.shape[-1] // ISO3_BLOCK_SIZE
+
+        # Reshape so each iso3 block of 128 elements becomes its own row
+        # along a new axis. The norm and rotation are then per-block.
+        blocks = x32.reshape(*leading, n_blocks, ISO3_BLOCK_SIZE)
+
+        # Per-block L2 norm — matches grp_norm in the C reference.
+        norm_sq = mx.sum(blocks * blocks, axis=-1, keepdims=True)
+        grp_norm = mx.sqrt(norm_sq)
+        safe_norm = mx.maximum(grp_norm, mx.array(_NORM_EPS, dtype=mx.float32))
+        unit = blocks / safe_norm
+
+        # Forward quaternion rotation in the unit sphere.
+        rotated = iso3_rotate_forward(unit)
+
+        # Find nearest centroid index per coordinate. Walks the boundary
+        # list once with a running ``where`` accumulator. Cheap because
+        # there are only 7 boundaries for 8 centroids.
+        indices = mx.zeros(rotated.shape, dtype=mx.uint8)
+        for i, boundary in enumerate(self._boundary_values):
+            indices = indices + (rotated > boundary).astype(mx.uint8)
+            del i  # boundary index unused; the cumulative count is what matters
+
+        # Norm correction: absorb the L2 shrinkage of the quantized unit
+        # vector into the stored norm so dequantization yields an
+        # unbiased magnitude. ``recon_sq = Σ centroid[idx]²`` exactly
+        # matches the C reference path.
+        centroid_values = self._centroids[indices.astype(mx.int32)]
+        recon_sq = mx.sum(centroid_values * centroid_values, axis=-1, keepdims=True)
+        recon_norm = mx.sqrt(recon_sq)
+        safe_recon = mx.maximum(recon_norm, mx.array(_NORM_EPS, dtype=mx.float32))
+        corrected_norm = grp_norm / safe_recon
+
+        # Pack indices back to (..., d) and norms to (..., n_blocks).
+        indices_flat = indices.reshape(*leading, n_blocks * ISO3_BLOCK_SIZE)
+        norms_flat = corrected_norm.reshape(*leading, n_blocks).astype(mx.float16)
+        return indices_flat, norms_flat
+
+    def dequantize(
+        self,
+        indices: mx.array,
+        norms: mx.array,
+        output_dtype: mx.Dtype,
+    ) -> mx.array:
+        """Inverse of :meth:`quantize`.
+
+        Args:
+            indices: ``(..., d)`` uint8 indices, ``d`` a multiple of 128.
+            norms: ``(..., d // 128)`` corrected fp16 norms.
+            output_dtype: target dtype of the reconstructed tensor.
+
+        Returns:
+            ``(..., d)`` reconstructed tensor in ``output_dtype``.
+        """
+        if indices.shape[-1] % ISO3_BLOCK_SIZE != 0:
+            raise ValueError(
+                f"IsoQuantizer.dequantize requires indices last dim divisible by "
+                f"{ISO3_BLOCK_SIZE}, got {indices.shape[-1]}"
+            )
+
+        leading = indices.shape[:-1]
+        n_blocks = indices.shape[-1] // ISO3_BLOCK_SIZE
+
+        # Look up centroid values for every index, then reshape to
+        # per-block layout for the inverse rotation.
+        centroid_values = self._centroids[indices.astype(mx.int32)]
+        blocks = centroid_values.reshape(*leading, n_blocks, ISO3_BLOCK_SIZE)
+
+        rotated_back = iso3_rotate_inverse(blocks)
+
+        # Multiply each block by its stored corrected norm. The expand
+        # broadcasts ``(..., n_blocks)`` against the trailing block dim.
+        norms_f32 = norms.astype(mx.float32)[..., None]
+        scaled = rotated_back * norms_f32
+
+        flat = scaled.reshape(*leading, n_blocks * ISO3_BLOCK_SIZE)
+        return flat.astype(output_dtype)
diff --git a/src/exo/worker/engines/mlx/rotorquant/rotation.py b/src/exo/worker/engines/mlx/rotorquant/rotation.py
new file mode 100644
index 000000000..4fa98ab4d
--- /dev/null
+++ b/src/exo/worker/engines/mlx/rotorquant/rotation.py
@@ -0,0 +1,104 @@
+"""IsoQuant block-diagonal quaternion rotation in pure MLX.
+
+Each 128-element vector is split into 32 groups of 4. Each group is
+rotated by a fixed unit quaternion via the Hamilton product
+``q_L * v`` (treating ``v`` as a pure quaternion). The inverse uses
+``conj(q_L) * v`` because the rotation quaternion is unit.
+
+This is mathematically equivalent to a block-diagonal SO(4) rotation
+with 3 degrees of freedom per block, costing 16 multiplies + 12 adds
+per group versus the ``O(d log d)`` randomized Hadamard used by the
+older native TurboQuant backend.
+"""
+
+import mlx.core as mx
+
+from exo.worker.engines.mlx.rotorquant.tables import (
+    ISO3_BLOCK_SIZE,
+    ISO3_GROUPS_PER_BLOCK,
+    quaternion_table,
+)
+
+_qw, _qx, _qy, _qz = quaternion_table()
+
+
+def _quat_mul(
+    aw: mx.array,
+    ax: mx.array,
+    ay: mx.array,
+    az: mx.array,
+    bw: mx.array,
+    bx: mx.array,
+    by: mx.array,
+    bz: mx.array,
+) -> tuple[mx.array, mx.array, mx.array, mx.array]:
+    """Hamilton product ``a * b`` of two quaternions, broadcast over groups.
+
+    Mirrors the ``quat_mul`` helper in ``ggml-iso-quant.c`` exactly.
+    All inputs broadcast against each other along the trailing group axis.
+    """
+    rw = aw * bw - ax * bx - ay * by - az * bz
+    rx = aw * bx + ax * bw + ay * bz - az * by
+    ry = aw * by - ax * bz + ay * bw + az * bx
+    rz = aw * bz + ax * by - ay * bx + az * bw
+    return rw, rx, ry, rz
+
+
+def iso3_rotate_forward(unit: mx.array) -> mx.array:
+    """Apply the per-group quaternion rotation to a unit-normalized tensor.
+
+    Args:
+        unit: ``(..., 128)`` fp32 tensor with unit-norm leading slices.
+
+    Returns:
+        ``(..., 128)`` fp32 tensor in the rotated coordinate frame.
+    """
+    if unit.shape[-1] != ISO3_BLOCK_SIZE:
+        raise ValueError(
+            f"iso3_rotate_forward expects last dim {ISO3_BLOCK_SIZE}, "
+            f"got {unit.shape[-1]}"
+        )
+
+    # Reshape into (..., groups, 4) so the four quaternion components map
+    # cleanly onto the trailing axis. The quaternion tables broadcast as
+    # (groups,) → (1, ..., groups) against the leading dims.
+    grouped = unit.reshape(*unit.shape[:-1], ISO3_GROUPS_PER_BLOCK, 4)
+    v0 = grouped[..., 0]
+    v1 = grouped[..., 1]
+    v2 = grouped[..., 2]
+    v3 = grouped[..., 3]
+
+    rw, rx, ry, rz = _quat_mul(_qw, _qx, _qy, _qz, v0, v1, v2, v3)
+
+    rotated = mx.stack([rw, rx, ry, rz], axis=-1)
+    return rotated.reshape(*unit.shape)
+
+
+def iso3_rotate_inverse(rotated: mx.array) -> mx.array:
+    """Inverse of :func:`iso3_rotate_forward`.
+
+    Uses the conjugate quaternion ``(qw, -qx, -qy, -qz)`` because the
+    rotation quaternion is unit.
+
+    Args:
+        rotated: ``(..., 128)`` fp32 tensor in the rotated frame.
+
+    Returns:
+        ``(..., 128)`` fp32 tensor back in the original frame.
+    """
+    if rotated.shape[-1] != ISO3_BLOCK_SIZE:
+        raise ValueError(
+            f"iso3_rotate_inverse expects last dim {ISO3_BLOCK_SIZE}, "
+            f"got {rotated.shape[-1]}"
+        )
+
+    grouped = rotated.reshape(*rotated.shape[:-1], ISO3_GROUPS_PER_BLOCK, 4)
+    q0 = grouped[..., 0]
+    q1 = grouped[..., 1]
+    q2 = grouped[..., 2]
+    q3 = grouped[..., 3]
+
+    rw, rx, ry, rz = _quat_mul(_qw, -_qx, -_qy, -_qz, q0, q1, q2, q3)
+
+    restored = mx.stack([rw, rx, ry, rz], axis=-1)
+    return restored.reshape(*rotated.shape)
diff --git a/src/exo/worker/engines/mlx/rotorquant/tables.py b/src/exo/worker/engines/mlx/rotorquant/tables.py
new file mode 100644
index 000000000..6202ea70b
--- /dev/null
+++ b/src/exo/worker/engines/mlx/rotorquant/tables.py
@@ -0,0 +1,102 @@
+"""Vendored constants for IsoQuant 3-bit KV cache compression.
+
+Tables are lifted verbatim from johndpope/llama-cpp-turboquant
+(``ggml/src/ggml-iso-quant.c``, MIT-licensed). Keeping them bit-exact
+against the upstream C source means our quantizer agrees with the
+llama.cpp reference modulo floating-point ordering.
+
+The 32 unit quaternions parameterize one Hamilton-product rotation per
+4D group; for a 128-dimensional head this is 32 groups × 4 dims. The
+8-entry centroid table is the Lloyd-Max optimum for the post-rotation
+N(0, 1/d)-like coordinate distribution.
+"""
+
+from typing import Final
+
+import mlx.core as mx
+
+#: Native block size used by the iso3 layout. Fits exactly one Llama/Qwen
+#: head dimension. Heads with other sizes must be a multiple of this and
+#: are processed as multiple iso3 blocks.
+ISO3_BLOCK_SIZE: Final[int] = 128
+
+#: Number of quaternion groups per ISO3 block (block_size / 4).
+ISO3_GROUPS_PER_BLOCK: Final[int] = ISO3_BLOCK_SIZE // 4
+
+#: Bits per element after compression (3-bit indices via 8-entry codebook).
+ISO3_BITS: Final[int] = 3
+
+# Hardcoded unit quaternion components, one per 4D group (32 groups).
+# Sourced verbatim from ggml-iso-quant.c lines 51-54 in
+# johndpope/llama-cpp-turboquant feature/planarquant-kv-cache.
+# fmt: off
+_ISO3_QW: Final[tuple[float, ...]] = (
+    0.5765609741, 0.3176580369, -0.3234235942, -0.5127438903,
+    0.9233905673, -0.3323571086, 0.5468608141, -0.2500519454,
+    -0.5812215805, 0.3228830695, -0.7299832702, -0.4535493255,
+    -0.7338157296, -0.2884652913, -0.9000198841, -0.0377033800,
+    0.5104404092, 0.2033989877, -0.2462528497, 0.2314069420,
+    0.0072374810, 0.3923372924, 0.4958070219, -0.7235037088,
+    -0.9383618832, 0.4430379272, -0.2075705230, 0.1983736306,
+    -0.8834578991, 0.7389573455, -0.0156172011, 0.7738668919,
+)
+
+_ISO3_QX: Final[tuple[float, ...]] = (
+    0.4450169504, -0.5780548453, 0.7089627385, -0.3940812945,
+    -0.0897334740, 0.4727236331, 0.5542563796, 0.0450818054,
+    -0.3657043576, -0.4298477769, 0.4666220546, 0.7556306720,
+    -0.5284956098, 0.7042509317, 0.0230921544, 0.7110687494,
+    0.3024962246, -0.1157865301, 0.7490812540, -0.2582575679,
+    -0.2255804837, 0.3838746250, -0.3209520578, -0.3477301002,
+    0.1824720055, 0.4032751918, 0.8433781862, 0.9533935785,
+    -0.0620501526, 0.0927560627, 0.2964956462, 0.2402082384,
+)
+
+_ISO3_QY: Final[tuple[float, ...]] = (
+    0.2695076466, -0.0201656222, -0.1687686443, -0.5415957570,
+    -0.2796611190, 0.3510629535, 0.2609911859, -0.2715902030,
+    -0.0937586129, 0.3095585108, -0.4123268127, -0.4394895136,
+    0.0626545250, -0.4811822474, -0.0407132693, -0.4566248953,
+    0.7834537029, -0.6187923551, 0.0809760988, -0.8879503012,
+    -0.8928058147, 0.8350352049, -0.6994170547, 0.5606835485,
+    0.2933705449, 0.7377059460, 0.4534837306, -0.0009816211,
+    -0.3632916510, -0.3959124386, 0.1631654203, 0.5088164806,
+)
+
+_ISO3_QZ: Final[tuple[float, ...]] = (
+    -0.6300023794, -0.7513582706, -0.6035611629, 0.5370919704,
+    0.2471584976, 0.7367672324, 0.5706370473, 0.9282674193,
+    0.7208684087, -0.7843156457, -0.2817355990, -0.1736787707,
+    0.4222335219, -0.4350655377, 0.4333281815, 0.5333415866,
+    0.1847889870, 0.7498788238, 0.6096553802, -0.3021556735,
+    -0.3898189068, 0.0377884321, 0.4024685621, 0.2031257302,
+    0.0107116764, -0.3112498820, 0.1999502629, -0.2273492515,
+    0.2892593443, 0.5372074246, 0.9408631325, 0.2907505929,
+)
+
+#: 3-bit Lloyd-Max centroids on the post-rotation coordinate distribution.
+#: Sourced verbatim from ISO_CENTROIDS_3BIT in ggml-iso-quant.c.
+_ISO3_CENTROIDS: Final[tuple[float, ...]] = (
+    -0.1906850000, -0.1178320000, -0.0657170000, -0.0214600000,
+    0.0214600000, 0.0657170000, 0.1178320000, 0.1906850000,
+)
+# fmt: on
+
+
+def quaternion_table() -> tuple[mx.array, mx.array, mx.array, mx.array]:
+    """Return the (qw, qx, qy, qz) quaternion-component arrays as fp32.
+
+    Each array has shape ``(ISO3_GROUPS_PER_BLOCK,)`` and the four arrays
+    together describe one unit quaternion per 4D rotation group.
+    """
+    return (
+        mx.array(_ISO3_QW, dtype=mx.float32),
+        mx.array(_ISO3_QX, dtype=mx.float32),
+        mx.array(_ISO3_QY, dtype=mx.float32),
+        mx.array(_ISO3_QZ, dtype=mx.float32),
+    )
+
+
+def centroid_table() -> mx.array:
+    """Return the 8-entry 3-bit centroid array as fp32."""
+    return mx.array(_ISO3_CENTROIDS, dtype=mx.float32)
diff --git a/src/exo/worker/engines/mlx/rotorquant/tests/__init__.py b/src/exo/worker/engines/mlx/rotorquant/tests/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/src/exo/worker/engines/mlx/rotorquant/tests/test_cache.py b/src/exo/worker/engines/mlx/rotorquant/tests/test_cache.py
new file mode 100644
index 000000000..96b1b962b
--- /dev/null
+++ b/src/exo/worker/engines/mlx/rotorquant/tests/test_cache.py
@@ -0,0 +1,213 @@
+"""Tests for ``RotorQuantKVCache`` including the deferred-prefill path."""
+
+from typing import cast
+
+import mlx.core as mx
+import pytest
+
+from exo.worker.engines.mlx.rotorquant.cache import RotorQuantKVCache
+from exo.worker.engines.mlx.rotorquant.tables import ISO3_BLOCK_SIZE
+
+
+def _kv_chunk(num_steps: int, *, seed: int = 0) -> tuple[mx.array, mx.array]:
+    """Build a (B, H, T, head_dim) prefill chunk for tests."""
+    keys = mx.random.normal(
+        shape=(1, 4, num_steps, ISO3_BLOCK_SIZE),
+        key=mx.random.key(seed),
+    )
+    values = mx.random.normal(
+        shape=(1, 4, num_steps, ISO3_BLOCK_SIZE),
+        key=mx.random.key(seed + 1),
+    )
+    return keys, values
+
+
+def test_deferred_prefill_keeps_storage_unallocated_during_prefill() -> None:
+    """Multi-token calls should not allocate the quantized live cache."""
+    cache = RotorQuantKVCache(defer_prefill=True)
+    keys, values = _kv_chunk(64)
+    cache.update_and_fetch(keys, values)
+
+    assert cache.is_deferred is True
+    assert cache.has_live_storage is False
+    # ``offset`` is the logical token count even during the deferred
+    # phase — it must advance so downstream RoPE positioning is correct.
+    assert cache.offset == 64
+    assert cache.size() == 64
+
+
+def test_deferred_prefill_flushes_on_first_decode_call() -> None:
+    """The first ``num_steps == 1`` call should flush the deferred buffer."""
+    cache = RotorQuantKVCache(defer_prefill=True)
+    pre_k, pre_v = _kv_chunk(48, seed=10)
+    cache.update_and_fetch(pre_k, pre_v)
+    assert cache.is_deferred is True
+
+    dec_k, dec_v = _kv_chunk(1, seed=11)
+    out_k, out_v = cache.update_and_fetch(dec_k, dec_v)
+
+    assert cache.is_deferred is False
+    assert cache.has_pending_storage is False
+    assert cache.has_live_storage is True
+    assert cache.offset == 49  # 48 prefill + 1 decode
+    assert out_k.shape == (1, 4, 49, ISO3_BLOCK_SIZE)
+    assert out_v.shape == (1, 4, 49, ISO3_BLOCK_SIZE)
+    assert out_k.dtype == dec_k.dtype
+
+
+def test_deferred_prefill_returns_raw_fp16_during_prefill() -> None:
+    """The fp16 view returned during prefill should be lossless."""
+    cache = RotorQuantKVCache(defer_prefill=True)
+    keys, values = _kv_chunk(32, seed=20)
+    out_k, out_v = cache.update_and_fetch(keys, values)
+
+    # Equality with the input within fp16 round-tripping tolerance.
+    diff_k = mx.max(mx.abs(out_k.astype(mx.float32) - keys.astype(mx.float32))).item()
+    diff_v = mx.max(mx.abs(out_v.astype(mx.float32) - values.astype(mx.float32))).item()
+    assert diff_k < 1e-3
+    assert diff_v < 1e-3
+
+
+def test_non_deferred_path_quantizes_immediately() -> None:
+    """With ``defer_prefill=False`` even prefill chunks hit the quantizer."""
+    cache = RotorQuantKVCache(defer_prefill=False)
+    keys, values = _kv_chunk(16, seed=30)
+    out_k, _ = cache.update_and_fetch(keys, values)
+
+    assert cache.is_deferred is False
+    assert cache.has_live_storage is True
+    assert cache.offset == 16
+    # Reconstruction is lossy through the 3-bit codebook — verify the
+    # output isn't bit-identical to the input but is still close in MSE.
+    err = (out_k.astype(mx.float32) - keys.astype(mx.float32)) ** 2
+    rel_mse = (mx.sum(err) / mx.sum(keys.astype(mx.float32) ** 2)).item()
+    assert 1e-6 < rel_mse < 0.2
+
+
+def test_decode_after_prefill_appends_correctly() -> None:
+    """After flush, additional decode tokens should append to the live cache."""
+    cache = RotorQuantKVCache(defer_prefill=True)
+    pre_k, pre_v = _kv_chunk(8, seed=40)
+    cache.update_and_fetch(pre_k, pre_v)
+
+    for step in range(3):
+        dk, dv = _kv_chunk(1, seed=50 + step)
+        out_k, _ = cache.update_and_fetch(dk, dv)
+        assert out_k.shape[2] == 9 + step
+
+    assert cache.offset == 11
+    assert cache.size() == 11
+
+
+def test_trim_works_in_both_phases() -> None:
+    """``trim`` should remove tokens from whichever buffer is live."""
+    pending_cache = RotorQuantKVCache(defer_prefill=True)
+    keys, values = _kv_chunk(20, seed=60)
+    pending_cache.update_and_fetch(keys, values)
+    assert pending_cache.size() == 20
+    pending_cache.trim(5)
+    assert pending_cache.size() == 15
+
+    flushed_cache = RotorQuantKVCache(defer_prefill=True)
+    flushed_cache.update_and_fetch(*_kv_chunk(10, seed=70))
+    flushed_cache.update_and_fetch(*_kv_chunk(1, seed=71))  # triggers flush
+    assert flushed_cache.size() == 11
+    flushed_cache.trim(3)
+    assert flushed_cache.size() == 8
+
+
+def test_deferred_prefill_advances_offset_across_chunks() -> None:
+    """Multi-chunk prefill must advance ``offset`` so RoPE position is right.
+
+    Regression for the bug where the deferred-prefill ``num_steps > 1``
+    branch appended to the scratch buffer but never updated
+    ``cache.offset``. Downstream attention reads ``cache.offset`` for the
+    RoPE position, so prompts longer than the per-chunk prefill step
+    (e.g. 4096 tokens) saw every chunk after the first positioned as if
+    it started at token 0, producing wrong logits.
+    """
+    cache = RotorQuantKVCache(defer_prefill=True)
+
+    chunk_one_k, chunk_one_v = _kv_chunk(100, seed=200)
+    cache.update_and_fetch(chunk_one_k, chunk_one_v)
+    assert cache.offset == 100
+    assert cache.size() == 100
+
+    chunk_two_k, chunk_two_v = _kv_chunk(150, seed=201)
+    cache.update_and_fetch(chunk_two_k, chunk_two_v)
+    assert cache.offset == 250
+    assert cache.size() == 250
+    assert cache.is_deferred is True
+    assert cache.has_live_storage is False
+
+    # After flush, ``offset`` should still equal the total prefill length
+    # (250) plus the decode token (1) — exactly 251.
+    decode_k, decode_v = _kv_chunk(1, seed=202)
+    out_k, _ = cache.update_and_fetch(decode_k, decode_v)
+    assert cache.is_deferred is False
+    assert cache.offset == 251
+    assert out_k.shape[2] == 251
+
+
+def test_first_call_decode_shaped_does_not_crash() -> None:
+    """A 1-token first call should flush an empty pending buffer cleanly.
+
+    Regression for the case where stream_generate or pipeline_parallel_prefill
+    delivers a 1-token prompt: ``update_and_fetch`` is called with
+    ``num_steps == 1`` before any deferred prefill has been appended. The
+    flush path must treat the empty-pending case as a no-op and let the
+    normal quantize-on-insert path handle the new token.
+    """
+    cache = RotorQuantKVCache(defer_prefill=True)
+    keys, values = _kv_chunk(1, seed=80)
+    out_k, out_v = cache.update_and_fetch(keys, values)
+
+    assert cache.is_deferred is False
+    assert cache.has_pending_storage is False
+    assert cache.has_live_storage is True
+    assert cache.offset == 1
+    assert out_k.shape == (1, 4, 1, ISO3_BLOCK_SIZE)
+    assert out_v.shape == (1, 4, 1, ISO3_BLOCK_SIZE)
+
+
+def test_state_round_trips_in_both_phases() -> None:
+    """``state`` must be a pytree of mx.array leaves only (no str tags).
+
+    MLX serializers and ``mx.eval`` walk the cache state expecting array
+    leaves; a Python ``str`` tag would raise at runtime.
+    """
+    pending = RotorQuantKVCache(defer_prefill=True)
+    pending.update_and_fetch(*_kv_chunk(8, seed=90))
+    pending_state_obj = pending.state
+    assert isinstance(pending_state_obj, tuple)
+    pending_state = cast(tuple[mx.array, ...], pending_state_obj)
+    assert len(pending_state) == 2
+    for leaf in pending_state:
+        assert isinstance(leaf, mx.array)
+
+    flushed = RotorQuantKVCache(defer_prefill=True)
+    flushed.update_and_fetch(*_kv_chunk(8, seed=91))
+    flushed.update_and_fetch(*_kv_chunk(1, seed=92))  # triggers flush
+    live_state_obj = flushed.state
+    assert isinstance(live_state_obj, tuple)
+    live_state = cast(tuple[mx.array, ...], live_state_obj)
+    assert len(live_state) == 4
+    for leaf in live_state:
+        assert isinstance(leaf, mx.array)
+
+    # Round-trip via the setter into a fresh cache and confirm the offset
+    # and observable size match.
+    restored = RotorQuantKVCache(defer_prefill=False)
+    restored._quantizer = flushed._quantizer  # type: ignore[reportPrivateUsage]
+    restored.head_dim = flushed.head_dim
+    restored.state = live_state
+    assert restored.offset == flushed.offset
+    assert restored.size() == flushed.size()
+
+
+def test_head_dim_must_be_block_aligned() -> None:
+    cache = RotorQuantKVCache(defer_prefill=False)
+    bad_keys = mx.zeros((1, 1, 2, 100), dtype=mx.float16)
+    bad_values = mx.zeros((1, 1, 2, 100), dtype=mx.float16)
+    with pytest.raises(ValueError):
+        cache.update_and_fetch(bad_keys, bad_values)
diff --git a/src/exo/worker/engines/mlx/rotorquant/tests/test_iso_quantizer.py b/src/exo/worker/engines/mlx/rotorquant/tests/test_iso_quantizer.py
new file mode 100644
index 000000000..8588fcd43
--- /dev/null
+++ b/src/exo/worker/engines/mlx/rotorquant/tests/test_iso_quantizer.py
@@ -0,0 +1,118 @@
+"""Correctness tests for the IsoQuant 3-bit quantizer.
+
+A bit-exact comparison against a precomputed llama.cpp CPU reference
+fixture is desirable but blocked on producing the fixture (which
+requires building the llama.cpp fork). For v1 we validate via
+self-consistency properties that any correct implementation must
+satisfy: rotation orthogonality, round-trip reconstruction error,
+norm-correction unbiasedness, and shape contracts.
+"""
+
+import math
+
+import mlx.core as mx
+import pytest
+
+from exo.worker.engines.mlx.rotorquant.quantizer import IsoQuantizer
+from exo.worker.engines.mlx.rotorquant.rotation import (
+    iso3_rotate_forward,
+    iso3_rotate_inverse,
+)
+from exo.worker.engines.mlx.rotorquant.tables import ISO3_BLOCK_SIZE
+
+
+def _random_kv(shape: tuple[int, ...], seed: int = 0) -> mx.array:
+    """Sample a Gaussian tensor matching a typical attention KV layout."""
+    return mx.random.normal(shape=shape, key=mx.random.key(seed))
+
+
+def test_quaternion_rotation_is_orthogonal() -> None:
+    """``||R x|| == ||x||`` for the per-block rotation."""
+    x = _random_kv((2, 4, 8, ISO3_BLOCK_SIZE))
+    rotated = iso3_rotate_forward(x.astype(mx.float32))
+    norm_in = mx.sqrt(mx.sum(x * x, axis=-1))
+    norm_out = mx.sqrt(mx.sum(rotated * rotated, axis=-1))
+    diff = mx.max(mx.abs(norm_in - norm_out)).item()
+    assert diff < 1e-4, f"rotation not orthogonal: max ||x|| - ||Rx|| = {diff}"
+
+
+def test_rotation_inverse_round_trips() -> None:
+    """``R^{-1} R x == x`` to floating-point precision."""
+    x = _random_kv((1, 2, 3, ISO3_BLOCK_SIZE), seed=1).astype(mx.float32)
+    restored = iso3_rotate_inverse(iso3_rotate_forward(x))
+    diff = mx.max(mx.abs(x - restored)).item()
+    assert diff < 1e-4, f"R^-1 R x diverged by {diff}"
+
+
+def test_quantize_dequantize_shapes() -> None:
+    """Indices and norms have the documented shapes for a typical KV layer."""
+    quantizer = IsoQuantizer()
+    keys = _random_kv((2, 8, 16, 256))  # head_dim 256 = 2 iso3 blocks
+    indices, norms = quantizer.quantize(keys)
+    assert indices.shape == (2, 8, 16, 256)
+    assert indices.dtype == mx.uint8
+    assert norms.shape == (2, 8, 16, 2)
+    assert norms.dtype == mx.float16
+    restored = quantizer.dequantize(indices, norms, mx.float16)
+    assert restored.shape == keys.shape
+    assert restored.dtype == mx.float16
+
+
+def test_quantize_indices_in_valid_range() -> None:
+    """Index values must lie in ``[0, 7]`` for the 3-bit codebook."""
+    quantizer = IsoQuantizer()
+    x = _random_kv((1, 1, 4, ISO3_BLOCK_SIZE), seed=2)
+    indices, _ = quantizer.quantize(x)
+    max_idx = int(mx.max(indices).item())
+    min_idx = int(mx.min(indices).item())
+    assert min_idx >= 0
+    assert max_idx <= 7
+
+
+def test_round_trip_reconstruction_is_within_3bit_budget() -> None:
+    """End-to-end reconstruction error stays within the 3-bit centroid budget.
+
+    With 8 Lloyd-Max centroids on a Gaussian-ish post-rotation
+    distribution, the per-element MSE should be around 0.04 of the
+    input variance. We use a generous threshold (0.15) so the test
+    catches outright bugs without flaking on RNG variance.
+    """
+    quantizer = IsoQuantizer()
+    x = _random_kv((1, 4, 32, ISO3_BLOCK_SIZE), seed=3).astype(mx.float32)
+    indices, norms = quantizer.quantize(x)
+    x_hat = quantizer.dequantize(indices, norms, mx.float32)
+
+    err = x - x_hat
+    rel_mse = (mx.sum(err * err) / mx.sum(x * x)).item()
+    assert rel_mse < 0.15, f"3-bit round-trip relative MSE {rel_mse} exceeds budget"
+
+
+def test_norm_correction_keeps_magnitudes_unbiased() -> None:
+    """The norm-correction trick should leave per-block ``||x||`` unbiased.
+
+    Without correction, centroid quantization shrinks the unit-vector
+    norm and the dequantized output systematically underestimates the
+    input magnitude. With correction the average magnitude ratio
+    should sit at ~1.0 across many random blocks.
+    """
+    quantizer = IsoQuantizer()
+    x = _random_kv((4, 8, 64, ISO3_BLOCK_SIZE), seed=4).astype(mx.float32)
+    indices, norms = quantizer.quantize(x)
+    x_hat = quantizer.dequantize(indices, norms, mx.float32)
+
+    norm_in = mx.sqrt(mx.sum(x * x, axis=-1))
+    norm_out = mx.sqrt(mx.sum(x_hat * x_hat, axis=-1))
+    ratio = (norm_out / mx.maximum(norm_in, 1e-8)).reshape(-1)
+    mean_ratio = float(mx.mean(ratio).item())
+    # Slack covers fp16 round-tripping plus the variance of a finite
+    # sample. The unbiased target is exactly 1.0.
+    assert math.isclose(mean_ratio, 1.0, abs_tol=0.05), (
+        f"norm-correction biased mean ratio = {mean_ratio}"
+    )
+
+
+def test_quantize_rejects_non_multiple_head_dim() -> None:
+    quantizer = IsoQuantizer()
+    bad = _random_kv((1, 1, 1, 100), seed=5)  # 100 is not a multiple of 128
+    with pytest.raises(ValueError):
+        quantizer.quantize(bad)
diff --git a/src/exo/worker/engines/mlx/utils_mlx.py b/src/exo/worker/engines/mlx/utils_mlx.py
index f1a411e85..06fc8399b 100644
--- a/src/exo/worker/engines/mlx/utils_mlx.py
+++ b/src/exo/worker/engines/mlx/utils_mlx.py
@@ -117,8 +117,7 @@ def log_request_shape(
         payload.update(extra)
 
     logger.info(
-        "[request-shape] "
-        f"{json.dumps(payload, ensure_ascii=True, sort_keys=True)}"
+        f"[request-shape] {json.dumps(payload, ensure_ascii=True, sort_keys=True)}"
     )
     logger.info(f"[request-shape] prompt label={label}\n{prompt}")
 
@@ -287,7 +286,9 @@ def _encode_one(self, pixel_values: mx.array) -> mx.array:
         hidden_states = mx.concatenate(all_real, axis=0)[None]
 
         if getattr(getattr(self._inner, "config", None), "standardize", False):
-            hidden_states = (hidden_states - self._inner.std_bias) * self._inner.std_scale
+            hidden_states = (
+                hidden_states - self._inner.std_bias
+            ) * self._inner.std_scale
 
         return hidden_states
 
@@ -840,6 +841,7 @@ def apply_chat_template(
     tokenizer: TokenizerWrapper,
     task_params: TextGenerationTaskParams,
     model_card: ModelCard | None = None,
+    suppress_empty_gemma4_thought_channel: bool = False,
 ) -> str:
     """Convert TextGenerationTaskParams to a chat template prompt.
 
@@ -901,6 +903,7 @@ def apply_chat_template(
             formatted_messages,
             add_generation_prompt=True,
             enable_thinking=task_params.enable_thinking,
+            suppress_empty_thought_channel=suppress_empty_gemma4_thought_channel,
         )
         if partial_assistant_content:
             prompt += partial_assistant_content
@@ -1195,7 +1198,21 @@ def _parse_single_tool(text: str) -> dict[str, Any]:
 def mx_all_gather_tasks(
     tasks: list[TextGeneration],
     group: mx.distributed.Group | None,
+    label: str = "tasks",
 ) -> tuple[list[TextGeneration], list[TextGeneration]]:
+    """Synchronize task lists across all ranks using only all_sum.
+
+    JACCL's ``all_gather`` deadlocks intermittently on 3-node pipelines —
+    the first all_gather in a sequence may succeed, but subsequent ones
+    hang.  ``all_sum`` is proven reliable (used by ``mx_any``,
+    ``mx_barrier``, and PipelineLastLayer decode broadcast without issue).
+
+    Strategy: use the same zero-contribution pattern as PipelineLastLayer.
+    Each rank fills its own slot in a ``[world_size, max_tasks, id_len]``
+    tensor with encoded task IDs and contributes zeros elsewhere.
+    ``all_sum`` merges them (non-overlapping slots ⇒ sum == gather).
+    """
+
     def encode_task_id(task_id: TaskId) -> list[int]:
         utf8_task_id = task_id.encode()
         return [
@@ -1208,30 +1225,53 @@ def decode_task_id(encoded_task_id: list[int]) -> TaskId:
         )
 
     uuid_byte_length = 36
+    cpu_stream = mx.default_stream(mx.Device(mx.cpu))
+    world_size: int = 1 if group is None else group.size()
+    rank: int = 0 if group is None else group.rank()
 
+    # --- Phase 1: exchange counts via all_sum ---
+    # Each rank contributes its count in its own slot of a [world_size]
+    # vector; other slots are zero.  all_sum merges them.
     n_tasks = len(tasks)
+    logger.debug(f"mx_all_gather_tasks[{label}]: gathering counts (n_tasks={n_tasks})")
+    counts_vec = [0] * world_size
+    counts_vec[rank] = n_tasks
     all_counts = cast(
         list[int],
-        mx.distributed.all_gather(mx.array([n_tasks]), group=group).tolist(),
+        mx.distributed.all_sum(
+            mx.array(counts_vec, dtype=mx.int32),
+            group=group,
+            stream=cpu_stream,
+        ).tolist(),
     )
+    logger.debug(f"mx_all_gather_tasks[{label}]: counts gathered: {all_counts}")
     max_tasks = max(all_counts)
-    world_size: int = 1 if group is None else group.size()
 
     if max_tasks == 0:
         return [], []
 
-    padded = [encode_task_id(task.task_id) for task in tasks] + [
+    # --- Phase 2: exchange task IDs via all_sum ---
+    # Build a [world_size, max_tasks, uuid_byte_length] tensor.  This rank
+    # fills its own slice; all other slices are zero.  all_sum merges them
+    # without overlap (each rank owns a unique slice).
+    padded_ids = [encode_task_id(task.task_id) for task in tasks] + [
         [0] * uuid_byte_length
     ] * (max_tasks - n_tasks)
+    assert all(len(eid) == uuid_byte_length for eid in padded_ids)
 
-    assert all(len(encoded_task_id) == uuid_byte_length for encoded_task_id in padded)
+    full = [[[0] * uuid_byte_length] * max_tasks for _ in range(world_size)]
+    full[rank] = padded_ids
 
+    gathered_flat = mx.distributed.all_sum(
+        mx.array(full, dtype=mx.int32),
+        group=group,
+        stream=cpu_stream,
+    )
     gathered = cast(
         list[list[list[int]]],
-        mx.distributed.all_gather(mx.array(padded), group=group)
-        .reshape(world_size, max_tasks, -1)
-        .tolist(),
+        gathered_flat.reshape(world_size, max_tasks, uuid_byte_length).tolist(),
     )
+
     all_task_ids: list[list[TaskId]] = [
         [decode_task_id(encoded_task_id) for encoded_task_id in rank_tasks[:count]]
         for rank_tasks, count in zip(gathered, all_counts, strict=True)
diff --git a/src/exo/worker/runner/llm_inference/batch_generator.py b/src/exo/worker/runner/llm_inference/batch_generator.py
index d0dc44a11..6065f75b2 100644
--- a/src/exo/worker/runner/llm_inference/batch_generator.py
+++ b/src/exo/worker/runner/llm_inference/batch_generator.py
@@ -62,6 +62,7 @@ def gen(self) -> Generator[T | None]:
 
 
 class InferenceGenerator(ABC):
+    group: mx.distributed.Group | None
     _cancelled_tasks: set[TaskId]
 
     def should_cancel(self, task_id: TaskId) -> bool:
@@ -93,6 +94,7 @@ def close(self) -> None: ...
 EXO_RUNNER_MUST_FAIL = "EXO RUNNER MUST FAIL"
 EXO_RUNNER_MUST_OOM = "EXO RUNNER MUST OOM"
 EXO_RUNNER_MUST_TIMEOUT = "EXO RUNNER MUST TIMEOUT"
+_DEFAULT_CANCEL_CHECK_INTERVAL = 50
 
 
 def _check_for_debug_prompts(task_params: TextGenerationTaskParams) -> None:
@@ -112,6 +114,18 @@ def _check_for_debug_prompts(task_params: TextGenerationTaskParams) -> None:
         time.sleep(100)
 
 
+def _safe_cancel_check_interval(interval: int) -> int:
+    """Normalize warmup-derived cancellation polling intervals for live generation."""
+    if interval > 0:
+        return interval
+
+    logger.warning(
+        "warmup returned a non-positive cancellation interval "
+        f"({interval}); using {_DEFAULT_CANCEL_CHECK_INTERVAL} tokens"
+    )
+    return _DEFAULT_CANCEL_CHECK_INTERVAL
+
+
 @dataclass(eq=False)
 class SequentialGenerator(InferenceGenerator):
     model: Model
@@ -147,12 +161,14 @@ class SequentialGenerator(InferenceGenerator):
     _logged_first_request_shape: bool = field(default=False, init=False)
 
     def warmup(self):
-        self.check_for_cancel_every = warmup_inference(
-            model=self.model,
-            tokenizer=self.tokenizer,
-            group=self.group,
-            model_id=self.model_id,
-            model_card=self.model_card,
+        self.check_for_cancel_every = _safe_cancel_check_interval(
+            warmup_inference(
+                model=self.model,
+                tokenizer=self.tokenizer,
+                group=self.group,
+                model_id=self.model_id,
+                model_card=self.model_card,
+            )
         )
 
     def submit(
@@ -165,7 +181,17 @@ def submit(
 
     def agree_on_tasks(self) -> None:
         """Agree between all ranks about the task ordering (some may have received in different order or not at all)."""
-        agreed, different = mx_all_gather_tasks(self._maybe_queue, self.group)
+        if not mx_any(bool(self._maybe_queue), self.group):
+            logger.debug("agree_on_tasks: no pending tasks on any rank")
+            return
+
+        logger.info(
+            f"agree_on_tasks: entering with {len(self._maybe_queue)} pending tasks"
+        )
+        agreed, different = mx_all_gather_tasks(
+            self._maybe_queue, self.group, label="task-queue"
+        )
+        logger.info(f"agree_on_tasks: {len(agreed)} agreed, {len(different)} different")
         self._queue.extend(task for task in self._maybe_queue if task in agreed)
         self._maybe_queue = [task for task in self._maybe_queue if task in different]
 
@@ -182,7 +208,13 @@ def agree_on_cancellations(self) -> None:
         if mx_any(has_cancel_all, self.group):
             self._cancelled_tasks.add(CANCEL_ALL_TASKS)
 
-        agreed, different = mx_all_gather_tasks(self._maybe_cancel, self.group)
+        if not mx_any(bool(self._maybe_cancel), self.group):
+            logger.debug("agree_on_cancellations: no pending cancellations on any rank")
+            return
+
+        agreed, different = mx_all_gather_tasks(
+            self._maybe_cancel, self.group, label="cancellations"
+        )
         self._cancelled_tasks.update(task.task_id for task in agreed)
         self._maybe_cancel = list(different)
 
@@ -377,12 +409,14 @@ def __post_init__(self) -> None:
         )
 
     def warmup(self):
-        self.check_for_cancel_every = warmup_inference(
-            model=self.model,
-            tokenizer=self.tokenizer,
-            group=self.group,
-            model_id=self.model_id,
-            model_card=self.model_card,
+        self.check_for_cancel_every = _safe_cancel_check_interval(
+            warmup_inference(
+                model=self.model,
+                tokenizer=self.tokenizer,
+                group=self.group,
+                model_id=self.model_id,
+                model_card=self.model_card,
+            )
         )
 
     def submit(
@@ -395,7 +429,17 @@ def submit(
 
     def agree_on_tasks(self) -> None:
         """Agree between all ranks about the task ordering (some may have received in different order or not at all)."""
-        agreed, different = mx_all_gather_tasks(self._maybe_queue, self.group)
+        if not mx_any(bool(self._maybe_queue), self.group):
+            logger.debug("agree_on_tasks: no pending tasks on any rank")
+            return
+
+        logger.info(
+            f"agree_on_tasks: entering with {len(self._maybe_queue)} pending tasks"
+        )
+        agreed, different = mx_all_gather_tasks(
+            self._maybe_queue, self.group, label="task-queue"
+        )
+        logger.info(f"agree_on_tasks: {len(agreed)} agreed, {len(different)} different")
         self._queue.extend(task for task in self._maybe_queue if task in agreed)
         self._maybe_queue = [task for task in self._maybe_queue if task in different]
 
@@ -412,7 +456,13 @@ def agree_on_cancellations(self) -> None:
         if mx_any(has_cancel_all, self.group):
             self._cancelled_tasks.add(CANCEL_ALL_TASKS)
 
-        agreed, different = mx_all_gather_tasks(self._maybe_cancel, self.group)
+        if not mx_any(bool(self._maybe_cancel), self.group):
+            logger.debug("agree_on_cancellations: no pending cancellations on any rank")
+            return
+
+        agreed, different = mx_all_gather_tasks(
+            self._maybe_cancel, self.group, label="cancellations"
+        )
         self._cancelled_tasks.update(task.task_id for task in agreed)
         self._maybe_cancel = list(different)
 
diff --git a/src/exo/worker/runner/llm_inference/runner.py b/src/exo/worker/runner/llm_inference/runner.py
index 49075cae5..080cb0338 100644
--- a/src/exo/worker/runner/llm_inference/runner.py
+++ b/src/exo/worker/runner/llm_inference/runner.py
@@ -7,7 +7,13 @@
 from anyio import WouldBlock
 from mlx_lm.tokenizer_utils import TokenizerWrapper
 
-from exo.shared.models.model_cards import ModelCard, ModelTask
+from exo.shared.models.model_cards import (
+    ModelCard,
+    ModelTask,
+    OutputParserType,
+    PromptRendererType,
+    ToolCallFormat,
+)
 from exo.shared.types.chunks import (
     ErrorChunk,
     TokenChunk,
@@ -91,6 +97,28 @@ def _should_skip_llm_warmup(group_size: int) -> bool:
     return True
 
 
+def _is_gemma4_model(model_id: ModelId, model_card: ModelCard | None) -> bool:
+    """Return whether a model should be treated as a Gemma 4 runtime."""
+    normalized_model_id = str(model_id).lower().replace("_", "-")
+    if "gemma-4" in normalized_model_id or "gemma4" in normalized_model_id:
+        return True
+    if model_card is None:
+        return False
+
+    if model_card.vision is not None and model_card.vision.model_type == "gemma4":
+        return True
+
+    runtime = model_card.runtime
+    if runtime is not None and (
+        runtime.prompt_renderer == PromptRendererType.Gemma4
+        or runtime.output_parser == OutputParserType.Gemma4
+    ):
+        return True
+
+    tooling = model_card.tooling
+    return tooling is not None and tooling.tool_call_format == ToolCallFormat.Gemma4
+
+
 class ExitCode(str, Enum):
     AllTasksComplete = "AllTasksComplete"
     Shutdown = "Shutdown"
@@ -484,20 +512,40 @@ def build(
         kv_backend = get_kv_cache_backend()
         # TODO: Remove this forced sequential fallback once quantized KV cache
         # backends support BatchGenerator history merge/extract semantics.
-        force_sequential = kv_backend in (
+        force_sequential_for_kv_backend = kv_backend in (
             "mlx_quantized",
             "turboquant",
             "turboquant_adaptive",
             "optiq",
+            "rotorquant",
+            "rotorquant_adaptive",
+        )
+        # Gemma 4 uses sliding-window RotatingKVCache entries. In distributed
+        # pipeline mode, the MLX BatchGenerator path currently produces
+        # degenerate token repetition after a valid prefill, while the
+        # sequential path is stable with the same prompt and default KV cache.
+        force_sequential_for_gemma4 = _is_gemma4_model(self.model_id, self.model_card)
+        no_batch_requested = os.environ.get("SKULK_NO_BATCH") or os.environ.get(
+            "EXO_NO_BATCH"
         )
-        if os.environ.get("EXO_NO_BATCH") or force_sequential:
-            if force_sequential and not os.environ.get("EXO_NO_BATCH"):
+        if (
+            no_batch_requested
+            or force_sequential_for_kv_backend
+            or force_sequential_for_gemma4
+        ):
+            if force_sequential_for_kv_backend and not no_batch_requested:
                 logger.warning(
                     "Quantized KV cache backend does not yet support "
                     "batch/history mode; forcing SequentialGenerator "
                     f"(kv_backend={kv_backend})"
                 )
                 logger.info(f"using SequentialGenerator (kv_backend={kv_backend})")
+            elif force_sequential_for_gemma4 and not no_batch_requested:
+                logger.warning(
+                    "Gemma 4 is not compatible with distributed BatchGenerator "
+                    "mode yet; forcing SequentialGenerator"
+                )
+                logger.info("using SequentialGenerator (model_family=gemma4)")
             else:
                 logger.info("using SequentialGenerator (batching disabled)")
             return SequentialGenerator(
diff --git a/src/exo/worker/tests/unittests/test_mlx/test_gemma4_prompt.py b/src/exo/worker/tests/unittests/test_mlx/test_gemma4_prompt.py
index 6bb0331c5..bdc9a97fb 100644
--- a/src/exo/worker/tests/unittests/test_mlx/test_gemma4_prompt.py
+++ b/src/exo/worker/tests/unittests/test_mlx/test_gemma4_prompt.py
@@ -1,7 +1,7 @@
 from exo.worker.engines.mlx.gemma4_prompt import render_gemma4_prompt
 
 
-def test_render_gemma4_prompt_does_not_append_empty_thought_channel_when_thinking_disabled():
+def test_render_gemma4_prompt_appends_empty_thought_channel_when_thinking_disabled():
     prompt = render_gemma4_prompt(
         [
             {"role": "system", "content": "You are helpful."},
@@ -20,7 +20,20 @@ def test_render_gemma4_prompt_does_not_append_empty_thought_channel_when_thinkin
         "Hello"
         "<turn|>\n"
         "<|turn>model\n"
+        "<|channel>thought\n"
+        "<channel|>"
     )
+
+
+def test_render_gemma4_prompt_can_suppress_empty_thought_channel_for_warmup():
+    prompt = render_gemma4_prompt(
+        [{"role": "user", "content": "Hello"}],
+        add_generation_prompt=True,
+        enable_thinking=False,
+        suppress_empty_thought_channel=True,
+    )
+
+    assert prompt.endswith("<|turn>model\n")
     assert "<|channel>thought" not in prompt
     assert "<channel|>" not in prompt
 
diff --git a/src/exo/worker/tests/unittests/test_mlx/test_kv_prefix_cache.py b/src/exo/worker/tests/unittests/test_mlx/test_kv_prefix_cache.py
index 6d2a077e3..d6738f757 100644
--- a/src/exo/worker/tests/unittests/test_mlx/test_kv_prefix_cache.py
+++ b/src/exo/worker/tests/unittests/test_mlx/test_kv_prefix_cache.py
@@ -1,4 +1,5 @@
 # type: ignore
+import os
 import time
 from typing import cast
 from unittest.mock import patch
@@ -109,15 +110,32 @@ def test_clear_on_empty_cache(self, mock_tokenizer):
 
 class TestKVCacheBackends:
     def test_unknown_backend_falls_back_to_default(self):
-        with patch(
-            "exo.worker.engines.mlx.cache.KV_CACHE_BACKEND",
-            cast(object, "mystery"),
+        with patch.dict(os.environ, {"SKULK_KV_CACHE_BACKEND": "mystery"}):
+            assert get_kv_cache_backend() == "default"
+
+    def test_rotorquant_backend_requires_experimental_gate(self):
+        with patch.dict(
+            os.environ,
+            {
+                "SKULK_KV_CACHE_BACKEND": "rotorquant_adaptive",
+                "SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT": "",
+            },
         ):
             assert get_kv_cache_backend() == "default"
 
+    def test_rotorquant_backend_can_be_explicitly_enabled(self):
+        with patch.dict(
+            os.environ,
+            {
+                "SKULK_KV_CACHE_BACKEND": "rotorquant",
+                "SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT": "1",
+            },
+        ):
+            assert get_kv_cache_backend() == "rotorquant"
+
     def test_make_kv_cache_default_backend(self):
         model = cast(Model, type("FakeModel", (), {"layers": [object(), object()]})())
-        with patch("exo.worker.engines.mlx.cache.KV_CACHE_BACKEND", "default"):
+        with patch.dict(os.environ, {"SKULK_KV_CACHE_BACKEND": "default"}):
             cache = make_kv_cache(model)
 
         assert len(cache) == 2
@@ -126,7 +144,7 @@ def test_make_kv_cache_default_backend(self):
     def test_make_kv_cache_mlx_quantized_backend(self):
         model = cast(Model, type("FakeModel", (), {"layers": [object(), object()]})())
         with (
-            patch("exo.worker.engines.mlx.cache.KV_CACHE_BACKEND", "mlx_quantized"),
+            patch.dict(os.environ, {"SKULK_KV_CACHE_BACKEND": "mlx_quantized"}),
             patch("exo.worker.engines.mlx.cache.KV_CACHE_BITS", 4),
             patch("exo.worker.engines.mlx.cache.CACHE_GROUP_SIZE", 32),
         ):
@@ -144,7 +162,7 @@ def make_cache(self):
 
         model = cast(Model, FakeModel())
         with (
-            patch("exo.worker.engines.mlx.cache.KV_CACHE_BACKEND", "mlx_quantized"),
+            patch.dict(os.environ, {"SKULK_KV_CACHE_BACKEND": "mlx_quantized"}),
             patch("exo.worker.engines.mlx.cache.KV_CACHE_BITS", 4),
             patch("exo.worker.engines.mlx.cache.CACHE_GROUP_SIZE", 32),
         ):
@@ -163,7 +181,7 @@ def make_cache(self):
 
         model = cast(Model, FakeModel())
         with (
-            patch("exo.worker.engines.mlx.cache.KV_CACHE_BACKEND", "mlx_quantized"),
+            patch.dict(os.environ, {"SKULK_KV_CACHE_BACKEND": "mlx_quantized"}),
             patch("exo.worker.engines.mlx.cache.KV_CACHE_BITS", 4),
             patch("exo.worker.engines.mlx.cache.CACHE_GROUP_SIZE", 32),
         ):
@@ -176,7 +194,7 @@ def make_cache(self):
     def test_make_kv_cache_mlx_quantized_backend_requires_bits(self):
         model = cast(Model, type("FakeModel", (), {"layers": [object()]})())
         with (
-            patch("exo.worker.engines.mlx.cache.KV_CACHE_BACKEND", "mlx_quantized"),
+            patch.dict(os.environ, {"SKULK_KV_CACHE_BACKEND": "mlx_quantized"}),
             patch("exo.worker.engines.mlx.cache.KV_CACHE_BITS", None),
             pytest.raises(ValueError, match="EXO_KV_CACHE_BITS"),
         ):
@@ -191,7 +209,7 @@ def make_cache(self):
 
         model = cast(Model, FakeModel())
         with (
-            patch("exo.worker.engines.mlx.cache.KV_CACHE_BACKEND", "turboquant"),
+            patch.dict(os.environ, {"SKULK_KV_CACHE_BACKEND": "turboquant"}),
             patch("exo.worker.engines.mlx.cache.TURBOQUANT_K_BITS", 3),
             patch("exo.worker.engines.mlx.cache.TURBOQUANT_V_BITS", 4),
         ):
@@ -208,10 +226,7 @@ def make_cache(self):
 
         model = cast(Model, FakeModel())
         with (
-            patch(
-                "exo.worker.engines.mlx.cache.KV_CACHE_BACKEND",
-                "turboquant_adaptive",
-            ),
+            patch.dict(os.environ, {"SKULK_KV_CACHE_BACKEND": "turboquant_adaptive"}),
             patch("exo.worker.engines.mlx.cache.TURBOQUANT_K_BITS", 3),
             patch("exo.worker.engines.mlx.cache.TURBOQUANT_V_BITS", 4),
             patch("exo.worker.engines.mlx.cache.TURBOQUANT_FP16_LAYERS", 1),
diff --git a/src/exo/worker/tests/unittests/test_mlx/test_prefill_path_selection.py b/src/exo/worker/tests/unittests/test_mlx/test_prefill_path_selection.py
index 7049aa9fe..e5d79c219 100644
--- a/src/exo/worker/tests/unittests/test_mlx/test_prefill_path_selection.py
+++ b/src/exo/worker/tests/unittests/test_mlx/test_prefill_path_selection.py
@@ -97,17 +97,15 @@ def _pipeline_parallel_prefill(**kwargs: object) -> None:
     assert prefill_mode_calls == [True, False]
 
 
-def test_prefill_uses_stream_generate_for_short_pipeline_prompts(
+def test_prefill_uses_pipeline_parallel_path_for_short_pipeline_prompts(
     monkeypatch: pytest.MonkeyPatch,
 ) -> None:
-    """Short prompts that fit in a single per-rank chunk must avoid pipeline prefill.
-
-    Regression for the Gemma 4 warmup wedge: pipeline_parallel_prefill hangs
-    when ``n_real == 1`` (the prompt fits inside a single per-rank chunk),
-    so any such prompt — including the 23-token warmup prompt — must route
-    through stream_generate even on pipeline-parallel models. PR #101's
-    pipeline-prefill widening removed this guard and reintroduced the hang
-    on Gemma 4; this test locks the guard back in.
+    """Short pipeline prompts must avoid upstream stream_generate prefill.
+
+    ``mlx_lm.generate_step`` prefetches a decode step before yielding even with
+    ``max_tokens=1``. In pipeline mode that hidden prefetch can leave sends/recvs
+    in flight, so short prompts use Skulk's explicit pipeline prefill path while
+    suppressing distributed progress callbacks.
     """
     calls: list[str] = []
     fake_cache = _FakeCache()
@@ -127,6 +125,12 @@ def test_prefill_uses_stream_generate_for_short_pipeline_prompts(
 
     def _pipeline_parallel_prefill(**kwargs: object) -> None:
         calls.append("pipeline_parallel_prefill")
+        assert kwargs["distributed_prompt_progress_callback"] is None
+        prompt_progress_callback = kwargs["prompt_progress_callback"]
+        assert callable(prompt_progress_callback)
+        prompt = kwargs["prompt"]
+        assert isinstance(prompt, list)
+        prompt_progress_callback(len(prompt), len(prompt))
 
     monkeypatch.setattr(
         generate_module,
@@ -140,6 +144,9 @@ def _stream_generate(*args: object, **kwargs: object) -> Iterator[object]:
 
     monkeypatch.setattr(generate_module, "stream_generate", _stream_generate)
 
+    def _fail_distributed_callback() -> None:
+        raise AssertionError("single-chunk pipeline prefill should not poll tasks")
+
     prefill_tps, prefill_tokens, snapshots = generate_module.prefill(
         model=_FakeModel(),
         tokenizer=object(),
@@ -148,22 +155,22 @@ def _stream_generate(*args: object, **kwargs: object) -> Iterator[object]:
         cache=[fake_cache],
         group=_FakeGroup(),
         on_prefill_progress=None,
-        distributed_prompt_progress_callback=None,
+        distributed_prompt_progress_callback=_fail_distributed_callback,
     )
 
-    assert calls == ["stream_generate"]
-    assert "pipeline_parallel_prefill" not in calls
+    assert calls == ["pipeline_parallel_prefill"]
+    assert "stream_generate" not in calls
     assert prefill_tokens == 23
     assert snapshots == []
     assert prefill_tps >= 0.0
     assert fake_cache.trim_calls == [2]
-    assert prefill_mode_calls == [False, False]
+    assert prefill_mode_calls == [True, False]
 
 
-def test_prefill_uses_stream_generate_at_single_chunk_boundary(
+def test_prefill_uses_pipeline_parallel_path_at_single_chunk_boundary(
     monkeypatch: pytest.MonkeyPatch,
 ) -> None:
-    """Prompts with only one real pipeline chunk must stay off pipeline prefill."""
+    """Prompts with one real pipeline chunk still use explicit pipeline prefill."""
     calls: list[str] = []
     fake_cache = _FakeCache()
     prefill_mode_calls: list[bool] = []
@@ -202,12 +209,12 @@ def _stream_generate(*args: object, **kwargs: object) -> Iterator[object]:
         distributed_prompt_progress_callback=None,
     )
 
-    assert calls == ["stream_generate"]
+    assert calls == ["pipeline_parallel_prefill"]
     assert prefill_tokens == 1366
     assert snapshots == []
     assert prefill_tps >= 0.0
     assert fake_cache.trim_calls == [2]
-    assert prefill_mode_calls == [False, False]
+    assert prefill_mode_calls == [True, False]
 
 
 def test_prefill_resets_pipeline_flags_after_unexpected_exception(
diff --git a/src/exo/worker/tests/unittests/test_mlx/test_warmup_request.py b/src/exo/worker/tests/unittests/test_mlx/test_warmup_request.py
index dcf1f7e1d..9ae003ac7 100644
--- a/src/exo/worker/tests/unittests/test_mlx/test_warmup_request.py
+++ b/src/exo/worker/tests/unittests/test_mlx/test_warmup_request.py
@@ -8,11 +8,17 @@
 
 
 class _FakeGroup:
+    def rank(self) -> int:
+        return 0
+
     def size(self) -> int:
         return 3
 
 
 class _SingleNodeGroup:
+    def rank(self) -> int:
+        return 0
+
     def size(self) -> int:
         return 1
 
@@ -33,9 +39,18 @@ def test_warmup_inference_uses_safe_default_user_content_without_instructions(
 ) -> None:
     captured: dict[str, object] = {}
 
-    def fake_apply_chat_template(*, tokenizer: object, task_params: object, model_card: object):
+    def fake_apply_chat_template(
+        *,
+        tokenizer: object,
+        task_params: object,
+        model_card: object,
+        suppress_empty_gemma4_thought_channel: bool = False,
+    ):
         del tokenizer, model_card
         captured["task_params"] = task_params
+        captured["suppress_empty_gemma4_thought_channel"] = (
+            suppress_empty_gemma4_thought_channel
+        )
         return "warmup prompt"
 
     def fake_mx_barrier(_group: object) -> None:
@@ -46,8 +61,8 @@ def fake_mlx_generate(**_kwargs: object):
             yield None
         return
 
-    def fake_all_gather(array: object, *, group: object):
-        del group
+    def fake_all_sum(array: object, *, group: object, stream: object = None):
+        del group, stream
         return array
 
     def fake_log_request_shape(
@@ -67,7 +82,7 @@ def fake_log_request_shape(
     monkeypatch.setattr(generate_mod, "mx_barrier", fake_mx_barrier)
     monkeypatch.setattr(generate_mod, "mlx_generate", fake_mlx_generate)
     monkeypatch.setattr(generate_mod, "log_request_shape", fake_log_request_shape)
-    monkeypatch.setattr(generate_mod.mx.distributed, "all_gather", fake_all_gather)
+    monkeypatch.setattr(generate_mod.mx.distributed, "all_sum", fake_all_sum)
 
     check_every = generate_mod.warmup_inference(
         model=object(),  # type: ignore[arg-type]
@@ -81,10 +96,13 @@ def fake_log_request_shape(
     assert check_every == 0
     assert task_params.instructions is None
     assert task_params.enable_thinking is False
-    assert task_params.temperature == 1.0
-    assert task_params.top_p == 0.95
-    assert task_params.top_k == 64
-    assert task_params.max_output_tokens == 1024
+    # Distributed warmup uses greedy sampling so every rank samples the same
+    # token from the shared logits (stochastic sampling can diverge).
+    assert task_params.temperature == 0.0
+    assert task_params.top_p == 1.0
+    assert task_params.top_k == 0
+    assert task_params.max_output_tokens == 1
+    assert captured["suppress_empty_gemma4_thought_channel"] is True
     first_message = task_params.input[0]
     assert first_message.content == "hello"
     assert captured["label"] == "warmup"
@@ -101,8 +119,14 @@ def test_warmup_inference_ignores_repeat_count_override_for_pipeline_groups(
 ) -> None:
     captured: dict[str, object] = {}
 
-    def fake_apply_chat_template(*, tokenizer: object, task_params: object, model_card: object):
-        del tokenizer, model_card
+    def fake_apply_chat_template(
+        *,
+        tokenizer: object,
+        task_params: object,
+        model_card: object,
+        suppress_empty_gemma4_thought_channel: bool = False,
+    ):
+        del tokenizer, model_card, suppress_empty_gemma4_thought_channel
         captured["task_params"] = task_params
         return "warmup prompt"
 
@@ -114,8 +138,8 @@ def fake_mlx_generate(**_kwargs: object):
             yield None
         return
 
-    def fake_all_gather(array: object, *, group: object):
-        del group
+    def fake_all_sum(array: object, *, group: object, stream: object = None):
+        del group, stream
         return array
 
     _clear_warmup_env(monkeypatch)
@@ -123,7 +147,7 @@ def fake_all_gather(array: object, *, group: object):
     monkeypatch.setattr(generate_mod, "apply_chat_template", fake_apply_chat_template)
     monkeypatch.setattr(generate_mod, "mx_barrier", fake_mx_barrier)
     monkeypatch.setattr(generate_mod, "mlx_generate", fake_mlx_generate)
-    monkeypatch.setattr(generate_mod.mx.distributed, "all_gather", fake_all_gather)
+    monkeypatch.setattr(generate_mod.mx.distributed, "all_sum", fake_all_sum)
 
     generate_mod.warmup_inference(
         model=object(),  # type: ignore[arg-type]
@@ -142,8 +166,14 @@ def test_warmup_inference_ignores_instruction_override_for_pipeline_groups(
 ) -> None:
     captured: dict[str, object] = {}
 
-    def fake_apply_chat_template(*, tokenizer: object, task_params: object, model_card: object):
-        del tokenizer, model_card
+    def fake_apply_chat_template(
+        *,
+        tokenizer: object,
+        task_params: object,
+        model_card: object,
+        suppress_empty_gemma4_thought_channel: bool = False,
+    ):
+        del tokenizer, model_card, suppress_empty_gemma4_thought_channel
         captured["task_params"] = task_params
         return "warmup prompt"
 
@@ -155,8 +185,8 @@ def fake_mlx_generate(**_kwargs: object):
             yield None
         return
 
-    def fake_all_gather(array: object, *, group: object):
-        del group
+    def fake_all_sum(array: object, *, group: object, stream: object = None):
+        del group, stream
         return array
 
     _clear_warmup_env(monkeypatch)
@@ -164,7 +194,7 @@ def fake_all_gather(array: object, *, group: object):
     monkeypatch.setattr(generate_mod, "apply_chat_template", fake_apply_chat_template)
     monkeypatch.setattr(generate_mod, "mx_barrier", fake_mx_barrier)
     monkeypatch.setattr(generate_mod, "mlx_generate", fake_mlx_generate)
-    monkeypatch.setattr(generate_mod.mx.distributed, "all_gather", fake_all_gather)
+    monkeypatch.setattr(generate_mod.mx.distributed, "all_sum", fake_all_sum)
 
     generate_mod.warmup_inference(
         model=object(),  # type: ignore[arg-type]
@@ -183,8 +213,14 @@ def test_warmup_inference_honors_repeat_and_instruction_overrides_for_single_nod
 ) -> None:
     captured: dict[str, object] = {}
 
-    def fake_apply_chat_template(*, tokenizer: object, task_params: object, model_card: object):
-        del tokenizer, model_card
+    def fake_apply_chat_template(
+        *,
+        tokenizer: object,
+        task_params: object,
+        model_card: object,
+        suppress_empty_gemma4_thought_channel: bool = False,
+    ):
+        del tokenizer, model_card, suppress_empty_gemma4_thought_channel
         captured["task_params"] = task_params
         return "warmup prompt"
 
@@ -196,8 +232,8 @@ def fake_mlx_generate(**_kwargs: object):
             yield None
         return
 
-    def fake_all_gather(array: object, *, group: object):
-        del group
+    def fake_all_sum(array: object, *, group: object, stream: object = None):
+        del group, stream
         return array
 
     _clear_warmup_env(monkeypatch)
@@ -206,7 +242,7 @@ def fake_all_gather(array: object, *, group: object):
     monkeypatch.setattr(generate_mod, "apply_chat_template", fake_apply_chat_template)
     monkeypatch.setattr(generate_mod, "mx_barrier", fake_mx_barrier)
     monkeypatch.setattr(generate_mod, "mlx_generate", fake_mlx_generate)
-    monkeypatch.setattr(generate_mod.mx.distributed, "all_gather", fake_all_gather)
+    monkeypatch.setattr(generate_mod.mx.distributed, "all_sum", fake_all_sum)
 
     generate_mod.warmup_inference(
         model=object(),  # type: ignore[arg-type]
@@ -224,33 +260,41 @@ def fake_all_gather(array: object, *, group: object):
     )
 
 
-def test_warmup_inference_stops_after_first_generated_token(
+def test_warmup_inference_requests_one_token_and_finishes_normally(
     monkeypatch: pytest.MonkeyPatch,
 ) -> None:
     generated_tokens = 0
+    captured: dict[str, object] = {}
 
-    def fake_apply_chat_template(*, tokenizer: object, task_params: object, model_card: object):
-        del tokenizer, task_params, model_card
+    def fake_apply_chat_template(
+        *,
+        tokenizer: object,
+        task_params: object,
+        model_card: object,
+        suppress_empty_gemma4_thought_channel: bool = False,
+    ):
+        del tokenizer, task_params, model_card, suppress_empty_gemma4_thought_channel
         return "warmup prompt"
 
     def fake_mx_barrier(_group: object) -> None:
         return None
 
-    def fake_mlx_generate(**_kwargs: object):
+    def fake_mlx_generate(**kwargs: object):
         nonlocal generated_tokens
-        for token in ("first", "second", "third"):
-            generated_tokens += 1
-            yield token
+        task = cast(TextGenerationTaskParams, kwargs["task"])
+        captured["max_output_tokens"] = task.max_output_tokens
+        generated_tokens += 1
+        yield "first"
 
-    def fake_all_gather(array: object, *, group: object):
-        del group
+    def fake_all_sum(array: object, *, group: object, stream: object = None):
+        del group, stream
         return array
 
     _clear_warmup_env(monkeypatch)
     monkeypatch.setattr(generate_mod, "apply_chat_template", fake_apply_chat_template)
     monkeypatch.setattr(generate_mod, "mx_barrier", fake_mx_barrier)
     monkeypatch.setattr(generate_mod, "mlx_generate", fake_mlx_generate)
-    monkeypatch.setattr(generate_mod.mx.distributed, "all_gather", fake_all_gather)
+    monkeypatch.setattr(generate_mod.mx.distributed, "all_sum", fake_all_sum)
 
     check_every = generate_mod.warmup_inference(
         model=object(),  # type: ignore[arg-type]
@@ -260,6 +304,7 @@ def fake_all_gather(array: object, *, group: object):
         model_card=None,
     )
 
+    assert captured["max_output_tokens"] == 1
     assert generated_tokens == 1
     assert check_every == 100
 
@@ -274,9 +319,7 @@ def test_warmup_helpers_prefer_blank_skulk_values_over_legacy_env(
     monkeypatch.setenv("EXO_DEBUG_WARMUP_INCLUDE_INSTRUCTIONS", "1")
 
     assert generate_mod._warmup_repeat_count() == 1  # pyright: ignore[reportPrivateUsage]
-    assert (
-        generate_mod._warmup_instructions(cast(object, _SingleNodeGroup())) is None
-    )  # pyright: ignore[reportPrivateUsage]
+    assert generate_mod._warmup_instructions(cast(object, _SingleNodeGroup())) is None  # pyright: ignore[reportPrivateUsage, reportArgumentType]
 
 
 def test_warmup_inference_enforces_minimum_cancel_check_interval_on_slow_start(
@@ -284,8 +327,14 @@ def test_warmup_inference_enforces_minimum_cancel_check_interval_on_slow_start(
 ) -> None:
     monotonic_values = iter([100.0, 112.0])
 
-    def fake_apply_chat_template(*, tokenizer: object, task_params: object, model_card: object):
-        del tokenizer, task_params, model_card
+    def fake_apply_chat_template(
+        *,
+        tokenizer: object,
+        task_params: object,
+        model_card: object,
+        suppress_empty_gemma4_thought_channel: bool = False,
+    ):
+        del tokenizer, task_params, model_card, suppress_empty_gemma4_thought_channel
         return "warmup prompt"
 
     def fake_mx_barrier(_group: object) -> None:
@@ -294,15 +343,15 @@ def fake_mx_barrier(_group: object) -> None:
     def fake_mlx_generate(**_kwargs: object):
         yield "first"
 
-    def fake_all_gather(array: object, *, group: object):
-        del group
+    def fake_all_sum(array: object, *, group: object, stream: object = None):
+        del group, stream
         return array
 
     _clear_warmup_env(monkeypatch)
     monkeypatch.setattr(generate_mod, "apply_chat_template", fake_apply_chat_template)
     monkeypatch.setattr(generate_mod, "mx_barrier", fake_mx_barrier)
     monkeypatch.setattr(generate_mod, "mlx_generate", fake_mlx_generate)
-    monkeypatch.setattr(generate_mod.mx.distributed, "all_gather", fake_all_gather)
+    monkeypatch.setattr(generate_mod.mx.distributed, "all_sum", fake_all_sum)
     monkeypatch.setattr(generate_mod.time, "monotonic", lambda: next(monotonic_values))
 
     check_every = generate_mod.warmup_inference(
diff --git a/src/exo/worker/tests/unittests/test_runner/test_event_ordering.py b/src/exo/worker/tests/unittests/test_runner/test_event_ordering.py
index f95d7c819..22f340887 100644
--- a/src/exo/worker/tests/unittests/test_runner/test_event_ordering.py
+++ b/src/exo/worker/tests/unittests/test_runner/test_event_ordering.py
@@ -120,11 +120,16 @@ def patch_out_mlx(monkeypatch: pytest.MonkeyPatch):
     )
     monkeypatch.setattr(mlx_batch_generator, "warmup_inference", make_nothin(1))
     monkeypatch.setattr(mlx_batch_generator, "_check_for_debug_prompts", nothin)
-    monkeypatch.setattr(mlx_batch_generator, "mx_any", make_nothin(False))
+
+    def fake_mx_any(bool_: bool, _group: object) -> bool:
+        return bool_
+
+    monkeypatch.setattr(mlx_batch_generator, "mx_any", fake_mx_any)
 
     def fake_all_gather(
-        tasks: list[TextGeneration], group: object
+        tasks: list[TextGeneration], group: object, label: str = "tasks"
     ) -> tuple[list[TextGeneration], list[TextGeneration]]:
+        del label
         return (tasks, [])
 
     monkeypatch.setattr(mlx_batch_generator, "mx_all_gather_tasks", fake_all_gather)
@@ -232,6 +237,77 @@ def size(self) -> int:
         return 2
 
 
+class EmptyCancelReceiver:
+    def collect(self) -> list[TaskId]:
+        return []
+
+
+def _make_sequential_generator() -> mlx_batch_generator.SequentialGenerator:
+    return mlx_batch_generator.SequentialGenerator(
+        model=object(),  # type: ignore[arg-type]
+        tokenizer=MockTokenizer(),  # type: ignore[arg-type]
+        group=MockGroup(),  # type: ignore[arg-type]
+        kv_prefix_cache=None,
+        tool_parser=None,
+        model_card=None,
+        model_id=MODEL_A_ID,
+        device_rank=0,
+        cancel_receiver=EmptyCancelReceiver(),  # type: ignore[arg-type]
+        event_sender=EventCollector(),  # type: ignore[arg-type]
+    )
+
+
+def test_empty_task_agreement_skips_task_id_collective(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    mx_any_calls: list[bool] = []
+
+    def fake_mx_any(bool_: bool, _group: object) -> bool:
+        mx_any_calls.append(bool_)
+        return False
+
+    def fail_all_gather(*_args: object, **_kwargs: object) -> None:
+        raise AssertionError("empty task agreement should not gather task IDs")
+
+    monkeypatch.setattr(mlx_batch_generator, "mx_any", fake_mx_any)
+    monkeypatch.setattr(mlx_batch_generator, "mx_all_gather_tasks", fail_all_gather)
+
+    _make_sequential_generator().agree_on_tasks()
+
+    assert mx_any_calls == [False]
+
+
+def test_empty_cancellation_agreement_skips_task_id_collective(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    mx_any_calls: list[bool] = []
+
+    def fake_mx_any(bool_: bool, _group: object) -> bool:
+        mx_any_calls.append(bool_)
+        return False
+
+    def fail_all_gather(*_args: object, **_kwargs: object) -> None:
+        raise AssertionError("empty cancellation agreement should not gather task IDs")
+
+    monkeypatch.setattr(mlx_batch_generator, "mx_any", fake_mx_any)
+    monkeypatch.setattr(mlx_batch_generator, "mx_all_gather_tasks", fail_all_gather)
+
+    _make_sequential_generator().agree_on_cancellations()
+
+    assert mx_any_calls == [False, False]
+
+
+def test_zero_warmup_cancel_interval_falls_back_to_safe_default(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    generator = _make_sequential_generator()
+    monkeypatch.setattr(mlx_batch_generator, "warmup_inference", make_nothin(0))
+
+    generator.warmup()
+
+    assert generator.check_for_cancel_every == 50
+
+
 def _run(tasks: Iterable[Task], send_after_ready: list[Task] | None = None):
     bound_instance = get_bound_mlx_ring_instance(
         instance_id=INSTANCE_1_ID,
@@ -385,7 +461,9 @@ def test_distributed_warmup_skip_env_is_ignored(
     patch_out_mlx: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch
 ) -> None:
     monkeypatch.setenv("SKULK_SKIP_LLM_WARMUP", "1")
-    monkeypatch.setattr(mlx_runner, "initialize_mlx", make_nothin(MockDistributedGroup()))
+    monkeypatch.setattr(
+        mlx_runner, "initialize_mlx", make_nothin(MockDistributedGroup())
+    )
 
     warmup_calls = 0
 
@@ -421,9 +499,13 @@ def fake_log_request_shape(
     ) -> None:
         logged.append((label, task_params, prompt, extra))
 
-    monkeypatch.setattr(mlx_batch_generator, "log_request_shape", fake_log_request_shape)
+    monkeypatch.setattr(
+        mlx_batch_generator, "log_request_shape", fake_log_request_shape
+    )
 
-    _run([INIT_TASK, LOAD_TASK, WARMUP_TASK, CHAT_TASK], send_after_ready=[SHUTDOWN_TASK])
+    _run(
+        [INIT_TASK, LOAD_TASK, WARMUP_TASK, CHAT_TASK], send_after_ready=[SHUTDOWN_TASK]
+    )
 
     assert logged == [
         (
diff --git a/src/exo/worker/tests/unittests/test_runner/test_turboquant_batch_guard.py b/src/exo/worker/tests/unittests/test_runner/test_turboquant_batch_guard.py
index a21cfa01f..cd470959a 100644
--- a/src/exo/worker/tests/unittests/test_runner/test_turboquant_batch_guard.py
+++ b/src/exo/worker/tests/unittests/test_runner/test_turboquant_batch_guard.py
@@ -4,8 +4,15 @@
 import pytest
 from mlx_lm.tokenizer_utils import TokenizerWrapper
 
+from exo.shared.models.model_cards import (
+    ModelCard,
+    ModelTask,
+    PromptRendererType,
+    RuntimeCapabilityCardConfig,
+)
 from exo.shared.types.common import ModelId
 from exo.shared.types.events import Event
+from exo.shared.types.memory import Memory
 from exo.shared.types.mlx import Model
 from exo.shared.types.tasks import TaskId
 from exo.utils.channels import mp_channel
@@ -29,7 +36,14 @@ class _FakeModel:
 
 @pytest.mark.parametrize(
     "kv_backend",
-    ["mlx_quantized", "turboquant", "turboquant_adaptive"],
+    [
+        "mlx_quantized",
+        "turboquant",
+        "turboquant_adaptive",
+        "optiq",
+        "rotorquant",
+        "rotorquant_adaptive",
+    ],
 )
 def test_builder_forces_sequential_for_quantized_kv_backends(
     monkeypatch: pytest.MonkeyPatch,
@@ -56,3 +70,40 @@ def test_builder_forces_sequential_for_quantized_kv_backends(
 
     assert isinstance(generator, SequentialGenerator)
     assert not isinstance(generator, BatchGenerator)
+
+
+def test_builder_forces_sequential_for_gemma4_runtime(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    _, cancel_recv = mp_channel[TaskId]()
+    event_send, _ = mp_channel[Event]()
+
+    monkeypatch.setattr(
+        "exo.worker.runner.llm_inference.runner.get_kv_cache_backend",
+        lambda: "default",
+    )
+
+    builder = Builder(
+        model_id=ModelId("local/custom-gemma-runtime"),
+        event_sender=event_send,
+        cancel_receiver=cancel_recv,
+        inference_model=cast(Model, cast(object, _FakeModel())),
+        tokenizer=cast(TokenizerWrapper, cast(object, _FakeTokenizer())),
+        group=cast(mx.distributed.Group | None, None),
+        model_card=ModelCard(
+            model_id=ModelId("local/custom-gemma-runtime"),
+            storage_size=Memory(),
+            n_layers=1,
+            hidden_size=1,
+            supports_tensor=False,
+            tasks=[ModelTask.TextGeneration],
+            runtime=RuntimeCapabilityCardConfig(
+                prompt_renderer=PromptRendererType.Gemma4
+            ),
+        ),
+    )
+
+    generator = builder.build()
+
+    assert isinstance(generator, SequentialGenerator)
+    assert not isinstance(generator, BatchGenerator)
diff --git a/website/docs/everything-different.md b/website/docs/everything-different.md
new file mode 100644
index 000000000..adab01a3a
--- /dev/null
+++ b/website/docs/everything-different.md
@@ -0,0 +1,95 @@
+---
+id: everything-different
+title: Everything Different About Skulk
+sidebar_position: 2
+---
+
+<!-- Copyright 2025 Foxlight Foundation -->
+
+This is the running list of where Skulk diverges from upstream [exo](https://github.com/exo-explore/exo).
+
+It is intentionally a **living document** — every meaningful change should land here when it ships, so anyone evaluating Skulk can see the surface area at a glance and the team has a single place to point at when describing what makes this fork distinct.
+
+If you are about to land a feature or fix that adds, removes, or materially changes a behavior relative to upstream, update this page in the same PR.
+
+## Inference and KV cache
+
+- **RotorQuant KV cache backend** — pure-MLX port of IsoQuant 3-bit (block-diagonal quaternion rotations + Lloyd-Max centroids) with **deferred prefill on Metal**, a contribution that does not exist in any upstream project (the llama.cpp fork ships it CUDA-only). GQA-native; no fallback for grouped-query models. See [KV cache backends](kv-cache-backends).
+- **TurboQuant native and adaptive backends** — randomized Hadamard rotation + Lloyd-Max centroids, with an adaptive variant that keeps edge attention layers in fp16 for accuracy.
+- **OptiQ KV cache integration** — wraps `mlx-optiq`'s rotated-space attention path so the rotation cost stays out of the per-token loop on supported (non-GQA) models.
+- **OptiQ mixed-precision weight quantization pipeline** — async wrapper around `mlx-optiq`'s sensitivity analysis and KL-divergence per-layer bit allocation, exposed as a model-store optimization job.
+- **KV prefix cache with snapshot/restore** — LRU-evicted prompt-prefix cache that snapshots SSM and rotating-window cache states so prefix matches are reusable across conversation turns even for hybrid Mamba/Transformer architectures.
+- **Pipeline-parallel prefill for short prompts** — pipelined models now route every prefill through the pipeline path, fixing prior warmup hangs on Gemma-class models.
+- **Force-sequential fallback** — quantized backends transparently fall back to a sequential generator when batch/history mode is incompatible with their cache layout.
+
+## Model capability system
+
+- **Two-layer capability model** — declarative `ModelCard` (with optional `reasoning`, `modalities`, `tooling`, and `runtime` sections) plus a normalized `ResolvedCapabilityProfile` derived from the card and conservative family defaults. This is the source of truth for prompt rendering, output parsing, tool-call handling, and the `/v1/models` `resolved_capabilities` field.
+- **Phase 2 thinking contract** — `enable_thinking`, `reasoning_effort`, and the dashboard thinking toggle are all driven by `supports_thinking_toggle`, so non-toggleable reasoning models behave correctly without leaking model-specific quirks into client code. See [model capabilities](model-capabilities).
+- **Output parser selection** — model cards declare `output_parser` (`generic`, `gemma4`, `gpt_oss`, `deepseek_v32`, etc.), so reasoning markers are normalized into structured `reasoning_content` per family.
+- **Model store metadata pipeline** — capability resolution feeds `/v1/models` so dashboards and clients can discover thinking, multimodal, and tool support without hardcoding model lists.
+
+## API surface
+
+- **Claude Messages API** — `/v1/messages` adapter, including streaming, tool use, image inputs, and capability-aware thinking controls.
+- **Ollama compatibility** — both `/api/chat` and `/api/generate`, with adapter-side reasoning normalization.
+- **OpenAI Responses API** — `/v1/responses` adapter alongside chat completions.
+- **Embeddings endpoint** for non-chat workloads.
+- **Model store endpoints** — search, add, download, capability resolution, optimization jobs, and registry management, all exposed under stable URLs and documented in the OpenAPI spec.
+- **Cluster-wide config endpoints** — `GET`/`POST` config that gossipsubs to every node and writes back to `skulk.yaml`. Stable KV cache backend selection goes through this same endpoint, so the dashboard Settings panel can switch supported backends cluster-wide without env vars. Experimental `rotorquant` values remain env-gated and fall back to `default` unless explicitly enabled.
+- **Tracing, downloads, instance previews, and placement endpoints** — distributed-system observability and pre-launch placement inspection that upstream does not expose.
+
+## Dashboard
+
+- **React dashboard (default)** — replaces upstream's Svelte UI with a typed React + styled-components app that ships with the binary. The legacy Svelte dashboard is kept only as a fallback in the repo.
+- **Cluster topology view** with live device icons, GPU stats, network mesh visualization, and connection status banners.
+- **Placement preview / placement manager** for inspecting and choosing valid placements before launching.
+- **Model store browser** with HuggingFace search, family sidebar, model filters, capability badges, recent models, and per-model launch controls.
+- **Reasoning-aware chat UI** that splits inline `<think>` and Gemma `<|channel>` markers into a dedicated thinking panel and merges them with `reasoning_content` deltas from the API.
+- **Image attachments and multimodal chat affordances** for vision models.
+- **Cluster-wide settings panel** that writes to `skulk.yaml` and syncs across nodes via gossipsub.
+- **Light and dark themes** with first-class theme tokens.
+
+## Centralized logging and observability
+
+- **Structured JSON stdout** when `logging.enabled` is set, configurable from the dashboard Settings panel and synced cluster-wide.
+- **Vector + VictoriaLogs + Grafana stack** — local Vector log shipper on each node, central VictoriaLogs storage, ready-made Grafana dashboards. Stack definition lives in `deployment/logging/`.
+- **Distributed tracing** opt-in via `EXO_TRACING_ENABLED`.
+
+## Model store
+
+- **Centralized model store host** — one node downloads, the rest of the cluster stages over the LAN.
+- **Persistent registry** with capability resolution and download tracking.
+- **Custom model card support** — add your own model with `POST /models/add`.
+- **Image and embedding model cards** behind feature flags.
+- **Optimization job pipeline** for mlx-optiq mixed-precision weight quantization.
+
+## Cluster operation
+
+- **Cluster-wide settings sync** for KV cache backend, logging, model store host, HF token, and other inference toggles.
+- **Bootstrap peer config from `skulk.yaml`, env, or CLI** for fixed-topology clusters.
+- **Election (bully algorithm) + master/worker split** for indexing events and broadcasting state.
+- **`SKULK_*` environment variables** alongside the legacy `EXO_*` set, so new options can land without colliding with upstream.
+- **`skulk.yaml`** as the canonical config file, with `exo.yaml` kept for backwards compatibility.
+
+## Build, type system, and dev workflow
+
+- **Strict basedpyright** type checking — zero-error policy for new code.
+- **Ruff** linting and **`nix fmt`** formatting in CI.
+- **Nix flake** for reproducible toolchain setup.
+- **Docusaurus docs site** with auto-generated OpenAPI per-endpoint pages and TypeDoc HTML reference for the dashboard, both built from source.
+- **Pre-commit checklist** documented in `CLAUDE.md` and enforced in CI.
+
+## Hardware and platform
+
+- **Apple Silicon as the primary target**, including RDMA over Thunderbolt 5 on supported hardware and matched macOS versions.
+- **Linux supported** (CPU-oriented in this fork; GPU work happens on Apple Silicon).
+
+## How to update this page
+
+When you ship something that distinguishes Skulk from upstream exo, add it here in the same PR. A few ground rules:
+
+- Group your bullet under the right category. If none fit, add a new category — but only when there's clearly a new area of divergence.
+- Lead with what the change *is*, then a sentence on *why* it matters compared to upstream.
+- Link to the relevant guide, code path, or issue when one exists.
+- Remove or rewrite bullets when they become inaccurate. This page is canonical, so stale bullets are worse than missing ones.
diff --git a/website/docs/intro.md b/website/docs/intro.md
index 479bf03a6..338ce0341 100644
--- a/website/docs/intro.md
+++ b/website/docs/intro.md
@@ -19,6 +19,7 @@ Skulk is a distributed inference platform, so these docs try to serve both of th
 If you are getting oriented, start with these pages:
 
 - [README](https://github.com/Foxlight-Foundation/Skulk/blob/main/README.md) for installation, first run, and quick-start paths
+- [Everything Different About Skulk](everything-different) for a single-page view of how this fork diverges from upstream exo
 - [API guide](api-guide) for the happy path: place a model, then call the API
 - [Model store guide](model-store) for shared model storage and download workflows
 - [KV cache backends](kv-cache-backends) for backend and runtime tuning
diff --git a/website/docs/kv-cache-backends.md b/website/docs/kv-cache-backends.md
index a7ea147ba..1636711d9 100644
--- a/website/docs/kv-cache-backends.md
+++ b/website/docs/kv-cache-backends.md
@@ -14,12 +14,28 @@ Skulk includes several opt-in KV cache backends for MLX text generation. These b
 - `mlx_quantized`: MLX LM built-in `QuantizedKVCache`
 - `turboquant`: correctness-first TurboQuant-inspired KV cache for standard `KVCache` layers
 - `turboquant_adaptive`: keeps outer KV layers in FP16 and applies TurboQuant to middle KV layers
-- `optiq`: **[NEW]** rotation-based KV cache via [mlx-optiq](https://mlx-optiq.pages.dev/) — uses randomized orthogonal rotations with Lloyd-Max quantization and rotated-space attention for superior long-context quality
+- `optiq`: rotation-based KV cache via [mlx-optiq](https://mlx-optiq.pages.dev/) — uses randomized orthogonal rotations with Lloyd-Max quantization and rotated-space attention. Falls back to default for GQA models.
+- `rotorquant`: **experimental, gated** pure-MLX IsoQuant-style storage/dequant cache. This is not the fused RotorQuant+QJL implementation from the RotorQuant paper.
+- `rotorquant_adaptive`: **experimental, gated** as above with FP16 protection on the first/last N attention layers.
 
 If `SKULK_KV_CACHE_BACKEND` is unset, or is set to `default`, Skulk behaves as before.
 
 ## Recommended Settings
 
+### Default Baseline (recommended while debugging)
+
+```bash
+SKULK_KV_CACHE_BACKEND=default \
+SKULK_MLX_HANG_DEBUG=1 \
+SKULK_MLX_HANG_DEBUG_INTERVAL_SECONDS=5 \
+uv run skulk -vv
+```
+
+Use the default backend first when validating distributed pipeline behavior. It
+removes KV-cache compression from the failure surface so hangs can be attributed
+to pipeline scheduling, task agreement, or the model path rather than an
+experimental cache implementation.
+
 ### mlx-optiq (best quality)
 
 ```bash
@@ -43,6 +59,23 @@ uv run skulk
 
 This mode keeps the first and last 4 KV layers in normal FP16-style cache and applies TurboQuant only to the middle KV layers. Proven stable across most models.
 
+### RotorQuant / IsoQuant (experimental only)
+
+The `rotorquant` names are intentionally gated behind an explicit opt-in:
+
+```bash
+SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT=1 \
+SKULK_KV_CACHE_BACKEND=rotorquant_adaptive \
+SKULK_ROTORQUANT_FP16_LAYERS=4 \
+uv run skulk
+```
+
+This backend is a pure-MLX IsoQuant-style cache that compresses storage and
+then returns fully dequantized fp16 K/V to normal MLX attention. It does not
+implement RotorQuant's fused Metal/CUDA attention path or QJL residual
+correction, so it should not be used as the default distributed inference
+baseline.
+
 ## Available Environment Variables
 
 | Variable | Backends | Default | Description |
@@ -54,6 +87,9 @@ This mode keeps the first and last 4 KV layers in normal FP16-style cache and ap
 | `SKULK_TQ_K_BITS` | `turboquant`, `turboquant_adaptive` | `3` | Key quantization bits |
 | `SKULK_TQ_V_BITS` | `turboquant`, `turboquant_adaptive` | `4` | Value quantization bits |
 | `SKULK_TQ_FP16_LAYERS` | `turboquant_adaptive` | `4` | Edge layers kept in FP16 |
+| `SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT` | `rotorquant`, `rotorquant_adaptive` | `0` | Required opt-in for experimental RotorQuant/IsoQuant backends |
+| `SKULK_ROTORQUANT_FP16_LAYERS` | `rotorquant_adaptive` | `4` | Edge layers kept in FP16 |
+| `SKULK_ROTORQUANT_DEFER_PREFILL` | `rotorquant`, `rotorquant_adaptive` | `1` | Set to `0` to disable deferred prefill while debugging |
 
 ## Invocation Examples
 
@@ -86,14 +122,16 @@ SKULK_KV_CACHE_BACKEND=turboquant_adaptive SKULK_TQ_K_BITS=3 SKULK_TQ_V_BITS=4 S
 | Backend | Memory | Quality | Speed | Notes |
 |---------|--------|---------|-------|-------|
 | `default` | Highest | Baseline | Fastest | No quantization |
-| `optiq` | Low | Best quantized | Near-baseline | Rotation-based, best long-context |
+| `optiq` | Low | Good | Near-baseline | Rotation-based, no GQA support |
 | `turboquant_adaptive` | Low | Good | Moderate | Proven stable, Hadamard-based |
-| `turboquant` | Lowest | Variable | Moderate | Most aggressive compression |
+| `turboquant` | Low | Variable | Moderate | All layers, Hadamard-based |
 | `mlx_quantized` | Low | Good | Moderate | MLX built-in quantization |
+| `rotorquant_adaptive` | Low | Experimental | Experimental | Gated pure-MLX IsoQuant storage/dequant cache |
+| `rotorquant` | Lowest | Experimental | Experimental | Gated pure-MLX IsoQuant storage/dequant cache |
 
 ## Supported Cache Layouts
 
-All quantized backends (optiq, turboquant, mlx_quantized) compress only standard `KVCache` entries and preserve these cache types unchanged:
+All quantized backends (optiq, turboquant, mlx_quantized, and the gated rotorquant variants) compress only standard `KVCache` entries and preserve these cache types unchanged:
 
 - `ArraysCache`
 - `RotatingKVCache`
@@ -107,8 +145,10 @@ Mixed cache layouts are supported:
 ## Current Limitations
 
 - All quantized KV cache backends force sequential generation (no batch/history mode)
+- Gemma 4 text generation also forces sequential generation for now because distributed BatchGenerator mode can produce degenerate repetition with its sliding-window cache layout
 - The optiq backend requires `mlx-optiq` to be installed (`pip install mlx-optiq`)
 - The optiq backend's `patch_attention()` monkey-patches MLX's SDPA — avoid switching between optiq and other backends within the same process lifetime without a restart
+- The rotorquant backends are disabled unless `SKULK_ENABLE_EXPERIMENTAL_ROTORQUANT=1` is set. If selected without the gate, Skulk falls back to `default`.
 
 ## About mlx-optiq
 
diff --git a/website/sidebars.ts b/website/sidebars.ts
index c809181b8..0f8edb67f 100644
--- a/website/sidebars.ts
+++ b/website/sidebars.ts
@@ -15,6 +15,11 @@ const sidebars: SidebarsConfig = {
       id: "intro",
       label: "Introduction",
     },
+    {
+      type: "doc",
+      id: "everything-different",
+      label: "Everything Different About Skulk",
+    },
     {
       type: "category",
       label: "Guides",