From ecec836fe34f86f9c6bf7573809ba488fc04e777 Mon Sep 17 00:00:00 2001 From: Lukas Scheucher Date: Sat, 16 May 2026 21:13:14 +0200 Subject: [PATCH] Polish the web demo: token visualization + portfolio explanations. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three threads bundled together since they all touch the same two files. * Visual redesign of index.html. Larger heading with accent on "GPT", card-based prompt+controls panel with elevated shadow, primary-styled generate button with hover/press states, output area as its own card. Light/dark mode polished — warmer backgrounds, better borders, brighter accent in dark mode. * Token visualization. Six soft pastel CSS variables (--tok-c0..c5) applied via .tok-N classes. Each token in both the live prompt preview (under the input) and the generated output gets a colored background that cycles, so token boundaries are visible at a glance. User-typed tokens are additionally underlined in the accent color so the boundary between human prompt and model continuation is obvious. main.js gains a renderPromptPreview() helper wired to the prompt input's "input" event for live updates. * "How it works" portfolio section below the demo. Sections cover the model architecture, what tokens are, browser-side inference via ONNX Runtime Web, lookahead sampling, the plateau LR schedule, and the tech stack. Links to source on GitHub and Karpathy's original repo. Also moved the initial prompt-preview render to fire before the ~10s ONNX model download, so users see colored tokens immediately rather than staring at "(empty)" while the model loads. Co-Authored-By: Claude Opus 4.7 (1M context) --- web/index.html | 393 +++++++++++++++++++++++++++++++++++++++++++------ web/main.js | 62 ++++++-- 2 files changed, 398 insertions(+), 57 deletions(-) diff --git a/web/index.html b/web/index.html index d7ce3b8..c6eb1a1 100644 --- a/web/index.html +++ b/web/index.html @@ -3,60 +3,367 @@ - nanoGPT — char-level demo + lukasGPT — char-level demo -

lukasGPT — char-level demo

-

Transformer inspired by Karpathy's tiny transformer, trained on Project Gutenberg books and running fully in your browser via ONNX Runtime Web.

-

- - - -
- - - - - - - +
+

lukasGPT

+

A tiny character-level transformer, trained on Project Gutenberg books and running fully in your browser via ONNX Runtime Web.

+

+
+ +
+ + +
+ Tokens +
+
+
+ + + + + + + +
+
initializing…

+
+

How it works

+

This page runs a small generative language model entirely in your browser. No server, no API call, no data leaving your device.

+ +

The model

+

A character-level transformer with ~12M parameters: 6 layers, 6 attention heads, a 384-dimensional residual stream, and a 1024-character context window. Trained from scratch on a subset of Project Gutenberg books published before 1919 — that's why the prose tends to sound vaguely Victorian.

+ +

Tokens

+

Each colored box above is one token — the model's atomic unit of input and output. Tokens are character-level here, so every box is a single letter, digit, punctuation mark, or piece of whitespace. The underlined ones are what you typed; the rest were sampled one at a time by the model. Adjacent tokens get cycling background colors so the boundaries are visible.

+ +

Browser-side inference

+

The trained PyTorch model is exported to ONNX and loaded by ONNX Runtime Web. The browser runs the actual matrix multiplications via WebGPU (when available) or WASM SIMD. Sampling — temperature, top-k, optional lookahead — is implemented in plain JavaScript and drives the model in a tight autoregressive loop, one token per forward pass.

+ +

Lookahead sampling

+

When the lookahead checkbox is on, instead of greedily sampling one token at a time, the sampler expands a depth-N tree with branching factor K, ranks all KN candidate paths by their joint probability, and picks one weighted by temperature. Trades wall-clock for more coherent output — useful for a small char-level model where greedy sampling drifts quickly.

+ +

Smart learning rate

+

Training uses a linear warmup over the first 100 iterations followed by ReduceLROnPlateau — the learning rate only drops when validation loss stops improving for several eval intervals, rather than following a predetermined decay schedule. The loss curve and lr staircase are both logged to TensorBoard during training.

+ +

Tech stack

+
    +
  • Training in PyTorch with mixed char/BPE tokenization, plateau LR schedule, and TensorBoard for monitoring
    • +
  • Export via torch.onnx.export with a dynamic time axis
    • +
  • Runtime: ONNX Runtime Web (WebGPU + multi-threaded WASM SIMD, with a service worker injecting COOP/COEP headers so SharedArrayBuffer works on GitHub Pages)
    • +
  • Hosting: GitHub Pages — static, free, no backend
    • +
  • Streamlit dashboard for inspecting checkpoint embeddings and token vocabularies during training
    • +
+ +

+ Source on GitHub · + Built on top of Karpathy's ng-video-lecture +

+
+