Modern German handwriting recognition, line-level, small enough to run in a browser tab.
Every milestone below had fixed exit criteria set before the work started;
DECISIONS.md records the choices those criteria left open.
Status: M0–M4 and M6 are complete. The line-level release is published.
- Demo: https://de-htr-web.vercel.app
- Model: https://huggingface.co/naeyn/de-htr-web
- Model card:
MODEL_CARD.md
| Milestone | State | Evidence |
|---|---|---|
| M0 — data and licenses | done | data/SOURCES.md, data/FONTS.md, data/split_manifest.json |
| M1 — eval harness and baselines | done | src/dehtr/evaluate.py, results/baseline_*.json |
| M2 — trained CTC model | done | results/m2/summary.json, results/m2/ctc_ratio*.json, checkpoints/m2/ctc_selected_ratio0.5_seed42_inference.pt |
| M3 — ONNX export and parity | done | models/dehtr_ctc_int8.onnx, results/m3/summary.json |
| M4 — browser demo | done | web/, results/m4/browser.json, live demo |
| M6 — release | done | Hugging Face model, production demo, results/m6/huggingface.json |
M5 full-page input is optional and intentionally skipped; D12 records why the honest single-line release is the shipped scope.
The browser defaults to de-htr-web v2, which scores 8.01% CER versus 8.61% for
v1 on the sealed split. Use the model picker to select v1 for comparison.
Before deploying, fetch https://huggingface.co/naeyn/de-htr-web-v2/resolve/main/model.onnx
to models/dehtr_v2_int8.onnx; the web build rejects a missing v2 artifact.
Numbers come from results/*.json, all written by the committed eval script; nothing
here is transcribed by hand.
| Engine | CER | WER | Size | Median latency/line |
|---|---|---|---|---|
Tesseract 5.3.4 deu (conventional floor) |
43.4% | 94.5% | 8.6 MB | 162 ms |
microsoft/trocr-base-handwritten (English, zero-shot) |
26.5% | 85.6% | 2.67 GB | 2130 ms |
Qwen2.5-VL-7B-Instruct-4bit (general VLM, zero-shot) |
9.4% | 23.6% | 5.64 GB | 2909 ms |
Latency is not measured on equal footing: Tesseract runs as a fresh subprocess per line,
so its 162 ms includes process spawn and model reload, while the neural baselines amortize
model load across the run (each results/baseline_*.json records its own method). The
Tesseract figure is therefore an upper bound, and the comparison flatters this project.
The VLM wins on accuracy among the M1 baselines — the expected outcome, and one we committed to reporting before running it. That comparison stays honest: the release claim is the Pareto position (small, offline, in-browser), never an accuracy win over a multi-GB VLM.
Synthetic-ratio selection used validation CER only, never test CER. Ratio 0.5
won (best val CER 6.92%). Final numbers are three seeds at that ratio; mean ± std is
sample standard deviation (ddof=1) over seeds 42/43/44, read from
final_seeds.test_cer_std_sample in results/m2/summary.json — which also carries the
population figure (0.117) so the two are not confused.
| Run | Val CER | Test CER | Test WER | Notes |
|---|---|---|---|---|
| Ablation ratio 0 | 8.22% | 9.54% | 36.88% | seed 42 |
| Ablation ratio 0.5 | 6.92% | 8.21% | 32.33% | seed 42; selected |
| Ablation ratio 0.8 | 7.18% | 8.36% | 33.93% | seed 42 |
| Final ratio 0.5 (seeds 42/43/44) | — | 8.15% ± 0.14% | 32.26% | mean ± sample std |
| TrOCR-small handwritten (trained reference) | 11.10% | 11.51% | 34.17% | encoder–decoder accuracy reference, not for release |
In this experiment the CTC 3-seed mean beats Tesseract by 35.26 CER points and the
trained TrOCR-small reference by 3.36 CER points. The release model has 9.73M
parameters; its quantized ONNX artifact is 9.84 MB. Inference checkpoint:
checkpoints/m2/ctc_selected_ratio0.5_seed42_inference.pt (38.98 MB).
Two caveats on those margins, both traceable to results/m2/summary.json
(convergence.runs):
- The TrOCR-small reference was not trained to convergence. Its best validation CER landed on its final epoch (12 of 12), so it was still improving when the fixed cap hit. Its 11.51% is an upper bound, and the 3.36-point gap is correspondingly an upper bound on the CTC advantage. It is an accuracy reference under a fixed budget, not a claim about encoder-decoder architectures.
- The selected CTC seed also hit its cap (best epoch 40 of 40), so the released model's own 8.21% is likewise not a converged figure. Seeds 43 and 44 peaked at 38 and 36, so the 40-epoch cap is otherwise reasonable. Both numbers being floors means the comparison is not systematically biased in one direction, but neither is squeezed dry.
Against the Qwen2.5-VL-7B baseline the CTC 3-seed mean (8.15%) is nominally ahead of 9.44%, but the 95% bootstrap intervals overlap substantially (CTC seed 42: 7.40–9.10; Qwen: 8.33–10.65) and no paired significance test was run. Treat those two as statistically indistinguishable on this test set; the release argument is size, offline execution, and latency, not accuracy.
Export parity uses a fresh PyTorch CPU evaluation as its same-backend control; the
M2 CUDA CER remains reported separately. Source: results/m3/summary.json.
The same seed-42 checkpoint scores 8.21% on CUDA (M2, m2_cuda_test_cer) and
8.5035% on PyTorch CPU — a 0.29-point backend difference from the CUDA and CPU GRU
kernels, not from the export. So two different numbers are correct depending on what is
being described: the M2 training result and its 8.15% three-seed mean are CUDA figures,
while the artifact this project actually ships measures 8.5681% (int8, CPU/WASM), the
number the demo page advertises. The margins above the Tesseract floor and the overlap
with Qwen hold under either figure.
| Runtime / artifact | Test CER | Delta | Size | Median latency/line |
|---|---|---|---|---|
| PyTorch CPU checkpoint | 8.5035% | control | 38.98 MB | ~57 ms |
| ONNX fp32, Python ORT | 8.5035% | 0.0000 points vs PyTorch | 38.93 MB | ~49 ms |
| ONNX int8, Python ORT | 8.5681% | +0.0646 points vs fp32 | 9.84 MB | ~72 ms |
| ONNX int8, Node ORT | 8.5681% | identical to Python | 9.84 MB | ~70 ms |
The int8 artifact stores large Conv, GRU, and Linear weights as symmetric per-channel
int8 with ONNX DequantizeLinear nodes. Python and Node produced identical predictions
for all 601 test lines. Release artifact: models/dehtr_ctc_int8.onnx.
The committed verifier runs the first 50 frozen test lines through the actual static
page in headless Chrome. Source: results/m4/browser.json.
| Runtime | Lines | CER | Delta vs Node on same lines | Cold load | Median inference |
|---|---|---|---|---|---|
| Browser WASM, Apple M1 / Chrome 151 | 50 | 5.7937% | +0.0397 points | 243 ms | 253 ms |
That 5.7937% is not comparable to the 8.5681% above: it covers only the first 50 test lines, an easier subset on which Node scores 5.7540%. The runtime comparison is the delta between those two, not the gap to the full-set figure. The artifact's accuracy is 8.5681% on all 601 lines.
The browser made no HTTP request after readiness, had no console/page errors, and met
the <1 s CPU/WASM gate. Canvas and Pillow resizing differ on one prediction; D12
defines the binding parity threshold as ≤0.2 absolute CER points. The public page is
de-htr-web.vercel.app.
DECISIONS.md decisions the exit criteria left open, with rationale
MODEL_CARD.md release scope, metrics, training data, limitations
LICENSE Apache-2.0 — this project's code and model artifacts
NOTICE copyright and redistributed-component notices
THIRD_PARTY_NOTICES.md third-party licenses in the browser distribution
data/SOURCES.md license register: datasets, deps, tools, models
data/FONTS.md 12 SIL OFL 1.1 fonts, each verified for ÄÖÜäöüß
data/fonts/ font files + their OFL texts (synthetic data for M2)
data/german_wikipedia_lines.* CC BY-SA 4.0 rendering text + its license notice
data/split_manifest.json FROZEN page-disjoint split — never regenerate it
data/raw/ gitignored corpora; rebuild with fetch_data.py
src/dehtr/data.py CSV + XML line loaders, split generator (freeze-protected)
src/dehtr/dataset.py training datasets, synthetic mixing, collation
src/dehtr/model.py CTC ResNet-18 + BiGRU architecture
src/dehtr/evaluate.py CER/WER + bootstrap 95% CI — the only source of numbers
src/dehtr/onnx_runtime.py ONNX export, int8 QDQ, Python evaluation, M3 gate
scripts/fetch_data.py reproducible Tier-1 download + md5 + license re-check
scripts/baseline_*.py the three M1 zero-shot baselines
scripts/train_ctc.py M2 CTC training
scripts/train_seq2seq_reference.py M2 TrOCR-small accuracy reference
scripts/run_m2.py M2 ablation → select → 3-seed → summarize
scripts/summarize_m2.py M2 rollup → results/m2/summary.json
scripts/run_m3.py fp32 → int8 → Python/Node parity orchestration
scripts/evaluate_onnx_node.mjs ONNX Runtime Node evaluator
scripts/build_web.mjs deterministic static release assembly
scripts/verify_web.mjs 50-line browser parity, latency, network gate
scripts/publish_hf.py hash-checked Hugging Face publication
models/dehtr_ctc_int8.onnx the v1 browser artifact
models/dehtr_v2_int8.onnx the v2 browser artifact (default)
checkpoints/m2/ selected CTC inference checkpoint
web/ static single-line browser UI and preprocessing
vercel.json static production headers and build configuration
results/ M1 baselines plus M2/M3 metrics and predictions
results/m2/ CTC ablations, final seeds, seq2seq reference, summary
results/m3/ fp32/int8 Python and Node parity evidence
results/m4/ browser metrics, predictions, and screenshot
results/m6/ Hugging Face publication receipt
The repo was prepared on macOS; M2 training ran on a rented NVIDIA RTX A5000 (an RTX
5070 Ti under WSL is a working alternative). mlx-vlm is marked
sys_platform == 'darwin', so uv sync works on Linux. Install Torch from the CUDA
12.8+ index for Blackwell/sm_120 when using the 5070 Ti, and keep WSL data on ext4,
not /mnt/c/.
uv sync # deps (skips the macOS-only VLM baseline)
npm ci # locked Node + browser dependencies
uv run python scripts/fetch_data.py # ~245 MB, md5-verified, re-checks the license
uv run python -m dehtr.data --verify # image+GT pairs parse from both XML and CSV
uv run python -m dehtr.evaluate --self-check # metric correctness on 5 oracle cases
npm run m3 # regenerate ONNX and all M3 parity evidence
npm run build:web # assemble the static site in dist/
npm run verify:web # browser UI + 50-line parity + network gateThese checks confirm a clean checkout before training or re-evaluation. The split
manifest is already committed and is frozen: --make-split refuses to overwrite it
without --force, and regenerating it invalidates every reported number. M2 results
live under results/m2/; M3 evidence under results/m3/;
browser evidence under results/m4/.
- 5,843 real line images across 385 pages. The collection ships line-level ground truth
for all 775 pages, but line crops for only 385 of them, so the split covers those
385; the loader records the remaining 5,976 ground-truth lines as
xml_lines_skipped_missing_imageindata/split_manifest.json. Those lines are recoverable by cropping the page images with the XML coordinates — unused here. Split 309/38/38 pages → 4,675/567/601 lines. - The split is page-disjoint, and "writer" means "page". The corpus provides no
writer identifiers, so the manifest uses
page_idas the writer proxy (writer_id_field). Each page carries one hand, but nothing in the corpus rules out one person having written several pages, so writer-level disjointness is a documented assumption rather than a verified property. The only identifying metadata is(gender, yearofbirth), which is far too coarse to check it: the 385 split pages fall into 90 such profiles, and every profile present in test also occurs in train. - Both Tier-1 records are CC BY 4.0, confirmed on the Zenodo records.
- The 77-page full-page set shares no page identifier with the line set, so it stays a clean held-out set under the same proxy caveat.
- 5,843 real lines is a small training set; synthetic rendered data compensates (fonts are already vetted and committed) — and synthetic data never appears in validation or test.