Read printed text for labels with no barcode - #6
Merged
Conversation
Plenty of what a technician has to enter is not encoded at all: a serial on a chassis label, a Wi-Fi password on a router, a product key on a card with no barcode beside it. That is the same job the rest of the app does, so the scanner now reads those too. OCR was on the explicitly-out-of-scope list, which is why the shape matters more than the feature. A decode is exact; a transcription is a reading, and a wrong character in a product key is worse than no product key. So text capture is a separate mode that never fires on its own: frame the label, capture, and the phone shows the lines it read. Tapping one is the send, and the line on screen is byte-for-byte what goes over the wire — no normalisation step, no edit field. Measured on rendered keys and serials, lines the engine read correctly scored 77-96% confidence and every misread line scored 56 or less. That is a real enough signal to show: below 80 a candidate turns amber and gains the word "check", and when every line comes back doubtful the panel says the read failed rather than offering noise under "tap the value to send". It flags, it never filters — a long random key is both the string the engine is least sure of and the one the technician most needs. The preview letterboxes in text mode instead of cropping. Under object-fit: cover on a portrait phone, both ends of a product key sit outside the visible frame while capture reads all of it; letterboxing is the only version where what is on screen is what gets read. Engine choice follows the existing native-first shape: TextDetector where it exists (ML Kit on Android), a WASM Tesseract elsewhere, loaded on demand so a phone that only scans barcodes never pays for it. Its assets are staged from node_modules into public/ocr at build time and served from our own origin, never the CDN the library defaults to. workerBlobURL is off so the worker loads from its path rather than a blob: URL, which means no CSP relaxation was needed. Recognition runs entirely on the device — the captured frame is read once and dropped, and no image leaves the phone. Values go over the wire as OCR_TEXT so the receiver can caption a transcription as one rather than passing it off as a decode. Also fixes a latent race the mode switch would have exposed: a detect() in flight when the decode loop stops could resolve afterwards, sending one last barcode or leaving two loops running. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ooGSXdKf1ZKVKSiTycKFs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plenty of what a technician has to enter is not encoded at all: a serial on a
chassis label, a Wi-Fi password on a router, a product key on a card with no
barcode beside it. That is the same job the rest of the app does, so the scanner
now reads those too.
OCR was on the explicitly-out-of-scope list, which is why the shape matters more
than the feature. A decode is exact; a transcription is a reading, and a wrong
character in a product key is worse than no product key. So text capture is a
separate mode that never fires on its own: frame the label, capture, and the
phone shows the lines it read. Tapping one is the send, and the line on screen is
byte-for-byte what goes over the wire — no normalisation step, no edit field.
Measured on rendered keys and serials, lines the engine read correctly scored
77-96% confidence and every misread line scored 56 or less. That is a real enough
signal to show: below 80 a candidate turns amber and gains the word "check", and
when every line comes back doubtful the panel says the read failed rather than
offering noise under "tap the value to send". It flags, it never filters — a long
random key is both the string the engine is least sure of and the one the
technician most needs.
The preview letterboxes in text mode instead of cropping. Under object-fit: cover
on a portrait phone, both ends of a product key sit outside the visible frame
while capture reads all of it; letterboxing is the only version where what is on
screen is what gets read.
Engine choice follows the existing native-first shape: TextDetector where it
exists (ML Kit on Android), a WASM Tesseract elsewhere, loaded on demand so a
phone that only scans barcodes never pays for it. Its assets are staged from
node_modules into public/ocr at build time and served from our own origin, never
the CDN the library defaults to. workerBlobURL is off so the worker loads from
its path rather than a blob: URL, which means no CSP relaxation was needed.
Recognition runs entirely on the device — the captured frame is read once and
dropped, and no image leaves the phone. Values go over the wire as OCR_TEXT so
the receiver can caption a transcription as one rather than passing it off as a
decode.
Also fixes a latent race the mode switch would have exposed: a detect() in flight
when the decode loop stops could resolve afterwards, sending one last barcode or
leaving two loops running.
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_012ooGSXdKf1ZKVKSiTycKFs