What are you using Ocrs for?

[robertknight]

If you are using Ocrs or tried to use it but chose an alternative, I would love to hear what you are using it for. Please leave a brief comment below. In particular I'm interested in what kinds of images (screenshots? photos? scans of physical media?) and languages you are trying to analyze. This will help to inform which images should be sourced or generated for training data.

[rjmac]

I've been using it for identifying Magic: The Gathering cards to look up current official rules text in my game utility app. Most MTG card scanners, as far as I can tell, try to match on the card art, but "OCR the text and then do a fuzzy match against the all the printed texts ever released" works shockingly well for the sorts of cards you would actually want to look up this way. I used to use Tesseract for this but the "prepare an image for optimum OCRing" step was always really finicky. OCRS gives just as good if not better scan results with no such prep step and no dynamic library hassles.

The image type is "reasonably well aligned almost-full-frame photos taken with a phone" then, and right now I only really care about English printings (though other languages - particularly Japanese - are becoming more common to see in north american games). Some cards are a little weird in terms of layout which is a challenge for OCRS (and wasn't any less of a challenge with Tesseract) but even these get identfied pretty reliably this way.

[laundmo]

I was looking at ocrs (and its issues) for a project trying to OCR item names in a game. My main concern right now is performance on repeated calls for many small images, as i can pretty reliably find and extract just the item labels but would need to OCR up to 20 every 100ms. I would also only use engine.get_text since i'm already only passing it the areas with text.

I'm considering ocrs because of its relatively clear instructions for training custom models, which i hope might allow me to use smaller, purpose-trained, models if i end up wanting to down the line.

Edit: I decided to test it, and it seems that specifically the detect_words model doesn't get much faster on smaller images. Additionally, foregoing the position information and just using engine.get_text has basically no speed benefit - which makes sense once you look at its implementation and see it just calls detect_words, find_text_lines, recognize_text

image size	get_text	detect_words	recognize_text
1920x1080	217ms	55ms	158ms
1000x60	121ms	53ms	67ms
250x60	74ms	50ms	21ms

I did not include times for find_text_lines because its always a few μs

[robertknight]

My main concern right now is performance on repeated calls for many small images, as i can pretty reliably find and extract just the item labels but would need to OCR up to 20 every 100ms.

If you already know where the text is in each image, you could try cropping out the labels and stacking them as lines in a temporary image which is passed to recognize_text. Internally recognize_text groups the lines into batches that have approximately the same width, and then runs recognition over each batch in parallel.

Edit: I should add that training a custom model could be a good option if you know exactly the font and alphabet you want to recognize. The default model pays a price for being fairly general.

[mrispoli24]

So I just built a ruby gem for using ocrs with Active Storage for Ruby on Rails. It works really well for most use cases. I integrated tesseract as well to see how that faired with lower quality images (like photos of receipts). This implementation performs just as well if not better than tesseract. I'm not sure what others are going to use it for but my primary use case is PDF documents which work amazing, and then photos of reciepts. I think I'm going to need to run some pre-processing in my library. However if anyone wants to compare ocrs with tesseract side by side you can check out my demo: https://activestorage-ocr-demo.fly.dev/

[cool-RR]

i started using ocrs for a program i made that lets me automatically click URLs or local paths seen as text in any desktop app, using the keyboard.

[GICodeWarrior]

I'm using ocrs (and rten) on screenshots from a PC game to recognize text and classify icons (repo is called fir).

ocrs and rten are working great for my use case because they are efficient, small, and I can compile to wasm or native. 🙏

First I process each screenshot to locate the elements I want to recognize/classify, then I pass these to ocrs or rten as needed to perform the recognition.

For text recognition, I slop-coded a synthetic data finetuning capability into ocrs_models (available in my fork). Then I ran that with the font I need to support and based on the latest checkpoint from HuggingFace.

I did find detection to be relatively slow, and I already have the text in lines, so I don't load a detection model and instead call recognize_text directly.

I also found rten threading dramatically slows processing for my use case (lots of small icons, etc.), so I disable threading.

unsafe {
    std::env::set_var("RTEN_NUM_THREADS", "1");
}

It would be cool if I could remove rayon altogether (both to save a little on binary size and overhead).

The next thing I probably need to figure out is support for other character sets (for folks who run the game with localizations enabled).

@robertknight thanks for building and maintaining these great projects!

[robertknight]

It would be cool if I could remove rayon altogether (both to save a little on binary size and overhead).

Removing rayon entirely is not practical given how much it is used throughout the library. Adding a safe API to set the thread count would be possible though. When using rten directly you can create a thread pool with a given number of threads via ThreadPool::with_num_threads and use it for inference with RunOptions::thread_pool. Ocrs doesn't expose an API to set the thread pool at the moment however.

[GICodeWarrior]

Makes sense, thanks. I was thinking it might be possible to shim rayon, similar to this:
https://github.com/shssoichiro/maybe-rayon/blob/master/src/lib.rs

I haven't attempted it though.