1552 implement needs ocr in sample caller

[Luis-manzur]

This pull request adds a new --ocr-available flag to the sample_caller script, enhancing its ability to detect when OCR (Optical Character Recognition) should be used for document extraction. It introduces a new utility module for robustly detecting when OCR is needed, and updates the workflow to leverage this logic when the flag is set.

this PR addresses -- #1552

[flooie]

Can you explain the point of the ocr utils. In particular is_doc_common_header. I'm not sure I understand why we are adding this?

Sample caller is meant to be, just that, a sample caller?

[Luis-manzur]

Can you explain the point of the ocr utils. In particular is_doc_common_header. I'm not sure I understand why we are adding this?

Sample caller is meant to be, just that, a sample caller?

I added it so I could test certain scenarios, such as in texbizct, where in some PDFs much of the information is in images rather than plain text.

So this is to simulate CL's behavior when necessary.

ocr_utils and the functions within it are the same functions initially used in CL to detect whether OCR should be used or not. The problem with that implementation is that its use case is for PACER, which is why I added the option to detect missing pages, what I consider a more general approach.

These changes should also be applied in CL

[grossir]

Closing this since

• it has grown stale
• For these special courts I think it will be better to just test the whole integration through the CL docker environment. This will just make juriscraper more complex for an uncommon use case.

We should eventually stop copying code from CL into juriscraper and viceversa (like we did with download_content). A good chunk is copied from here https://github.com/freelawproject/courtlistener/blob/main/cl/lib/recap_utils.py

The problem with that implementation is that its use case is for PACER, which is why I added the option to detect missing pages, what I consider a more general approach.

These kind of fixes should be on doctor. We should probably write about the problematic courts / documents first