Cloud OCR using 3rd party API

[aldavies1]

Describe the pain point and your solution
Could you add a function to select 3rd party API tools (user bringing own key) to perform the OCR? E.g. Claude, Groq, OpenAI etc? I'd like to keep the existing on-device options but having cloud OCR option could be quite useful for those of us who don't have a copilot+ PC so can't use the new windows AI framework.

Mode which would include change

• Settings Window
• General

Describe alternatives you've tried or considered
This approach is similar to applications such as Open Whispr, and the following screenshot shows an example of how this application has implemented this kind of functionality.

Screenshots or sketches

[TheJoeFin]

For some reason the images are not showing

[aldavies1]

That's odd. I can see them. I'll send to you by return email.

[aldavies1]

Hey, ich finde sie haben ihre arbeit kann toll gemacht. Falls sie mal irgenmdwann vorhaben einen Bediensteten zu angarien , der ihnen manchmal nur behilfig sein soll um was laut vorzulesen , dann lassen sie mich es wissen. ;-)

Es hat mich gefreut. Alles Gute.

[AjiTae]

Workaround :)

You can reuse the existing Tesseract integration by pointing it to a custom executable that mimics the expected interface.

Requirements

Your executable must:

1. Return a list of languages:

your.exe --list-langs

2. Perform OCR and print the result to stdout:

your.exe "C:\image\path.png" - -l lang

---

Example (Node.js + OpenRouter)

1. Configure Text-Grab Tesseract to use ocr-proxy.bat:

@echo off
set OPENROUTER_API_KEY=<your key here, or set it globally>
"C:\Program Files\nodejs\node.exe" "C:\scripts\ocr-proxy.js" %*

2. ocr-proxy.js:

import fs from 'node:fs/promises';
import os from 'node:os';
import { randomUUID } from 'node:crypto';

try {
  const args = process.argv.slice(2);

  const defaultModel = 'openai/gpt-4o';
  const defaultLanguage = 'auto';

  const optionsTable = {
    'en': {
      lang: 'English'
    },
    'auto': {},
    'son': {
      model: 'anthropic/claude-3.7-sonnet'
    },
    'son-en': {
      lang: 'English',
      model: 'anthropic/claude-3.7-sonnet'
    }
  };

  if (args[0] === '--list-langs') 
    listLangsAndExit(Object.keys(optionsTable));

  const { lang = defaultLanguage, model = defaultModel } = 
    optionsTable[args[args.indexOf('-l') + 1]];
  const [ image ] = args;

  process.stdout.write(await openRouterOCR({ image, lang, model }));
} catch (e) {
  process.stdout.write(String(e));
}

// Use OpenRouter API to extract text from image, you can write any wrapper you want
async function openRouterOCR({ image, lang, model }) {
  if (!model || model === 'auto') model = defaultModel;
  lang = (!lang || lang === 'auto') ? '' : lang + ' ';

  const empty = randomUUID();
  const imageBuffer = await fs.readFile(image);
  const response = await fetch('https://openrouter.ai/api/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model,
      messages: [
        {
          role: 'user',
          content: [
            {
              type: 'text',
              text:
                `Extract all readable ${lang} text from this image as accurately as possible.` +
                ` Return only the extracted text, nothing else.` +
                ` Return exactly ${empty} if no text.`,
            },
            {
              type: 'image_url',
              image_url: {
                url: `data:image/jpeg;base64,${imageBuffer.toString('base64')}`,
              },
            },
          ],
        },
      ],
      max_tokens: 2000, // more?
      temperature: 0.0,
    }),
  });

  const data = await response.json();

  if (!response.ok)
    throw new Error(data?.error?.message || 'Failed to get response');

  const text = data.choices[0].message.content.trim();
  
  if (text === empty) 
    return '';
  
  return text;
}

//  List of available languages, can use it to also provide model selection (currently 3ch limit)
function listLangsAndExit(langs) {
  process.stdout.write([
    `List of available languages in "/usr/local/share/tessdata/" (${langs.length}):`,
    ...langs
  ].join(os.EOL));

  process.exit(0);
}

---

This setup allows you to intercept Tesseract calls and forward them to any backend (e.g., OpenRouter) while preserving compatibility with the expected CLI interface.