WordKeep - PDF OCR Application

A Laravel 12 API server + Angular 21 client for processing PDF documents using AI vision models for OCR (Optical Character Recognition). Designed for digitizing old books with high fidelity text extraction. Supports a local LM Studio model as primary provider with automatic Groq cloud fallback.

Features

Authentication & Security

• User Authentication - Token-based auth with Laravel Sanctum
• Email Verification - Verify email before account activation
• Password Reset - Email-based password reset flow
• Password Change - Two-step modal (verify current password → set new); email notification on success
• Two-Factor Authentication - TOTP-based 2FA (Google Authenticator, 1Password, etc.); enable/disable from Account Security tab; 8 single-use recovery codes generated on enable; email notification on enable and disable
• User Isolation - Each user sees only their own documents

Landing Page (Guest)

• Charter at / — Guest visitors land on a medieval-style charter (Latin body, illuminated drop cap, twin wax seals on cord, Order of the Steadfast Knight imprint); authenticated users are redirected to /dashboard
• Latin↔English Hover Swap — On desktop, hover or keyboard-focus any Latin block (legend, opening, article titles/bodies, manu propria signatures, closing) and the Latin fades out while English fades in; un-hover/un-focus restores. On touch, tap toggles via aria-expanded
• Sign-in / Sign-up CTAs — Footer offers "Subscribe to the charter" (/register) and "Resume the office" (/login)

Dashboard & Recent Activity

• Recent Activity Feed — Dashboard shows up to 10 most recent (document, work-type) entries, ordered by latest activity. Each card displays the filename, author (when present), a colored work-type badge (OCR, Page Linking, Book View, Translation — {Language}), the last page worked + total pages, a progress bar, and a relative timestamp (2h ago, 3d ago, or DD/MM/YYYY for entries older than a week)
• One Entry Per Work Type — A single book worked on for both OCR and Page Linking surfaces as two separate entries
• Click to Resume — Clicking an entry navigates to the appropriate reader (/extraction/:id, /page-linking/:id, /book-view/:id, or /translation/:id/:language); each reader auto-resumes from the last page worked
• Empty State — A helper message ("No recent activity yet — open a document to get started") shows for fresh users

List View Sorting

• Sort Dropdown on Every List Page — Documents, Extraction, Page Linking, Book View, and Translation each render a sort selector next to the section header
• Three Choices — Last modified (default, most-recent first), Title (A→Z), Title (Z→A); switching sort resets pagination to page 1
• Section-Aware "Last modified" — Each list sorts by the timestamp most relevant to that section: Documents uses documents.updated_at; Extraction by the user's most recent ocr work_status; Page Linking by page_linking work_status; Book View by book_view work_status; Translation by the translation row's own updated_at. Documents with no work_status of the relevant type sink to the bottom
• Server-Side Sort — Applied at the database query level so pagination stays correct across pages
• Choice Persists Per Page — Each list page remembers its sort independently across navigation and reloads via localStorage (wk:sort:<page> keys)

Document Management

• PDF Upload - Drag-and-drop or file selection with progress
• Document Metadata - Author, description, genre fields
• Document Renaming - Rename documents with restore original option
• Cover Preview - View first page as document cover
• Page-by-Page Processing - Extract individual pages as images
• Work Status Tracking - Remembers last accessed page per document and per work type. Bumps the timestamp on any in-page work — running or saving an OCR edit, toggling page-ignore, saving a page-linking decision, translating, saving/deleting a page translation, dismissing a wrong-language flag — so the document surfaces in Recent Activity even without page navigation
• Separate Sections - Documents section for management, Text Extraction for OCR

Smart PDF Analysis

• Content Type Detection - Automatically detects page type (Text, Scanned, Mixed)
• Direct Text Extraction - Fast plain text extraction from text-based PDFs (no AI needed)
• Hybrid Page Support - Pages with both text and images detected automatically
• Memory-Efficient Parsing - Handles large scanned PDFs without memory issues

AI-Powered OCR

• Local LM Studio (Primary) - Uses local vision models (e.g., Qwen2.5-VL) when available; auto-starts the LM Studio server via lms CLI if installed but not running
• Groq Cloud Fallback - Auto-falls back to Groq free API (Llama 4 Scout) when LM Studio is unavailable or model not downloaded
• Automatic Liveness Check - Detects LM Studio availability per request (configurable cache TTL)
• High-Fidelity Transcription - Optimized prompts for exact text reproduction with diacritics preservation
• Markdown Output - Results formatted in CommonMark Markdown with formatting
• Override Re-run OCR - Re-process cached pages when needed
• Large Image Optimization - Auto-resize images for provider compatibility

Bulk OCR Processing

• Batch Processing - Process entire documents in background
• Progress Tracking - Real-time progress updates
• Cancel Operations - Cancel ongoing bulk OCR at any time

Editing & Output

• WYSIWYG Editor - Rich text editor with formatting (bold, italic, underline, strike, blockquotes, lists, headings, links)
• Manual Transcription - Type text while viewing the page image; saved only on explicit Save
• Edit Mode Toggle - Switch between view and edit modes
• Per-Page Editing - Edit OCR text for each page independently
• Clear Page Text - Delete extracted text for individual pages
• Page Ignore — Toggle any page as ignored from the extraction reader; ignored pages show a badge and blurred gray overlay, all action buttons are disabled, and the page is excluded from book view and export; counts as completed for both OCR and page linking progress; skipped by First/Next Unprocessed in both extraction and linking views

Footnotes

• Footnote CRUD - Create, edit, and delete footnotes per document
• Page Linking - Link footnotes to specific pages via [^N] references in text
• Insert Button - Add footnote references through a dropdown (prevents manual typing)
• Unlink - Remove footnote references from page text without deleting the footnote
• Auto-Renumbering - Deleting a footnote renumbers all higher footnotes and updates page text refs
• Cancel Undo - Cancelling an edit unlinks any footnotes linked during that session
• Clickable References - [^N] refs in view mode render as clickable superscript numbers that scroll to the footnote panel
• Validation - Server rejects manually typed/removed [^N] refs; must use insert/unlink buttons

Spell Checking

• Multi-Language — Checks English, French, German, Spanish, Italian, Latin, Romanian
• Custom Romanian Dictionary — 215,604 entries with expanded re-/ne- prefix forms
• User Dictionary — Add/remove words per language; persists to server, shared across devices
• Inline Suggestions — Click underlined word → popup with suggestions, Ignore, Add to Dictionary
• No Browser Interference — Native browser spell check disabled globally

User Profile

• Profile Dropdown - Click avatar circle to open menu with Profile Info, Account Security, Personalisation, My Dictionary, Logout
• Profile Picture - Upload and circle-crop avatar photo (JPEG/PNG/WEBP, max 5 MB); displayed in header across all pages
• Name Editing - Inline name editing with validation
• Email Change - Two-step flow (current password confirmation → new email address); verification link sent to new address; 5-day expiry
• Pending Email Banner - Shows pending change with Resend Verification and Cancel options
• Account Security Tab - Change password and manage two-factor authentication from /profile/security

Theme & Layout Personalisation

• Six Layout Variants — Pick from Reading Room (default editorial baseline), Workbench (slim top + bottom dock), Compendium (newspaper masthead), Glass (floating capsule), Scriptorium (manuscript folio with rubric column), or Console (terminal prompt + status bar). Selection persists in localStorage (wk:layout) and is applied via data-layout on <html>.
• Six Palettes — Reading Room (cream + wine + brass), Cobalt (bone + cobalt + vermilion), Monastic (vellum + lapis + sepia), Grove (birch + moss + ochre), Ferrous (ash + rust + steel), Plum (alabaster + plum + gilt). Each palette ships full light + dark token sets; activity badges use 4 well-spaced hues per palette so the four work types stay visually distinct.
• Theme Mode — Auto (follows system prefers-color-scheme), Light, or Dark; segmented control in the header for one-click switching.
• Personalisation Page — /profile/personalisation, a third sub-page alongside Info and Security. Hosts the layout picker (6 cards), palette picker (6 swatches), and theme-mode segmented control. Selections take effect instantly with no reload.
• Console-only / Command Palette — when on the Console layout, pressing / opens a centred terminal-style overlay for keyboard-driven navigation (autocomplete on routes, ↑↓ to move, Enter to navigate, Esc to close). The listener is scoped so typing / inside an <input>, <textarea>, or [contenteditable] doesn't hijack — same pattern as GitHub / Linear / Notion.

Page Linking

• Consecutive Page Pairs — View two adjacent pages side-by-side to determine how text flows between them
• New Row / Continue Row — Tag each boundary as a new paragraph row or a continuation of the previous page
• Progress Tracking — Per-document progress bar shows how many pairs have been tagged out of total
• Work Status Resume — Remembers the last viewed pair and resumes from there on next visit
• First/Next Unprocessed — Jump directly to the next untagged pair without manual navigation
• Page Ignore — Pages marked as ignored show a blurred gray overlay in the pair viewer; affected pairs display an IGNORED indicator instead of decision buttons and offer a plain Next button to advance without saving

AI Translation

• Bulk Translation via Gemini 2.5 Flash — Translate all pages of a document into a target language using Google's Gemini 2.5 Flash model
• Per-Page Translation — Translate individual pages from the translation reader; supports override-retry for failed pages
• Footnote Translation — Footnotes are translated alongside pages; visible as tooltips in the translation reader
• Cost Estimation — Estimate token usage and cost before starting a bulk run; no charge if zero pages to translate
• Progress Tracking — Live progress bar with polling; shows pages completed out of total
• Cancel / Re-translate — Cancel an in-progress run; re-translate with override flag to overwrite existing translations
• Translation List — Paginated list of all translations with status badges, progress, and open-in-reader action
• Wrong-Language Detection — Pages and footnotes whose translation output is detected as the source language instead of the target are flagged; separate count badges on each translation card show how many pages and footnotes are affected; inside the reader a dedicated navigation bar cycles through all pages with any failure (page text or footnote), and banners offer Solve / Retry (retry with a stronger prompt) and Dismiss (accept result) actions; the footnote tooltip's translate button highlights in amber and shows "Retry Translation" when the footnote is flagged
• Manual Paragraph Splitting — When a single paragraph exceeds the 3000-word translation limit the page is flagged as needs_split; a split editor in the left pane lets the user click to add split points (snapped to word boundaries), inspect per-segment word counts, and confirm; the resulting chunks are translated independently and rejoined with the correct separator (space for mid-paragraph cuts, blank line for paragraph-boundary cuts)
• Delete Translation — Remove a translation record and all associated page/footnote translations

Book View & Export

• Document List — Paginated list of ready documents with OCR and page linking progress bars; resume-from-last-page badge; export dropdown
• E-Reader Layout — 1-page or 2-page side-by-side mode with virtual CSS pagination (no full-page reloads)
• Virtual Pagination — Page breaks computed via Range.getClientRects() for line-level accuracy; variable viewport heights prevent partial lines
• Font Size Control — Adjustable from 70% to 160% in 10% steps; reading position preserved across size changes
• Reading Position — Auto-saved on every page turn via work status; resumes from last page on next visit
• Footnote Refs — [^N] refs render as clickable superscript numbers; click opens a tooltip showing the footnote text (× to close)
• Markdown Export — Assembles pages into a .md file respecting CONTINUE_ROW/NEW_ROW links; footnote definitions appended at the bottom in standard extended Markdown syntax for compatibility with previewers like Jotbird; available from both list and reader

Other Features

• Language Detection - Automatically detects document language on upload (LM Studio: image-based vision detection; fallback: OCR + ELD text analysis). Retry is available from the document detail page if detection fails.
• Result Caching - OCR results cached per page to avoid redundant AI calls
• Themed Dialogs - Styled confirmation dialogs for destructive operations

Requirements

Pick a workflow before installing:

• Sail (Docker) — everything (PHP, MySQL, Ghostscript, Composer, Angular client) runs inside containers. Simpler dependency setup, slower IDE feedback on macOS/Windows.
• Herd (host PHP) — PHP and MySQL run on the host. Faster feedback, requires installing the dependencies yourself.

For Sail

• Docker — Docker Desktop (macOS/Windows) or Docker Engine (Linux)
• Windows: WSL2 is required — install Ubuntu from the Microsoft Store, then enable WSL integration for it in Docker Desktop under Resources → WSL Integration. Run all commands from the Ubuntu terminal, not PowerShell or CMD.

Windows performance warning: Docker on Windows is significantly slower if your project lives on the Windows filesystem (C:\Users\...). Always clone the repository inside the WSL2 filesystem (e.g. ~/dev/wordkeep) and work from there. Performance will be on par with Linux.

For Herd

• Laravel Herd — https://herd.laravel.com (macOS / Windows). Linux users: install PHP 8.4+, MySQL 8+, and Composer manually.
• Node 20+ for the Angular client.
• Ghostscript on PATH (gs on Linux/Mac, gswin64c on Windows).

Both

• LM Studio with a vision-capable model (optional if Groq API key configured)

Installation

Option A — Herd (host PHP)

1. Clone the repository:

   git clone <repository-url>
   cd wordkeep

2. Install dependencies:

   composer install
   cd wordkeep-client && npm install && cd ..

3. Configure PHP (host php.ini). Increase these limits — required for running the test suite and uploading large PDFs:

   memory_limit = 512M
   upload_max_filesize = 128M
   post_max_size = 130M
   max_execution_time = 300

   Locate your active php.ini with php --ini. On Laravel Herd (Windows) it lives at ~/.config/herd/bin/php84/php.ini (swap php84 for your installed PHP version). Restart Herd (or your web server) after editing. (Sail users skip this — the Docker image already ships these limits in docker/8.5/php.ini.)

4. Configure environment:

   cp .env.herd.example .env
   php artisan key:generate

5. Configure OCR providers in .env (LM Studio defaults to http://localhost:1234/v1 — fine for Herd):

   LM_STUDIO_MODEL=qwen2.5-vl-7b-instruct   # must match the model key shown by `lms ls`
   GROQ_API_KEY=your-groq-api-key
   GEMINI_API_KEY=your-gemini-api-key

6. Set up the database:

   • Create a MySQL database called wordkeep (Herd ships with a MySQL service).
   • Update DB_DATABASE, DB_USERNAME, DB_PASSWORD in .env to match.
   • Run migrations and link storage:

     php artisan migrate
     php artisan storage:link

7. Configure the dev-server proxy. The Angular dev server proxies /api to your backend; the target is read from wordkeep-client/.env.local. If you're using Herd's reverse proxy (http://wordkeep.test), copy the example and set the target:

   cd wordkeep-client
   cp .env.local.example .env.local
   # then edit to set API_TARGET=http://wordkeep.test and API_URL=http://wordkeep.test/api
   cd ..

   If you're using php artisan serve --port=80 instead, the defaults (http://localhost) work and .env.local is optional.

8. Start the dev servers (three terminals, or use the JetBrains run configs in .run/):

   # Terminal 1: API
   php artisan serve
   
   # Terminal 2: queue worker (OCR / translation jobs)
   php artisan queue:work
   
   # Terminal 3: Angular dev server
   cd wordkeep-client && npm start

   Access the app at http://localhost:4200.

9. LM Studio (optional): see OCR Provider Setup below.

Option B — Sail (Docker)

1. Clone the repository:

   git clone <repository-url>
   cd wordkeep

2. Add the Sail alias (once per machine, then restart your terminal or run source ~/.bashrc):

   echo "alias sail='[ -f sail ] && sh sail || sh vendor/bin/sail'" >> ~/.bashrc

3. Configure environment:

   cp .env.sail.example .env

4. Configure OCR providers in .env (LM Studio defaults to http://host.docker.internal:1234/v1 so the Sail container can reach it on the host):

   # LM Studio (primary - optional if Groq configured)
   LM_STUDIO_BASE_URL=http://host.docker.internal:1234/v1
   LM_STUDIO_MODEL=qwen2.5-vl-7b-instruct   # must match the model key shown by `lms ls`
   LM_STUDIO_TIMEOUT=120
   LM_STUDIO_MAX_TOKENS=4096
   LM_STUDIO_LIVENESS_CACHE_TTL=0
   LM_STUDIO_AUTO_START=true                 # auto-start via lms CLI if server not running
   LM_STUDIO_START_TIMEOUT=10                # seconds to wait for server to become ready
   
   # Groq cloud fallback (free API key from https://console.groq.com)
   GROQ_API_KEY=your-groq-api-key
   GROQ_MODEL=meta-llama/llama-4-scout-17b-16e-instruct
   
   # Google Gemini (for AI translation)
   GEMINI_API_KEY=your-gemini-api-key

5. Start the containers and generate the app key:

   sail up -d
   sail artisan key:generate

6. Run migrations:

   sail artisan migrate

7. Create public storage symlink (required for serving avatars and other public files):

   sail artisan storage:link

8. LM Studio (optional): install LM Studio and download a vision-capable model (e.g., Qwen2.5-VL-7B-Instruct). The app auto-starts the server via lms CLI when a request arrives and the server is not running. If LM Studio is not installed or the configured model is not downloaded, requests fall back to Groq automatically. Skip entirely if using Groq only.

Headed E2E Tests (visual debugging)

Running Playwright with --headed works best from the host, not from inside the client container. The container exposes the server on port 80 and the Angular dev server on port 4200, both reachable from the host, so the host's Playwright drives the browser natively while the API and frontend stay containerized.

One-time host setup (Node 20+ required):

cd wordkeep-client
npm install
npx playwright install chromium

Why not run headed inside the container? Chromium's renderer can't reliably draw into a window over forwarded X11. On Docker Desktop specifically, bind-mounted Unix sockets (the conventional X11-forwarding pattern) don't proxy connections at all — connect() returns ECONNREFUSED. Even with a TCP-bridge workaround, Chromium-specific extensions (MIT-SHM, GLX) silently fail to initialize and the window contents stay transparent. Running on the host sidesteps all of this.

Client Setup (Sail only)

The Angular client runs as a Docker service alongside the backend. It starts automatically with sail up -d — no separate setup needed.

Access the app at http://localhost:4200

npm install runs automatically on container start. On first run this takes a minute; subsequent restarts are fast because node_modules is stored in a named Docker volume.

(Herd users: cd wordkeep-client && npm start.)

Running Tests

Backend Tests

Sail requires sail up -d to be running (tests use the MySQL testing database). Herd uses your local MySQL.

# Sail
sail artisan test
sail artisan test --filter=AuthControllerTest
sail php vendor/bin/phpunit

# Herd
php artisan test
php artisan test --filter=AuthControllerTest
vendor/bin/phpunit

Frontend Tests

# Sail
sail exec client npm test
sail exec client npm test -- --coverage

# Herd
cd wordkeep-client
npm test
npm test -- --coverage

E2E Tests (Playwright)

For Sail: requires sail up -d to be running. For Herd: requires the API (php artisan serve) and the Angular dev server (npm start) to be running.

Important: The API .env must have CACHE_STORE=array before running E2E tests. Without it, login rate limiting will throttle the test suite after the 5th test (429 errors). The JetBrains run configs (.run/) handle this automatically — they patch .env before the run and restore it after. If running manually, set it yourself:

# In the project root .env
CACHE_STORE=array

# Run headless in the container (browsers are pre-installed in the client container)
sail exec -e API_URL=http://server/api client npx playwright test

# Run headed with slow motion (visual debugging — Playwright runs on host; see setup above)
cd wordkeep-client

# Install browsers (first time only)
npx playwright install chromium

# Run headless — viewport auto-detected to match your primary monitor
npx playwright test

# Run headed (visible browser, maximised on whichever monitor it pops up on)
HEADED=1 SLOW_MO=2000 npx playwright test --workers=1

The JetBrains run configs automate both flows. Sail variants live in .run/sail/ with a (Sail) suffix in the IDE Run dropdown ("Run E2E tests (Sail)" runs headless inside the container; "Run E2E tests headed, slow mo 2 secs, 1 worker (Sail)" runs from the host). Herd variants live in .run/ root without the suffix.

Viewport behaviour

By default the suite detects your primary monitor's resolution (PowerShell on Windows, system_profiler on macOS, xdpyinfo on Linux) and uses it as the headless viewport. On CI (CI env detected) and when detection fails, it falls back to 1366×768. Setting HEADED=1 switches to a maximised visible browser on whichever monitor it lands on (viewport: null + --start-maximized). Either path can be overridden explicitly with VIEWPORT_WIDTH / VIEWPORT_HEIGHT.

Per-developer config: wordkeep-client/.env.local

The Angular dev server's /api proxy and Playwright fixtures both read wordkeep-client/.env.local (gitignored). Copy .env.local.example and edit:

cd wordkeep-client
cp .env.local.example .env.local

Variable	Default	Description
API_TARGET	http://localhost	Where ng serve proxies /api requests to. Set to http://wordkeep.test for Herd's reverse proxy.
API_URL	http://localhost/api	Absolute base URL the Playwright fixtures hit for login + seeding. From inside the Sail client container, http://server/api.

Defaults match Sail (host port 80 → server container) and Herd via php artisan serve --port=80. Sail's in-container Playwright run config sets API_URL=http://server/api explicitly and ignores .env.local.

Other E2E environment variables

Variable	Default	Description
HEADED	0	Set to 1 to run with a visible browser maximised on whichever monitor the OS picks. Pairs naturally with SLOW_MO for visual debugging.
VIEWPORT_WIDTH	(auto-detected)	Browser viewport width in pixels. Falls back to 1366 on CI / detection failure. Set explicitly to override the auto-detected value.
VIEWPORT_HEIGHT	(auto-detected)	Browser viewport height in pixels. Falls back to 768 on CI / detection failure. Pair with VIEWPORT_WIDTH.
SLOW_MO	(unset)	Milliseconds Playwright pauses between actions — for visual debugging. Typical values: 1000–2000. Has no effect in headless mode.

Bash / zsh example (host, headed):

VIEWPORT_WIDTH=1920 VIEWPORT_HEIGHT=1080 SLOW_MO=2000 npx playwright test --headed

PowerShell example (Windows host):

$env:VIEWPORT_WIDTH="1920"; $env:VIEWPORT_HEIGHT="1080"; npx playwright test

Inside the client container (Sail, headless):

sail exec -e API_URL=http://server/api -e VIEWPORT_WIDTH=1920 -e VIEWPORT_HEIGHT=1080 client npx playwright test

Configuration

Environment Variables

Variable	Default	Description
LM_STUDIO_BASE_URL	http://localhost:1234/v1	LM Studio API endpoint
LM_STUDIO_MODEL	local-model	Model name in LM Studio
LM_STUDIO_TIMEOUT	120	Request timeout in seconds
LM_STUDIO_MAX_TOKENS	4096	Maximum tokens for AI response
LM_STUDIO_IMAGE_MAX_DIMENSION	512	Max dimension for language detection images
LM_STUDIO_OCR_IMAGE_MAX_DIMENSION	768	Max dimension for OCR images
LM_STUDIO_LIVENESS_CACHE_TTL	0	Seconds to cache LM Studio availability (0 = check every request)
LM_STUDIO_AUTO_START	true	Auto-start LM Studio server via lms CLI if not running
LM_STUDIO_START_TIMEOUT	30	Seconds to wait for auto-started server to become ready
GROQ_API_KEY	(empty)	Groq API key for cloud fallback (get one free)
GROQ_MODEL	meta-llama/llama-4-scout-17b-16e-instruct	Groq vision model
GROQ_BASE_URL	https://api.groq.com/openai/v1	Groq API endpoint
GROQ_TIMEOUT	120	Groq request timeout in seconds
GROQ_MAX_TOKENS	4096	Max tokens for Groq response
GHOSTSCRIPT_PATH	gs	Path to Ghostscript executable (gs on Linux/Mac, gswin64c on Windows)
PDF_IMAGE_FORMAT	png	Image format for page extraction
PDF_IMAGE_DPI	150	DPI for page image extraction
PDF_MAX_FILE_SIZE	104857600	Max upload size in bytes (100MB)
OCR_CACHE_ENABLED	true	Enable OCR result caching
GEMINI_API_KEY	(empty)	Google Gemini API key for AI translation
GEMINI_MODEL	gemini-2.5-flash-preview-04-17	Gemini model for translation

Optional opt-out flags (heavy local testing)

These all default to behavior-preserving values — set them only if you want to suppress a background process during heavy E2E or load testing:

Variable	Default	Effect when set to false
LANGUAGE_DETECTION_ENABLED	true	DetectDocumentLanguage job exits early on dispatch — useful to avoid Groq TPM rate limits when uploading many test documents back-to-back.
BULK_OCR_ENABLED	true	ProcessBulkOcr job exits early — keeps the document at pending so a cancel call can win the race during testing.
QUEUE_WORKER_MANAGED	(unset)	When set truthy, the auto-spawn EnsureQueueWorkerIsRunning listener short-circuits — set this if a separate process supervisor (e.g. supervisord) is already running the worker, to avoid duplicate workers.

API Endpoints

Interactive API documentation is available at /docs/api (served by Scramble).

File Storage Structure

storage/app/private/documents/
└── {document_id}/
    ├── {uuid}.pdf              # Original PDF file
    └── pages/
        ├── page_001.png        # Extracted page images (on-demand)
        ├── page_002.png
        └── ...

OCR Provider Setup

Option A: LM Studio (Local, Primary)

1. Download LM Studio from https://lmstudio.ai/
2. Install the lms CLI (bundled with LM Studio)
3. Download a vision-capable model inside LM Studio (recommended: Qwen2.5-VL-7B-Instruct)
4. Set LM_STUDIO_MODEL in .env to the exact model key shown by lms ls
5. You do not need to start the server manually — the app starts it automatically on first request

Auto-start behavior:

• When an OCR request arrives, the app checks if LM Studio's server is running
• If not, it calls lms server start in the background and polls for up to LM_STUDIO_START_TIMEOUT seconds
• The model is not pre-loaded; LM Studio loads it automatically on the first inference request
• If the server is already running but the configured model is not available, the app falls back to Groq immediately
• LM Link is supported — if the model lives on another device via LM Link, it takes a few seconds to appear after server start; the polling window gives it time to connect
• Disable auto-start with LM_STUDIO_AUTO_START=false

PATH note: The lms CLI must be accessible on the host PATH. If auto-start doesn't work despite lms being installed, ensure it's in your system PATH and restart your terminal.

Recommended Models

Model	VRAM	Quality	Speed
Qwen2.5-VL-7B-Instruct	~8GB	Good	Fast
Qwen2.5-VL-72B-Instruct	~48GB	Excellent	Slow
LLaVA-1.6-34B	~24GB	Very Good	Medium

Option B: Groq Cloud (Free Fallback)

Used automatically when LM Studio is unavailable or the configured model is not downloaded. No GPU required.

1. Create a free account at https://console.groq.com
2. Generate an API key
3. Add GROQ_API_KEY=your-key to .env

The app checks LM Studio availability on each request (configurable via LM_STUDIO_LIVENESS_CACHE_TTL). If LM Studio is down or the model is missing, requests go to Groq automatically.

Language detection also adapts: uses LM Studio's vision model when available, otherwise OCRs sample pages via Groq and detects language from the extracted text using the ELD library.

License

[Add your license here]