Skip to content

feat(documents): extract bounded local text and exact evidence from repository documents #465

Description

@styler-ai

Why

Repositories often contain design, operational, release, and product evidence in text-bearing PDF and office documents. ProjectAtlas can classify these files, but it cannot yet route an agent to bounded extracted facts with an exact page, section, sheet, row, or cell origin.

What Changes

  • Add deterministic offline extraction for selected text-bearing PDF and OOXML formats.
  • Start with embedded text only; raster OCR, multimodal vision, hosted inference, and generated summaries are out of scope.
  • Use established license-compatible parsers and return a typed unavailable reason when no trustworthy bounded implementation fits.
  • Enforce raw-file, archive-member, decompressed-byte, compression-ratio, page, row, cell, text-byte, time, cancellation, and worker ceilings.
  • Preserve extractor identity/version, completeness, warnings, exact source-file identity, and document locators.
  • Feed the existing classification, summary, lexical search, purpose, relation, freshness, and next-call flows instead of adding a broad document-query tool.
  • Invalidate from the source document and publish through the existing atomic generation.

Capabilities

  • bounded-document-extraction: local embedded-text extraction with explicit resource and safety limits.
  • exact-document-evidence: typed page, section, sheet, row, and cell locators suitable for bounded agent navigation.

Architecture Diagrams

Release Scope

Target v0.8.0 after v0.5 performance/distribution and v0.6 interface/continuity work. Accept formats independently; do not hold a proven PDF slice for an unready spreadsheet or presentation parser.

Non-Goals

  • OCR, image understanding, hosted services, LLM summarization, or document mutation.
  • A generic document framework, second database, sidecar text authority, eighth crate, or new MCP tool by default.
  • Executing macros, formulas, external relationships, embedded programs, or repository code.
  • Claiming semantic implementation links from ordinary document prose.

Acceptance Criteria

  • Supported formats return bounded text with exact typed locators, extractor provenance, and complete/partial/unavailable state.
  • Malformed, truncated, encrypted, password-protected, oversized, recursive, high-compression, external-link, symlink, special-file, and out-of-root inputs fail safely without network access.
  • Full and incremental add/edit/rename/delete/ignore behavior preserves one current atomic generation and the last complete generation on failure.
  • Schema/key/index/query/transaction design is proven first with migration/reopen, rollback, corruption refusal, prepared batches, and query-plan tests.
  • Existing CLI/MCP summary, search, relation, and exact-navigation workflows expose the evidence without implicit mutation or unbounded output.
  • Packaged Windows, Linux, macOS arm64, and macOS x86-64 tests cover dependency availability and identical typed fallbacks.
  • Performance gates measure raw/decompressed bytes, pages/cells, wall time, CPU, RSS, staged bytes, persistent-size amplification, changed-document cost, parallel crossover, and cancellation latency.
  • Representative agent tasks improve correct-document and exact-evidence selection without increasing wrong-file visits, broad reads, calls, output, or backtracking beyond frozen bounds.

Pre-Mortem

  • Parser inputs become an archive-bomb or path-traversal surface: validate containers and relationships before extraction under hard limits.
  • Extracted text loses its source location: reject facts without exact file and page/section/sheet provenance.
  • A sidecar becomes competing authority: persist only normalized facts in the existing generation and invalidate from the original file.
  • One difficult format stalls the whole release: gate and ship formats independently.
  • Extraction dominates scan resources: perform bounded work off-writer under the existing host-wide budget and publish once.

OpenSpec Tasks

OpenSpec change: to be created when the first independently accepted v0.8.0 format is scheduled. No implementation task is authorized by this backlog issue.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions