Skip to content

feat(selection): 增加剪贴板回退和截图 OCR #7

Description

@v833

Suggested labels: enhancement

Background

Current behavior:

  • The app depends on the Windows text selection service exposed through selection-hook.

Problem:

  • Some PDFs, canvases, remote desktops, images, video subtitles, and custom controls do not expose selected text.
  • Users receive no guided fallback when the native selection API cannot return content.

Needed change:

  • Add a safe clipboard fallback and an optional screenshot OCR workflow.

Goal

Extend useful selection coverage without silently reading or replacing unrelated clipboard content.

User Stories

  • As a user in an unsupported application, I want a clear fallback using copied text.
  • As a user viewing an image or scanned PDF, I want to select a screen region and run OCR locally.
  • As a privacy-conscious user, I want recognized text reviewed before it is sent to a model.

Requirements

1. Clipboard fallback

  • Detect native selection failure and present a Copy then retry flow.
  • Preserve and restore the previous clipboard when automatic capture is enabled.
  • Never send clipboard content until the user chooses an AI action.

2. Screenshot OCR

  • Add an explicit region-capture command with clear cancellation.
  • Prefer a proven local OCR engine and support Chinese and English initially.
  • Show recognized text for correction before invoking actions.

3. Source awareness

  • Label results as native selection, clipboard, or OCR source.
  • Keep OCR disabled in blacklisted applications.

Acceptance Criteria

  • Unsupported native selection produces a clear fallback instead of failing silently.
  • Clipboard fallback preserves previous clipboard contents when configured.
  • OCR region selection supports cancel and never captures outside the chosen region.
  • Recognized text can be reviewed and edited before sending.
  • OCR runs locally by default and no image is uploaded implicitly.
  • Result source is visible to the user.
  • Tests cover clipboard restoration, cancellation, OCR errors, and privacy exclusions.

Test Plan

  • Automated tests: clipboard state helper and OCR workflow state machine.
  • Manual verification: browser canvas, scanned PDF, image, video subtitle, and remote desktop samples.
  • Regression coverage: supported native text selection remains the fastest path.

Out of Scope

  • Full-screen continuous OCR.
  • Cloud OCR without explicit opt-in.
  • Document layout reconstruction.

Dependencies / Risks

  • OCR increases package size and should be evaluated against an optional-download design.
  • Clipboard restoration can race with other applications and requires conservative timing.
  • Region capture must respect multi-monitor DPI and application blacklist rules.

Effort Estimate

  • Implementation: 8-12 days.
  • Tests + docs: 3-4 days.
  • Total: 11-16 days.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions