Merged
Conversation
Introduce GrabTemplate, TemplateRegion, and PostGrabContext classes to enable reusable OCR templates for fixed-layout documents. Templates define named, numbered regions and an output format for assembling OCR results. PostGrabContext encapsulates all data from a grab action, supporting both simple and template-based post-processing. This lays the foundation for structured data extraction and advanced OCR workflows.
Replaces settings-based template storage with a robust file-based system (GrabTemplateManager) supporting CRUD, migration, and error handling. Implements GrabTemplateExecutor for OCR region extraction and flexible output template formatting with modifiers and escapes. Includes comprehensive unit tests for both manager and executor.
- Add TemplateId property to ButtonInfo for template actions - Include Grab Templates in available post-grab actions - Add ApplyTemplate_Click action handling in PostGrabActionManager - Overload ExecutePostGrabAction to accept PostGrabContext - Improve BarcodeUtilities with null checks and error handling - Update comments and summaries for clarity and maintainability
Added three user-scoped settings: GrabTemplatesJSON (string), AddToContextMenu (bool), and RegisterOpenWith (bool). Updated App.config, Settings.settings, and Settings.Designer.cs to support these options. Also updated the code generation version in Settings.Designer.cs.
Add specific handling for COMException when saving settings exceeds the ApplicationDataContainer 8 KB size limit. Log a clear debug message suggesting large data be stored in a file. Also, clarify the generic exception debug message.
Introduce a new "Grab Templates" section in the Post-Grab Actions editor, allowing users to create, view, and delete OCR region templates. Add UI for managing templates, including a list view and actions to open the Grab Frame or delete templates. Update Fullscreen Grab settings to link to the template management UI, making template-based OCR more accessible and discoverable. Includes supporting code for loading, refreshing, and deleting templates, as well as minor UI consistency improvements.
Users can now define and save "Grab Templates" in the Grab Frame window by Ctrl+dragging regions and using a new "Save as Template" button. Templates store region layouts for structured extraction in fullscreen grabs. Added UI for naming and formatting templates, and logic to display template region overlays during fullscreen selection. Post-grab actions now receive region context for template-based processing. Includes XAML cleanup and is fully opt-in via Post-Grab Actions settings.
- Add "Edit Grab Template" button to open templates in GrabFrame for region editing with reference image support - Implement inline template editor for name/output template changes - Show region index badges on template regions for clarity - Save and load template reference images in a dedicated folder - Improve XAML consistency, tooltips, and button labeling - Ensure template IDs and creation dates are preserved on edit - Update region badges dynamically as regions are modified
Introduces InlinePickerRichTextBox, enabling users to insert and remove inline "chip" elements (with label and value) via a popup picker triggered by typing '{'. Includes supporting InlineChipElement and InlinePickerItem classes, plus XAML styles for a compact, modern UI. Chips are serializable to and from plain text, supporting scenarios like variable or tag insertion.
Introduce InlinePickerRichTextBox for editing Grab Template output, enabling users to insert region chips via a picker UI. Hide the old plain TextBox and update all related logic to use the new control, including serialization and deserialization of template output. Ensure region chips are kept in sync with word borders, and improve file handling for template images. Also includes XAML cleanup and minor refactoring for maintainability.
Removed the hidden OutputTemplateBox TextBox from the UI. Updated TemplateOutputBox (InlinePickerRichTextBox) to support multi-line and tab input, and clarified its tooltip.
TheJoeFin
added
enhancement
Added Microsoft.WindowsAppSDK.WinUI version 1.8.260204000 to the project file to enable or support WinUI features. No other changes were made.
Refactor content area and image rect calculations to use WPF's layout engine and PointToScreen, eliminating hardcoded offsets and improving accuracy across DPI settings. Update border and resize settings for better appearance and usability. These changes ensure robust, layout-independent screen coordinate handling.
- Fix Viewbox scaling to use both width and height (Uniform) - Crop frozen image to selection for GrabFrame continuity - Add GrabFrame constructor for pre-loaded cropped images - Explicitly size overlay canvas to match image pixels - Rescale word borders from history for accurate overlays - Refine content/non-content area calculations for sizing - Improve background color sampling for word borders - Add robustness checks for invalid/unmeasured sizes - Slightly increase GrabFrame title bar height for UI polish These changes resolve overlay misalignment and scaling issues, ensuring accurate OCR word border placement and a consistent user experience when grabbing, freezing, and editing regions.
Add visual dimming for template regions not referenced in the output template, both in the GrabFrame editor and FullscreenGrab preview. WordBorder controls now support a dimmed border state, and region highlighting updates dynamically as the template is edited. Also includes minor XAML cleanup for menu item formatting. This improves clarity when editing templates by making unused regions visually distinct.
Introduce "Grab Template" as a new LookupItemKind and post-grab action. Add TemplateId to LookupItem and update icon mapping. Overload LaunchFullScreenGrab to allow preselecting a template. Update FullscreenGrab to auto-check or add preselected templates in the context menu, and rename the edit menu item. Add "Manage Grab Templates..." to EditTextWindow and GrabFrame. In QuickSimpleLookup, display templates, allow Ctrl+click to edit, and Enter to launch a grab with the template. Integrate templates throughout lookup, grab, and post-processing flows.
- Add ListViewScrollFix.xaml to correct scrollbar thumb sizing by properly binding ViewportSize in GridViewScrollViewerTemplate. - Refactor PostGrabActionEditor layout for better scrolling and alignment using Grids and max heights. - Replace "Open Grab Frame" with "New Template from Image" workflow; update button logic and help text accordingly. - Change word region highlighting: referenced regions are now highlighted orange instead of dimming unused ones. - Remove unused/hidden template edit button and clean up related code.
Extend Grab Templates to support regex pattern placeholders alongside
existing region placeholders. Users can reference saved StoredRegex
patterns from the Regex Manager using {p:PatternName:mode} syntax.
New placeholder syntax:
- {p:Name:first} - first regex match
- {p:Name:last} - last regex match
- {p:Name:all:separator} - all matches joined by separator
- {p:Name:N} - Nth match (1-based)
- {p:Name:1,3} - specific matches joined by separator
Key changes:
- New TemplatePatternMatch model for pattern references
- GrabTemplate now supports pattern-only and hybrid templates
- GrabTemplateExecutor processes pattern placeholders via full-area OCR
- InlinePickerRichTextBox shows grouped popup (Regions/Patterns sections)
- PatternMatchModeDialog for configuring match mode after selection
- GrabFrame integrates pattern items in template editor
- 15 new tests covering all match modes and edge cases
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add null guards for controls accessed during InitializeComponent before XAML template is fully loaded (IndicesErrorText, OkButton, IndicesTextBox). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Save _triggerStart before opening the PatternMatchModeDialog. The modal dialog causes OnLostKeyboardFocus to fire, which nulls _triggerStart. Using the saved pointer prevents the TextRange null argument crash. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When a template's output references no regions (only pattern placeholders), don't draw the region overlay rectangles during fullscreen grab selection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Refactored GrabTemplate to a partial class and added a unique Id property. Replaced inline regex with [GeneratedRegex] partial methods for better performance. Simplified menu item check logic in FullscreenGrab. Removed an unused using directive from QuickSimpleLookup.
Image/canvas sizing now uses device-independent pixels (DIPs) to ensure overlays align correctly at all DPI scales. Replaces inline regexes for template parsing with [GeneratedRegex] static partial methods for improved performance and maintainability.
There was a problem hiding this comment.
Pull request overview
Adds “Grab Templates” to Text Grab to let power users define reusable OCR region/pattern templates and apply them during Fullscreen Grab / from Quick Simple Lookup, while also addressing UI/layout overlap and Grab Frame freeze word-border positioning.
Changes:
- Introduces GrabTemplate models, file-based persistence, execution (region OCR + regex pattern placeholders), and template management UI.
- Integrates templates into Fullscreen Grab post-actions, Quick Simple Lookup, and Grab Frame (template save/edit mode + inline placeholder picker).
- Fixes/adjusts Grab Frame freeze/image sizing and various UI layout issues (incl. Post-Grab Actions editor overlap/scrolling).
Reviewed changes
Copilot reviewed 42 out of 43 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| Text-Grab/Views/QuickSimpleLookup.xaml.cs | Adds Grab Templates into lookup items and enables launching template edit/apply flows. |
| Text-Grab/Views/GrabFrame.xaml.cs | Adds template editing/saving mode, improves freeze sizing/word-border placement, and supports pattern placeholder picking. |
| Text-Grab/Views/GrabFrame.xaml | UI changes for template save panel + layout adjustments. |
| Text-Grab/Views/FullscreenGrab.xaml.cs | Adds preselected template support and live overlay rendering of template regions during selection. |
| Text-Grab/Views/FullscreenGrab.xaml | Adds overlay host canvas for template region outlines. |
| Text-Grab/Views/EditTextWindow.xaml.cs | Adds entry point to manage templates. |
| Text-Grab/Views/EditTextWindow.xaml | Adds “Manage Grab Templates…” menu item and formatting tidy-up. |
| Text-Grab/Utilities/WindowUtilities.cs | Adds overload to launch Fullscreen Grab with a preselected template id. |
| Text-Grab/Utilities/PostGrabActionManager.cs | Includes templates as post-grab actions; adds ExecutePostGrabAction overload accepting PostGrabContext. |
| Text-Grab/Utilities/ImageMethods.cs | Uses GrabFrame layout-derived content rect for accurate screen capture bounds (DPI/layout-safe). |
| Text-Grab/Utilities/GrabTemplateManager.cs | Implements GrabTemplate CRUD, migration from settings, and reference-image persistence. |
| Text-Grab/Utilities/GrabTemplateExecutor.cs | Implements template execution (region OCR + output formatting + regex pattern placeholders). |
| Text-Grab/Utilities/BarcodeUtilities.cs | Adds guard/exception handling around barcode decode for robustness. |
| Text-Grab/Themes/Generic.xaml | Adds default styles for new inline chip/picker controls. |
| Text-Grab/Text-Grab.csproj | Updates package versions. |
| Text-Grab/Styles/ListViewScrollFix.xaml | Adds a template override to correct ListView/GridView scrollbar thumb sizing. |
| Text-Grab/Services/SettingsService.cs | Handles ApplicationDataContainer 8KB limit with a specific COMException catch. |
| Text-Grab/Properties/Settings.settings | Adds GrabTemplatesJSON setting (used for migration away from settings). |
| Text-Grab/Properties/Settings.Designer.cs | Regenerates settings designer and adds GrabTemplatesJSON property. |
| Text-Grab/Pages/FullscreenGrabSettings.xaml.cs | Adds “Manage Templates” button handler. |
| Text-Grab/Pages/FullscreenGrabSettings.xaml | Adds “Grab Templates” section and button in settings UI. |
| Text-Grab/Models/TemplateRegion.cs | New model for ratio-based template regions. |
| Text-Grab/Models/TemplatePatternMatch.cs | New model describing saved-regex references and match modes. |
| Text-Grab/Models/PostGrabContext.cs | New post-grab pipeline context object (text + region + DPI + language). |
| Text-Grab/Models/LookupItem.cs | Adds GrabTemplate lookup item kind + TemplateId field. |
| Text-Grab/Models/GrabTemplate.cs | New template model (regions + pattern matches + output template + reference sizes). |
| Text-Grab/Models/ButtonInfo.cs | Adds TemplateId to support template-backed post-grab actions. |
| Text-Grab/Extensions/ControlExtensions.cs | Fixes Viewbox scale factor calculation to match Uniform behavior (width/height aware). |
| Text-Grab/Controls/WordBorder.xaml.cs | Adds template badge/index + output-referenced highlighting support. |
| Text-Grab/Controls/WordBorder.xaml | Adds template index badge UI. |
| Text-Grab/Controls/PostGrabActionEditor.xaml.cs | Adds template list + create/edit/delete flows and refresh logic. |
| Text-Grab/Controls/PostGrabActionEditor.xaml | Expands editor layout and adds Grab Templates management section. |
| Text-Grab/Controls/PatternMatchModeDialog.xaml.cs | New dialog for configuring pattern match mode + separator/indices. |
| Text-Grab/Controls/PatternMatchModeDialog.xaml | New dialog UI for pattern match configuration. |
| Text-Grab/Controls/NotifyIconWindow.xaml | Minor XAML attribute ordering change. |
| Text-Grab/Controls/InlinePickerRichTextBox.cs | New inline placeholder picker rich text box with chips and grouped popup. |
| Text-Grab/Controls/InlinePickerItem.cs | New model for picker items (display/value/group). |
| Text-Grab/Controls/InlineChipElement.cs | New chip control used inside the rich text box. |
| Text-Grab/App.xaml | Imports ListView scroll fix resources. |
| Text-Grab/App.config | Adds GrabTemplatesJSON entry to user settings. |
| Tests/Tests.csproj | Updates test-related package versions. |
| Tests/GrabTemplateManagerTests.cs | Adds unit tests for template persistence, migration robustness, and action bridging. |
| Tests/GrabTemplateExecutorTests.cs | Adds unit tests for output formatting and pattern placeholder behavior. |
Files not reviewed (1)
- Text-Grab/Properties/Settings.Designer.cs: Language not supported
Comment on lines
+365
to
+367
| if (!available.Contains(num)) | ||
| issues.Add($"Placeholder {{{{num}}}} references region {num} which does not exist."); | ||
|
|
There was a problem hiding this comment.
ValidateOutputTemplate builds an incorrect placeholder string in the error text: "Placeholder {{{{num}}}} ..." will render as "Placeholder {{num}}" instead of showing the actual missing placeholder (e.g. {3}). Use an interpolated format that includes the numeric value so the message is actionable (e.g. render {3}).
Comment on lines
+58
to
+62
| /// <param name="captureRegion"> | ||
| /// The screen rectangle (in WPF units, pre-DPI-scaling applied by caller) | ||
| /// that bounds the user's selection. Template region ratios are applied to | ||
| /// this rectangle's width/height. | ||
| /// </param> |
There was a problem hiding this comment.
The XML doc for ExecuteTemplateAsync says captureRegion is in WPF units, but OcrUtilities.GetTextFromAbsoluteRectAsync treats the rect as physical screen pixels (it converts to System.Drawing.Rectangle and screen-captures). This mismatch is likely to cause future callers to pass the wrong coordinate space; update the doc to explicitly state the expected units (physical pixels).
Comment on lines
+446
to
+450
| /// <summary> | ||
| /// Draws scaled template region overlays inside the current selection border. | ||
| /// Regions are scaled using UniformToFill semantics relative to the template's | ||
| /// reference image dimensions so they fill the selected area proportionally. | ||
| /// </summary> |
There was a problem hiding this comment.
UpdateTemplateRegionOverlays’s summary says it uses “UniformToFill semantics”, but the implementation positions regions directly via stored ratios against selWidth/selHeight (i.e., stretch-to-fill). Either adjust the scaling logic to match the documented behavior or update the comment so it reflects what the code actually does.
Comment on lines
+468
to
+470
| // Stretch: scale each axis independently to fill the selection exactly | ||
| double scaleX = selWidth / template.ReferenceImageWidth; | ||
| double scaleY = selHeight / template.ReferenceImageHeight; |
There was a problem hiding this comment.
scaleX/scaleY are computed but never used, which makes the overlay math harder to follow and suggests a leftover from an earlier scaling approach. Remove these variables or use them in the region placement calculations (and keep the comment in sync).
Owner
Author
|
@copilot open a new pull request to apply changes based on the comments in this thread |
Contributor
|
@TheJoeFin I've opened a new pull request, #627, to work on those changes. Once the pull request is ready, I'll request review from you. |
…ables Co-authored-by: TheJoeFin <7809853+TheJoeFin@users.noreply.github.com>
Fix doc/comment accuracy and dead code in grab template execution
Separate regular actions and templates in the context menu, add section separators, and improve handling of preselected templates. Enforce exclusive selection among template actions and ensure check states are managed correctly. Improves menu clarity, usability, and robustness.
Implement centralized state management and synchronization for post-grab actions (including templates and custom actions) across all open FullscreenGrab windows. Refactor menu item handling to use snapshot-based logic, ensuring exclusive template selection and consistent "LastUsed" persistence. Add keyboard shortcut support (Ctrl+1–9) for toggling actions and propagate changes to all windows. Update UI refresh logic and introduce comprehensive unit and WPF tests for the new synchronization features.
Introduce region, window, freeform, and adjust-after selection styles for Fullscreen Grab, wire the new UI and settings surfaces, and add focused tests for the supporting models and utilities. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Guard barcode reads against invalid or disposed bitmaps, ensure random-access-stream bitmap copies stay usable after the source stream is released, and add regression coverage for both cases. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Refactor cropping logic to use a new TryGetBitmapCropRectForSelection method, which accurately maps selection rectangles to bitmap coordinates even when zoom or pan transforms are applied. Remove code that forcibly resets zoom/pan state. Increase selection handle size for better usability. Add unit tests to verify cropping correctness under various transform scenarios.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 61 out of 62 changed files in this pull request and generated 5 comments.
Files not reviewed (1)
- Text-Grab/Properties/Settings.Designer.cs: Language not supported
Comments suppressed due to low confidence (1)
Text-Grab/Utilities/PostGrabActionManager.cs:199
- Post-grab action LastUsed persistence is keyed by
action.ButtonTextinGetCheckState/SaveCheckState. With templates, names are user-editable and not guaranteed unique, so LastUsed state can collide between different template actions and can be lost when a template is renamed. Consider persisting check state using a stable, unique key (e.g.,TemplateIdwhen present, otherwiseClickEvent+ButtonText).
Comment on lines
+422
to
+432
| private void AddPostGrabActionMenuItem(ContextMenu contextMenu, ButtonInfo action, bool isChecked, bool stayOpen, int shortcutIndex) | ||
| { | ||
| MenuItem menuItem = new() | ||
| { | ||
| Header = action.ButtonText, | ||
| IsCheckable = true, | ||
| Tag = action, | ||
| IsChecked = isChecked, | ||
| StaysOpenOnClick = stayOpen, | ||
| InputGestureText = $"Ctrl+{shortcutIndex}" | ||
| }; |
There was a problem hiding this comment.
AddPostGrabActionMenuItem always sets InputGestureText = $"Ctrl+{shortcutIndex}", but keyboard handling only supports Ctrl+1..Ctrl+9. When more than 9 actions are enabled (more likely now that templates are actions), menu items will show shortcuts like Ctrl+10 that don't work. Consider omitting InputGestureText (or capping shortcut assignment) for indices > 9.
Comment on lines
734
to
747
| MainDataGrid.ItemsSource = null; | ||
|
|
||
| ItemsDictionary.AddRange(ParseStringToRows(contentToParse, true)); | ||
| lookupItems = [.. ItemsDictionary]; | ||
|
|
||
| if (DefaultSettings.LookupSearchHistory) | ||
| { | ||
| AddHistoryItemsToItemsDictionary(); | ||
| SearchHistory.IsChecked = true; | ||
| } | ||
|
|
||
| AddGrabTemplatesToItemsDictionary(); | ||
|
|
||
| MainDataGrid.ItemsSource = ItemsDictionary; |
There was a problem hiding this comment.
LoadDataGridContent appends parsed rows/history/templates into ItemsDictionary without clearing it first. If the user loads a CSV via “Import from CSV” (or other code paths that call LoadDataGridContent without a prior clear), history and templates will be duplicated each time. Consider clearing ItemsDictionary (and syncing lookupItems) at the start of LoadDataGridContent before adding rows/history/templates.
Comment on lines
+31
to
+55
| public static readonly DependencyProperty ItemsSourceProperty = | ||
| DependencyProperty.Register( | ||
| nameof(ItemsSource), | ||
| typeof(IEnumerable<InlinePickerItem>), | ||
| typeof(InlinePickerRichTextBox), | ||
| new PropertyMetadata(null)); | ||
|
|
||
| public static readonly DependencyProperty SerializedTextProperty = | ||
| DependencyProperty.Register( | ||
| nameof(SerializedText), | ||
| typeof(string), | ||
| typeof(InlinePickerRichTextBox), | ||
| new FrameworkPropertyMetadata( | ||
| string.Empty, | ||
| FrameworkPropertyMetadataOptions.BindsTwoWayByDefault)); | ||
|
|
||
| #endregion Dependency Properties | ||
|
|
||
| #region Properties | ||
|
|
||
| public IEnumerable<InlinePickerItem> ItemsSource | ||
| { | ||
| get => (IEnumerable<InlinePickerItem>)GetValue(ItemsSourceProperty); | ||
| set => SetValue(ItemsSourceProperty, value); | ||
| } |
There was a problem hiding this comment.
ItemsSource's dependency property defaults to null, but the CLR wrapper getter casts directly to IEnumerable<InlinePickerItem>, which will throw NullReferenceException when GetValue(...) returns null. This is already exercised by GrabFrame.xaml.cs using TemplateOutputBox.ItemsSource ?? [] (the ?? won't help if the getter throws). Return an empty sequence when unset (and/or set the DP default to an empty collection).
Comment on lines
+41
to
+47
| public override void OnApplyTemplate() | ||
| { | ||
| base.OnApplyTemplate(); | ||
|
|
||
| if (GetTemplateChild(PartRemoveButton) is Button removeButton) | ||
| removeButton.Click += (s, e) => RemoveRequested?.Invoke(this, EventArgs.Empty); | ||
| } |
There was a problem hiding this comment.
OnApplyTemplate attaches a new Click handler to PART_RemoveButton every time the template is applied. Since OnApplyTemplate can be called multiple times over a control's lifetime, this can lead to duplicate event handlers and multiple RemoveRequested invocations per click. Consider detaching any previous handler or using RemoveHandler/a stored delegate before reattaching.
Comment on lines
+94
to
+100
| <MenuItem | ||
| x:Name="FreeformSelectionMenuItem" | ||
| Click="SelectionStyleMenuItem_Click" | ||
| Header="Freeform - Draw any shape" | ||
| InputGestureText="P" | ||
| IsCheckable="True" | ||
| Tag="Freeform" /> |
There was a problem hiding this comment.
The Freeform selection menu item advertises the shortcut key as "P", but the code-behind handles Freeform selection style on the "D" key (Key.D). This makes the UI hint incorrect and confusing; update InputGestureText to match the actual shortcut (or update the key handling to match the hint).
Owner
Author
|
@copilot open a new pull request to apply changes based on the comments in this thread |
Contributor
|
@TheJoeFin I've opened a new pull request, #629, to work on those changes. Once the pull request is ready, I'll request review from you. |
…sSource, duplicate handlers, Freeform key hint Co-authored-by: TheJoeFin <7809853+TheJoeFin@users.noreply.github.com>
Fix five review issues: shortcut hints, ItemsDictionary duplicates, null ItemsSource, duplicate chip handlers, Freeform key hint
Replaced ComboBox with a set of RadioButtons for selecting the pattern match mode in the dialog, improving visibility and usability. Updated event handling and logic to support the new selection method. Increased dialog height to fit the revised layout.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #564