Skip to content

fix(analyzer): stop quadratic backtracking in in_pan low pattern #2068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
uwezkhan wants to merge 2 commits into
base: main
Choose a base branch
from
+25 −1
Open

fix(analyzer): stop quadratic backtracking in in_pan low pattern #2068

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e9aa8fa
fix(analyzer): stop quadratic backtracking in in_pan low pattern
uwezkhan Jun 17, 2026
ba62a43
test(analyzer): use perf_counter for in_pan backtracking guard
uwezkhan Jul 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...yzer/presidio_analyzer/predefined_recognizers/country_specific/india/in_pan_recognizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ class InPanRecognizer(PatternRecognizer):
),
Pattern(
"PAN (Low)",
r"\b((?=.*?[a-zA-Z])(?=.*?[0-9]{4})[\w@#$%^?~-]{10})\b",
r"\b((?=[\w@#$%^?~-]*?[a-zA-Z])(?=[\w@#$%^?~-]*?[0-9]{4})[\w@#$%^?~-]{10})\b",

@uwezkhan uwezkhan Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at bounding both scans to {0,9}, but kept the token-class ? form. The quadratic cost came from . walking across whitespace, so each lookahead rescanned the rest of the text at every word boundary. Scoping them to [\w@#$%^?~-] makes each scan stop at the first non-token character, so the work is already linear in input length (100 KB dropped from ~32 s to ~9 ms).

Before: {0,9}? would cap the scan at 9 characters. After: the current form also caps it, because the trailing \b forces the token to end at a boundary. The difference is that the class contains non-word symbols (@#$%^?~-), so an explicit {0,9} would change which spans match for tokens built from those symbols, while the *? form preserves the existing matches exactly. Given the scan is already bounded by the token class, I'd rather not shift match semantics.

0.01,
),
]
Expand Down
24 changes: 24 additions & 0 deletions presidio-analyzer/tests/test_in_pan_recognizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import time

import pytest

from tests import assert_result
Expand Down Expand Up @@ -47,3 +49,25 @@ def test_when_pan_in_text_then_all_pans_found(
expected_position[1],
expected_score,
)


def test_low_confidence_pan_with_embedded_digits_still_detected(recognizer, entities):
# The low-confidence pattern keeps matching a 10-char token that carries a
# letter and a four-digit run; scoping the lookaheads to the token class
# leaves this detection unchanged.
results = recognizer.analyze("A1111DFSFS", entities)
assert len(results) == 1
assert_result(results[0], entities[0], 0, 10, 0.01)


def test_low_confidence_pattern_does_not_backtrack(recognizer, entities):
# The low-confidence pattern used `.*?` lookaheads that rescanned the whole
# remaining text at every word boundary, so a long boundary-rich string with
# no four-digit run cost time quadratic in its length. Matching must now be
# linear and finish well within the regex timeout.
text = "a " * 50000
start = time.perf_counter()
results = recognizer.analyze(text, entities)
elapsed = time.perf_counter() - start
assert results == []
assert elapsed < 10
Comment on lines +68 to +73

@uwezkhan uwezkhan Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Switched the guard to perf_counter (monotonic, higher resolution) and pushed. The threshold stays at 10s on purpose: the patched pattern runs this input in single-digit milliseconds, so it is a coarse did-not-blow-up guard rather than a tight perf assertion, which keeps it stable across CI machines.

Loading