Skip to content

fix(analyzer): normalize case before fi hetu and sg uen checksums#2141

Open
uwezkhan wants to merge 1 commit into
data-privacy-stack:mainfrom
uwezkhan:fi-sg-checksum-case
Open

fix(analyzer): normalize case before fi hetu and sg uen checksums#2141
uwezkhan wants to merge 1 commit into
data-privacy-stack:mainfrom
uwezkhan:fi-sg-checksum-case

Conversation

@uwezkhan

@uwezkhan uwezkhan commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Change Description

validate_result in two recognizers compares a check character against an upper-case reference without normalizing case. Patterns are matched with regex.IGNORECASE, so a valid ID written with a lower-case letter matches the pattern but is then dropped by the checksum, so the entity is not returned.

  • FiPersonalIdentityCodeRecognizer compares the HETU control character against "0123456789ABCDEFHJKLMNPRSTUVWXY".
  • SgUenRecognizer compares the UEN prefix, entity-type and check letter against upper-case tables (all three formats).

Before: 131052-308t and 53125226d match the regex, fail validation, and are not detected.
After: the check character is upper-cased before the comparison, so both detect at the same span and score as their upper-case forms. Invalid checksums stay rejected, so no new false positives.

Same normalization #2076 applied to the ES NIF/NIE recognizers. Added lower-case regression cases to both recognizer test files.

Issue reference

No existing issue.

Checklist

  • I have reviewed the contribution guidelines
  • I agree to follow this project's Code of Conduct
  • I confirm that I have the right to submit this contribution and that it does not knowingly contain proprietary or confidential code.
  • My code includes unit tests
  • All unit tests and lint checks pass locally
  • My PR contains documentation updates / additions if required

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant