Fix ISO 8601 date pattern accepting impossible month/day values#2113
Open
jichaowang02-lang wants to merge 1 commit into
Open
Fix ISO 8601 date pattern accepting impossible month/day values#2113jichaowang02-lang wants to merge 1 commit into
jichaowang02-lang wants to merge 1 commit into
Conversation
The "ISO 8601 datetime" pattern in DateRecognizer used `[01]\d` for the month and `[0-3]\d` for the day. These ranges admit impossible values: month `00` and `13`-`19`, and day `00` and `32`-`39`. As a result strings such as `2024-13-15T14:30:00Z` and `2024-12-32T14:30Z` were detected as DATE_TIME. Every other date pattern in this same file already constrains the month to `01`-`12` and the day to `01`-`31`; only the ISO 8601 pattern was loose. Tighten the ISO month/day fields to match (using non-capturing groups so existing capture-group positions are unaffected). No valid ISO 8601 datetime is lost, since those values are not valid dates to begin with. Adds parametrized cases for invalid month (00, 13) and day (00, 32).
| Pattern( | ||
| "ISO 8601 datetime", | ||
| r"\b(\d{4}-[01]\d-[0-3]\dT[0-2]\d:[0-5]\d:[0-5]\d\.\d+([+-][0-2]\d:[0-5]\d|Z))|(\d{4}-[01]\d-[0-3]\dT[0-2]\d:[0-5]\d:[0-5]\d([+-][0-2]\d:[0-5]\d|Z))|(\d{4}-[01]\d-[0-3]\dT[0-2]\d:[0-5]\d([+-][0-2]\d:[0-5]\d|Z))\b", | ||
| r"\b(\d{4}-(?:0[1-9]|1[0-2])-(?:0[1-9]|[12]\d|3[01])T[0-2]\d:[0-5]\d:[0-5]\d\.\d+([+-][0-2]\d:[0-5]\d|Z))|(\d{4}-(?:0[1-9]|1[0-2])-(?:0[1-9]|[12]\d|3[01])T[0-2]\d:[0-5]\d:[0-5]\d([+-][0-2]\d:[0-5]\d|Z))|(\d{4}-(?:0[1-9]|1[0-2])-(?:0[1-9]|[12]\d|3[01])T[0-2]\d:[0-5]\d([+-][0-2]\d:[0-5]\d|Z))\b", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
"ISO 8601 datetime"pattern inDateRecognizeruses[01]\dfor the month and[0-3]\dfor the day. These ranges admit values that cannot occur in a real date:00and13–1900and32–39So strings like
2024-13-15T14:30:00Zor2024-12-32T14:30Zare detected asDATE_TIME.Every other date pattern in this same file already constrains the month to
01–12([1-9]|0[1-9]|1[0-2]) and the day to01–31([1-9]|0[1-9]|[1-2][0-9]|3[0-1]). Only the ISO 8601 pattern was left loose — this looks like an oversight rather than intent.Fix
Tighten the month/day fields of the ISO pattern to valid ISO 8601 ranges (zero-padded, since ISO 8601 always uses 2-digit fields):
Non-capturing groups are used so the existing capture-group positions in the pattern are unchanged. No valid ISO 8601 datetime is lost — the excluded values are not valid dates.
Tests
Added parametrized cases for invalid month (
00,13) and day (00,32). All existingtest_date_recognizer.pycases still pass (39 passed total).ruff check .is clean.