Add the Ukrainian conjunction й (#267 comment, #269 follow-up) - #306
Merged
Conversation
Ukrainian writes і and й for the same conjunction, alternating on the
surrounding vowel and consonant for euphony ("Олесь і Олена" but
"Марія й Петро"), so real data carries both spellings. #269's candidate
list read "Ukrainian і/та" and й was never considered; it therefore
shipped recognizing one of the two and reading the other as a name
piece -- 'Олесь й Олена Коваленки' put the й in middle.
Pure data: no code path changes, and the entry inherits both existing
guards. The single-letter carve-out (Google Code issue 11) still holds
it back until the segment has enough rootname pieces, so a 3-piece
'Іван й Олена' stays split exactly as 'і' and 'y' do; and the initial
regex's dotted branch is Unicode-aware, so a punctuated 'Й.' remains an
initial. The second is worth a standing test rather than a comment --
и/і/й each open real given names (Игорь, Ірина, Йосип) -- so pin all
three dotted forms.
The differential harness needs nothing: no corpus name contains й, and
the #269 rule in expected_changes.toml already classifies this diff
shape (Cyrillic, fields title/first/middle) had one appeared.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #306 +/- ##
=======================================
Coverage 98.45% 98.45%
=======================================
Files 41 41
Lines 2776 2776
=======================================
Hits 2733 2733
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Where this came from
A reply on #267 noted, while confirming that a bare Cyrillic letter should read as the conjunction rather than an initial:
We ship
и,іandта— notй. #269's candidate list read "Cyrillic conjunctions:и; Ukrainianі/та", soйwas never considered and rejected; it just never made the list.йis the euphonic alternate ofі. Ukrainian picks between them by the surrounding vowel and consonant, not by meaning — "Олесь і Олена" but "Марія й Петро" — so real data carries both spellings and recognizing one of the two mis-parses the other:The change
One entry in
nameparser/config/conjunctions.py. Pure data — no code path changes — and it inherits both guards the existing single-letter conjunctions rely on:Іван й Оленаstays split exactly asіandydo.initialregex is Unicode-aware, so a punctuatedЙ.remains an initial and is untouched.That second one seemed worth a standing test rather than a comment:
и/і/йeach open real given names (Игорь, Ірина, Йосип), so the shipped entries would be a trap if membership could outrank a punctuated initial.test_267_dotted_cyrillic_initial_wins_over_conjunctionpins all three dotted forms.Tests
Two rows added to the
#269non-Latin parametrization — the title-chain shape theи/і/таrows use, and the given-name join from the issue comment — plus the dotted-initial guard. Both new rows were watched failing on the old data before the entry landed.2699 passed, 32 skipped, 11 xfailed; mypy and ruff clean.Notes
corpus*.jsonlcontainsй, so there is no new diff; and had one appeared, the existingfeat(#269) non-Latin titles/conjunctions recognizedrule inexpected_changes.tomlalready classifies exactly this shape (Cyrillic, fieldstitle/first/middle).Also checked, no action needed
The rest of that comment holds up against current behavior:
И. Иванов,І. Франко,И. С. Петровall parse as initials — the typography point Should a bare 'И' parse as an initial or the conjunction 'and'? (e.g. 'Хосе И Мария Сантос') #267 already settled.А. П. и Е. Н. Ивановыgivesfirst: 'А.',middle: 'П. и Е. Н.',last: 'Ивановы'. Lossy-looking, but identical to the Latin analoguesA. P. and E. N. IvanovsandA. P. y E. N. Santos— the general multi-person-string limitation, not a Cyrillic bug.Ивановы→Иванов/Иванова) are morphological normalization; nothing analogous exists for any language here.иserves ru/bg/sr/mk,і/й/таserve uk/be.🤖 Generated with Claude Code