Skip to content

Commit ed71eb4

Browse files
derek73claude
andcommitted
Report the ambiguous-suffix readings instead of guessing silently
particles_ambiguous emitted PARTICLE_OR_GIVEN; suffix_acronyms_ambiguous emitted nothing, for the structurally identical situation -- a word in an ambiguous vocabulary where the parser picks one of two readings. "parse('derek gulbranson MA').ambiguities" was empty even though the parser had just chosen credential over surname. Two kinds now emit: SUFFIX_OR_FAMILY (new) at the peel in _assign, in BOTH directions -- "John Smith MA" reports that MA was read as a suffix, "Jack MA" that it was read as the family name. Both are guesses. SUFFIX_OR_NICKNAME (reserved since the core API landed) at classify, for delimited content the vocabulary cannot settle: "(MBA)" escapes to suffix on vocabulary alone, so "(JD)" keeping the nickname reading was a silent coin-flip. Emitted at classify rather than at the escape in extract, which runs before tokenize and so has no token index to point at. The emission has to be at the DECISION site, not on tag presence. In "Joao da Silva do Amaral de Souza" the token 'do' carries vocab:suffix-ambiguous but sits mid-name and is never at the peel boundary -- no choice was made, so nothing is reported. Same reasoning excludes "Ma, Jack", where a comma fixes the family before the question arises: that mirrors the existing rule that PARTICLE_OR_GIVEN is not emitted on the FAMILY_COMMA path. An ambiguity is a property of a decision, not of a token. Corpus impact is ~1%: 5 of 486 names. test_every_ambiguity_kind_has_a_registered_trigger caught the new kind immediately and the case table's exact-ambiguities assertion caught all six affected rows -- both guards did their job. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 8147ac6 commit ed71eb4

8 files changed

Lines changed: 126 additions & 12 deletions

File tree

docs/release_log.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ Release Log
2424
- Add ``PatronymicRule`` with the members ``EAST_SLAVIC`` and ``TURKIC``. v1's single ``patronymic_name_order`` flag enabled both detectors at once; ``Policy(patronymic_rules=...)`` lets you enable either one alone
2525
- Add ``PolicyPatch`` and the ``UNSET`` sentinel for partial policy deltas that compose -- set-valued fields union, scalar fields override with later winning. This is the mechanism locale packs are built from, and ``UNSET`` is only needed when you must distinguish "not set" from a real ``False`` or ``None``
2626
- Add ``Token``, ``Span`` and ``Role``: every field is backed by tokens carrying exact ``(start, end)`` offsets into the original string, reachable with ``tokens_for(Role.GIVEN)``. This replaces v1's ``*_list`` attributes and makes it possible to highlight or re-slice the input the parse came from
27-
- Add ``Ambiguity`` and the ``AmbiguityKind`` enum, so a parse reports what it had to guess at instead of guessing silently. The kinds emitted today are ``particle-or-given``, ``unbalanced-delimiter`` and ``comma-structure``; ``order`` and ``suffix-or-nickname`` are reserved and not yet emitted
27+
- Add ``Ambiguity`` and the ``AmbiguityKind`` enum, so a parse reports what it had to guess at instead of guessing silently. The kinds emitted today are ``particle-or-given``, ``suffix-or-family``, ``suffix-or-nickname``, ``unbalanced-delimiter`` and ``comma-structure``; ``order`` is reserved and not yet emitted. The two suffix kinds cover the post-nominals that are also ordinary words: ``"John Smith MA"`` reports that ``MA`` was read as a credential rather than a surname, and ``"JEFFREY (JD) BRICKEN"`` that the delimited ``JD`` was read as a nickname rather than a suffix. A reading the vocabulary settles on its own — ``"John Smith M.A."``, ``"Andrew Perkins (MBA)"`` — is not a guess and reports nothing
2828
- Add ``ParsedName`` output and comparison methods: ``render(spec)``, ``initials()``, ``capitalized()`` (which returns a new value rather than mutating in place), ``as_dict()``, ``replace(**fields)``, ``matches()`` and ``comparison_key()``
2929
- Add ``Locale``, the public pack type. Writing your own needs no registration -- construct a ``Locale`` and pass it to ``parser_for()``
3030
- Ship a fully typed public API (PEP 561): the core modules are checked under strict mypy settings, and nameparser 2.0 has no runtime dependencies

docs/usage.rst

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -318,6 +318,27 @@ so branching on a kind needs no import:
318318
>>> [t.text for t in name.ambiguities[0].tokens]
319319
['Van']
320320

321+
The post-nominals that double as ordinary surnames report the same way.
322+
``MA`` after a full name is read as a credential, but after a single
323+
given name it stays the surname — either way the choice is recorded:
324+
325+
.. doctest::
326+
327+
>>> parse("John Smith MA").suffix
328+
'MA'
329+
>>> [a.kind.value for a in parse("John Smith MA").ambiguities]
330+
['suffix-or-family']
331+
>>> parse("Jack MA").family
332+
'MA'
333+
334+
A reading the vocabulary settles on its own is not a guess and reports
335+
nothing — periods make ``M.A.`` unambiguously a credential:
336+
337+
.. doctest::
338+
339+
>>> parse("John Smith M.A.").ambiguities
340+
()
341+
321342
Most names report none. A non-empty ``ambiguities`` is a useful signal
322343
for routing a record to human review instead of trusting it silently.
323344

nameparser/_pipeline/_assign.py

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,9 @@ def _assign_main(seg_idx: int, state: ParseState,
124124
# every piece is a strict suffix (v1's are_suffixes tail rule, with
125125
# the roman-numeral special: a final roman numeral after a
126126
# non-initial piece is a suffix)
127+
# (piece, reading taken, reading declined) when the peel had to
128+
# resolve a bare ambiguous acronym; both directions are guesses
129+
ambiguous_pick: tuple[tuple[int, ...], str, str] | None = None
127130
k = len(rest)
128131
while k > 0:
129132
piece = pieces[rest[k - 1]]
@@ -145,10 +148,18 @@ def _assign_main(seg_idx: int, state: ParseState,
145148
# deliberately peels UNambiguous suffixes even when nothing is
146149
# left ("Smith PhD" -> suffix, a classified fix), because there
147150
# the vocabulary is not in doubt.
148-
if (k - 1 >= 2 and len(piece) == 1
149-
and "vocab:suffix-ambiguous" in tokens[piece[0]].tags):
151+
bare_ambiguous = (len(piece) == 1
152+
and "vocab:suffix-ambiguous" in tokens[piece[0]].tags)
153+
if bare_ambiguous and k - 1 >= 2:
154+
ambiguous_pick = (piece, "a suffix", "an ordinary surname")
150155
k -= 1
151156
continue
157+
if bare_ambiguous and k >= 2:
158+
# not peeled, so it stays the last name piece -- the family
159+
# name under every order. (k < 2 means it is the only piece
160+
# left and lands in the given position, which is not the
161+
# fork this reports.)
162+
ambiguous_pick = (piece, "a family name", "a post-nominal")
152163
break
153164
name_pieces, suffix_pieces = rest[:k], rest[k:]
154165
if not name_pieces and suffix_pieces:
@@ -159,6 +170,14 @@ def _assign_main(seg_idx: int, state: ParseState,
159170
_set_roles(tokens, pieces[piece_idx], roles[pos])
160171
for piece_idx in suffix_pieces:
161172
_set_roles(tokens, pieces[piece_idx], Role.SUFFIX)
173+
if ambiguous_pick is not None:
174+
piece, taken, declined = ambiguous_pick
175+
ambiguities.append(PendingAmbiguity(
176+
AmbiguityKind.SUFFIX_OR_FAMILY,
177+
f"{tokens[piece[0]].text!r} written without periods is both "
178+
f"a post-nominal and a surname; read as {taken} rather than "
179+
f"{declined}",
180+
tuple(piece)))
162181
# leading ambiguous particle read as a name (#121 surfaced)
163182
if name_pieces:
164183
head = pieces[name_pieces[0]]

nameparser/_pipeline/_classify.py

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,10 @@
1919
import dataclasses
2020

2121
from nameparser._lexicon import _normalize
22-
from nameparser._pipeline._state import ParseState, WorkToken
22+
from nameparser._pipeline._state import (
23+
ParseState, PendingAmbiguity, WorkToken,
24+
)
25+
from nameparser._types import AmbiguityKind, Role
2326
from nameparser._pipeline._vocab import (
2427
is_initial, period_joined_vocab, suffix_as_written,
2528
)
@@ -73,4 +76,20 @@ def classify(state: ParseState) -> ParseState:
7376
tokens = tuple(
7477
dataclasses.replace(t, tags=_tags_for(t, state))
7578
for t in state.tokens)
76-
return dataclasses.replace(state, tokens=tokens)
79+
# Delimited content whose vocabulary cannot settle it: extract's
80+
# escape sends an UNambiguous suffix straight through ("(MBA)" ->
81+
# suffix) and keeps everything else as a nickname, so an AMBIGUOUS
82+
# acronym in there was a coin the parser had to call. Reported here
83+
# rather than at the escape itself, which runs before tokenize and
84+
# so has no token index to point at.
85+
ambiguities = list(state.ambiguities)
86+
for i, token in enumerate(tokens):
87+
if (token.role is Role.NICKNAME
88+
and "vocab:suffix-ambiguous" in token.tags):
89+
ambiguities.append(PendingAmbiguity(
90+
AmbiguityKind.SUFFIX_OR_NICKNAME,
91+
f"delimited {token.text!r} is also a post-nominal; read "
92+
f"as a nickname rather than a suffix",
93+
(i,)))
94+
return dataclasses.replace(state, tokens=tokens,
95+
ambiguities=tuple(ambiguities))

nameparser/_types.py

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -231,9 +231,16 @@ class AmbiguityKind(StrEnum):
231231
#: two-word name under a non-default name_order). Not yet emitted;
232232
#: planned for 2.x.
233233
ORDER = "order"
234-
#: Reserved: a trailing word reads plausibly as either a suffix or
235-
#: a nickname. Not yet emitted; planned for 2.x.
234+
#: Delimited content is an ambiguous suffix acronym, so it reads
235+
#: plausibly as either a post-nominal or a nickname -- "JEFFREY
236+
#: (JD) BRICKEN" keeps the nickname reading, where the
237+
#: unambiguous "(MBA)" escapes to suffix on vocabulary alone.
236238
SUFFIX_OR_NICKNAME = "suffix-or-nickname"
239+
#: An ambiguous suffix acronym written without periods, which is
240+
#: also an ordinary surname -- "John Smith MA" reads MA as a
241+
#: post-nominal because a family name remains, "Jack MA" reads it
242+
#: as the family name because none would.
243+
SUFFIX_OR_FAMILY = "suffix-or-family"
237244
#: A leading ambiguous particle was read as a given name -- "Van
238245
#: Johnson" parses given="Van", but "Van" is also a family-name
239246
#: particle in other names.

tests/v2/cases.py

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -87,23 +87,27 @@ def __post_init__(self) -> None:
8787
"them first (spans cannot reorder)"),
8888
Case("ambiguous_surname_acronyms", "Jack MA",
8989
{"given": "Jack", "family": "MA"},
90+
ambiguities=("suffix-or-family",),
9091
notes="'ma'/'do' joined suffix_acronyms_ambiguous: with only "
9192
"two pieces, 'one of them is a credential' is the less "
9293
"likely reading, so the ambiguous acronym stays the "
9394
"family name (v1 parity via its reserve_last)"),
9495
Case("ambiguous_surname_acronym_with_suffix", "Jack MA Jr",
9596
{"given": "Jack", "family": "MA", "suffix": "Jr"},
97+
ambiguities=("suffix-or-family",),
9698
notes="'Jr' peels first; 'MA' would then be the only piece "
9799
"left beside the given name, so it stays family"),
98100
Case("ambiguous_acronym_is_a_suffix_when_a_family_name_remains",
99101
"John Smith MA",
100102
{"given": "John", "family": "Smith", "suffix": "MA"},
103+
ambiguities=("suffix-or-family",),
101104
notes="the other half of the same rule: three pieces means "
102105
"peeling 'MA' still leaves given+family, so the "
103106
"credential reading wins (v1 parity)"),
104107
Case("ambiguous_acronym_suffix_with_middle", "John Q Smith MA",
105108
{"given": "John", "middle": "Q", "family": "Smith",
106-
"suffix": "MA"}),
109+
"suffix": "MA"},
110+
ambiguities=("suffix-or-family",)),
107111
Case("initial_shaped_not_conjunction", "john e. smith",
108112
{"given": "john", "middle": "e.", "family": "smith"},
109113
notes="v1 is_conjunction excludes initials at classify too"),
@@ -399,11 +403,14 @@ def __post_init__(self) -> None:
399403
Case("whitespace", " ", {}),
400404
Case("bare_ambiguous_acronym", "John Ed",
401405
{"given": "John", "family": "Ed"},
402-
notes="'ed' is an ambiguous acronym; bare form is a name (C1)"),
406+
ambiguities=("suffix-or-family",),
407+
notes="'ed' is an ambiguous acronym; bare form is a name (C1), "
408+
"and the parse reports which reading it took"),
403409
Case("comma_ambiguous_acronym", "Smith, Ed",
404410
{"given": "Ed", "family": "Smith"}),
405411
Case("ambiguous_acronym_with_suffix", "John Ed III",
406-
{"given": "John", "family": "Ed", "suffix": "III"}),
412+
{"given": "John", "family": "Ed", "suffix": "III"},
413+
ambiguities=("suffix-or-family",)),
407414
Case("phd_split", "John Ph. D.",
408415
{"given": "John", "suffix": "Ph. D."},
409416
notes="v1 fix_phd; healed via the stable 'joined' tag"),

tests/v2/test_contracts.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,10 @@
1111
AmbiguityKind.PARTICLE_OR_GIVEN: "Van Johnson",
1212
AmbiguityKind.UNBALANCED_DELIMITER: 'Jon "Nick Smith',
1313
AmbiguityKind.COMMA_STRUCTURE: "Smith, John, Extra, Jr.",
14+
AmbiguityKind.SUFFIX_OR_NICKNAME: "JEFFREY (JD) BRICKEN",
15+
AmbiguityKind.SUFFIX_OR_FAMILY: "John Smith MA",
1416
# no emitter yet -- arrives with locale-pack order detection (2.x)
1517
AmbiguityKind.ORDER: None,
16-
# no emitter yet -- arrives with suffix/nickname refinement (2.x)
17-
AmbiguityKind.SUFFIX_OR_NICKNAME: None,
1818
}
1919

2020

tests/v2/test_parser.py

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,3 +170,44 @@ def test_phd_split_mid_name_is_a_suffix() -> None:
170170
assert pn.suffix == "Ph. D."
171171
assert pn.family == "Smith"
172172
assert pn.middle == ""
173+
174+
175+
def test_ambiguous_acronym_reports_the_reading_it_took() -> None:
176+
# 'ma' is both a post-nominal and a surname, so whichever way the
177+
# peel resolves it is a guess -- the same shape as the leading
178+
# ambiguous particle that already reports PARTICLE_OR_GIVEN
179+
took_suffix = parse("John Smith MA")
180+
assert took_suffix.suffix == "MA"
181+
assert [a.kind for a in took_suffix.ambiguities] == \
182+
[AmbiguityKind.SUFFIX_OR_FAMILY]
183+
assert [t.text for t in took_suffix.ambiguities[0].tokens] == ["MA"]
184+
185+
took_family = parse("Jack MA")
186+
assert took_family.family == "MA"
187+
assert [a.kind for a in took_family.ambiguities] == \
188+
[AmbiguityKind.SUFFIX_OR_FAMILY]
189+
190+
191+
@pytest.mark.parametrize("text", [
192+
"John Smith M.A.", # periods decide it; no guess
193+
"Ma, Jack", # a comma fixes the family
194+
"Joao da Silva do Amaral de Souza", # 'do' mid-name, never at the peel
195+
"John Smith PhD", # unambiguous vocabulary
196+
])
197+
def test_no_suffix_ambiguity_when_nothing_was_guessed(text: str) -> None:
198+
assert [a for a in parse(text).ambiguities
199+
if a.kind is AmbiguityKind.SUFFIX_OR_FAMILY] == []
200+
201+
202+
def test_delimited_ambiguous_acronym_reports_suffix_or_nickname() -> None:
203+
# inside delimiters the competing readings are suffix and nickname:
204+
# "(MBA)" is unambiguously a credential and escapes to suffix, while
205+
# "(JD)" could be either, so it keeps the nickname reading -- a
206+
# guess, and until now a silent one
207+
n = parse("JEFFREY (JD) BRICKEN")
208+
assert n.nickname == "JD"
209+
assert [a.kind for a in n.ambiguities] == \
210+
[AmbiguityKind.SUFFIX_OR_NICKNAME]
211+
assert [t.text for t in n.ambiguities[0].tokens] == ["JD"]
212+
# the unambiguous one decided on vocabulary, so it is not a guess
213+
assert parse("Andrew Perkins (MBA)").ambiguities == ()

0 commit comments

Comments
 (0)