Merge pull request #290 from derek73/claude/v2-core-api-critique-91a960

Post-rc1 API polish: StrEnum Role, Parser conveniences, dead-vocabulary repair

Author: derek73

AGENTS.md

@@ -38,7 +38,9 @@ relatives); a position cannot be confused with a look-alike.
 assignment, ever. If a parse is almost right and you want to fix one
 field, you call ``.replace()``, which returns a new
 :class:`~nameparser.ParsedName` with that field changed and everything
-else — tokens, spans, the rest of the roles — carried over unchanged.
+else — tokens, spans, the rest of the roles — carried over unchanged
+(``replace()`` tokens carry no vocabulary tags; :meth:`Parser.revise
+<nameparser.Parser.revise>` is the tag-preserving form).
 ``str()`` renders the default view; nothing about calling it mutates
 the value you called it on.
 
@@ -148,10 +150,10 @@ Some calls are irreducibly ambiguous — both readings are legitimate,
 and no amount of rule-tuning resolves them without breaking some other
 name. Those surface as entries on ``ParsedName.ambiguities`` instead
 of being silently guessed away. The canonical example: a leading "Van"
-in "Van Johnson" reads as a given name (that's the common case for
-that shape), but "Van" is also a family-name particle in plenty of
-other names, so the parse records a ``particle-or-given`` ambiguity
-alongside its answer. You can inspect ``ambiguities`` to decide, case
+reads as a given name — the right call for the actor Van Johnson, the
+wrong one for a bare "Van Buren", and nothing in the two-word shape
+distinguishes them — so the parse records a ``particle-or-given``
+ambiguity alongside its answer. You can inspect ``ambiguities`` to decide, case
 by case, whether your data needs a second look.
 
 An ambiguity records a *decision*, not a word. The same token in a
@@ -174,7 +176,8 @@ signal to act on; do not read an empty one as a guarantee.
 :class:`Tokens <nameparser.Token>` also carry tags — a second, independent label alongside their
 role, recording how a token was classified rather than what part of
 the name it belongs to — but only a handful of them are part of the
-stable API: ``particle``, ``conjunction``, ``initial``, and ``joined``.
+stable API, collected in :data:`~nameparser.STABLE_TAGS`: ``particle``,
+``conjunction``, ``initial``, and ``joined``.
 Any tag written with a namespace prefix, like ``vocab:...``, is
 provenance information for debugging how a token got classified — it
 can change shape between releases and isn't something to match against

docs/concepts.rst

@@ -28,6 +28,12 @@ accepts a plain set of lowercase words, keyword by field name (``titles``
 above; ``particles``, ``suffix_words``, and the rest work the same
 way) — see :doc:`modules` for the full field list.
 
+Vocabulary entries are matched one word at a time (``given_name_titles``
+excepted), so a multi-word entry like ``titles={"grand moff"}`` can
+never match; the constructor warns when it sees one
+(``capitalization_exceptions`` keys included — they are looked up per
+word too).
+
 Removing works the same way, and drops the word from recognition:
 
 .. doctest::
@@ -260,6 +266,16 @@ listed below.
      - Excludes bidirectional control characters the same way.
        Defaults to ``True``.
 
+To apply a :class:`PolicyPatch <nameparser.PolicyPatch>` directly --
+without going through a locale pack -- call :meth:`Policy.patched()
+<nameparser.Policy.patched>`:
+
+.. doctest::
+
+    >>> from nameparser import Policy, PolicyPatch
+    >>> Policy().patched(PolicyPatch(middle_as_family=True))
+    Policy(middle_as_family=True)
+
 Family-first name order
 ~~~~~~~~~~~~~~~~~~~~~~~~
 

docs/customize.rst

@@ -149,7 +149,8 @@ You don't need to touch nameparser's registry to use your own pack —
 :class:`~nameparser.PolicyPatch` is a :class:`~nameparser.Policy`-shaped
 patch: every field defaults to :data:`~nameparser.UNSET` (leave it
 alone) instead of to a concrete value, so a pack only ever states what
-it changes.
+it changes. A patch can also be applied directly, without a pack —
+see :meth:`Policy.patched() <nameparser.Policy.patched>`.
 
 The ``policy`` half works that way, but the ``lexicon`` half does not.
 A pack's :class:`~nameparser.Lexicon` is a complete value in its own

docs/locales.rst

@@ -23,6 +23,19 @@ Results
 .. autoclass:: nameparser.Token
    :members:
 
+.. py:data:: nameparser.STABLE_TAGS
+   :value: frozenset({"particle", "conjunction", "initial", "joined"})
+
+   The four :attr:`Token.tags <nameparser.Token.tags>` values that are
+   stable API: ``particle`` (a word from the particle vocabulary,
+   "de"/"van", wherever it lands — combine with ``Role.FAMILY`` for
+   actual family particles),
+   ``conjunction`` (a joining word, "and"/"y"), ``initial`` (an
+   initial-shaped word, "J."), and ``joined`` (a continuation of the
+   previous token within one merged piece, so the suffix view renders
+   "Ph. D." as one credential). Every other tag is namespaced
+   (``vocab:...``) and unstable — never match against those.
+
 .. autoclass:: nameparser.Span
    :members:
 

docs/modules.rst

@@ -30,9 +30,13 @@ Release Log
     - Add ``name_order`` and the constants ``GIVEN_FIRST``, ``FAMILY_FIRST`` and ``FAMILY_FIRST_GIVEN_LAST``, so a family-first name can be parsed as written rather than reordered by hand (closes the configuration half of #270)
     - Add ``PatronymicRule`` with the members ``EAST_SLAVIC`` and ``TURKIC``. v1's single ``patronymic_name_order`` flag enabled both detectors at once; ``Policy(patronymic_rules=...)`` lets you enable either one alone
     - Add ``PolicyPatch`` and the ``UNSET`` sentinel for partial policy deltas that compose -- set-valued fields union, scalar fields override with later winning. This is the mechanism locale packs are built from, and ``UNSET`` is only needed when you must distinguish "not set" from a real ``False`` or ``None``
-    - Add ``Token``, ``Span`` and ``Role``: every field is backed by tokens carrying exact ``(start, end)`` offsets into the original string, reachable with ``tokens_for(Role.GIVEN)``. This replaces v1's ``*_list`` attributes and makes it possible to highlight or re-slice the input the parse came from
+    - Add ``Token``, ``Span`` and ``Role``: every field is backed by tokens carrying exact ``(start, end)`` offsets into the original string, reachable with ``tokens_for(Role.GIVEN)``. This replaces v1's ``*_list`` attributes and makes it possible to highlight or re-slice the input the parse came from. ``Role`` is a ``StrEnum``, so members compare and stringify as their field names (``Role.GIVEN == "given"``), matching ``AmbiguityKind``; ``tokens_for()`` accepts a role's string name too, and raises ``ValueError`` for an unknown role
+    - Add ``STABLE_TAGS`` to the public API: the four documented ``Token.tags`` values (``particle``, ``conjunction``, ``initial``, ``joined``)
+    - Add ``Policy.patched(patch)``, applying a ``PolicyPatch`` directly without wrapping it in a locale pack
+    - Add ``Parser.matches(a, b)`` and ``Parser.capitalized(name)``: the ``ParsedName`` methods of the same names fall back to the default configuration for str/omitted arguments, which is silently wrong for names parsed with a custom ``Parser``
+    - Add ``Parser.revise(name, **fields)``: ``ParsedName.replace()`` with the replacement text classified by the parser's vocabulary, so particle/initial/suffix-join behavior survives the edit
     - Add ``Ambiguity`` and the ``AmbiguityKind`` enum, so a parse reports what it had to guess at instead of guessing silently. The kinds emitted today are ``particle-or-given``, ``suffix-or-name``, ``suffix-or-nickname``, ``unbalanced-delimiter`` and ``comma-structure``; ``order`` is reserved and not yet emitted. The two suffix kinds cover the post-nominals that are also ordinary words: ``"John Smith MA"`` reports that ``MA`` was read as a credential rather than a surname, and ``"JEFFREY (JD) BRICKEN"`` that the delimited ``JD`` was read as a nickname rather than a suffix. A reading the vocabulary settles on its own — ``"John Smith M.A."``, ``"Andrew Perkins (MBA)"`` — is not a guess and reports nothing
-    - Add ``ParsedName`` output and comparison methods: ``render(spec)``, ``initials()``, ``capitalized()`` (which returns a new value rather than mutating in place), ``as_dict()``, ``replace(**fields)``, ``matches()`` and ``comparison_key()``
+    - Add ``ParsedName`` output and comparison methods: ``render(spec)``, ``initials()``, ``capitalized()`` (which returns a new value rather than mutating in place), ``as_dict()`` (whose ``include_empty`` flag is keyword-only, unlike ``HumanName.as_dict()``'s), ``replace(**fields)``, ``matches()`` and ``comparison_key()``
     - Add ``Locale``, the public pack type. Writing your own needs no registration -- construct a ``Locale`` and pass it to ``parser_for()``
     - Ship a fully typed public API (PEP 561): the core modules are checked under strict mypy settings, and nameparser 2.0 has no runtime dependencies
 
@@ -60,6 +64,9 @@ Release Log
     - Fix the pre-comma piece being routed to ``first`` when everything after the comma is a suffix or title: ``"Andrews, M.D."`` now reads family ``Andrews`` / suffix ``M.D.`` where 1.x read given ``M.D.`` / family ``Andrews``, and ``"Smith, Dr."`` moves ``Smith`` from ``first`` to ``family`` (the title was already correct in 1.x). The piece before a comma is definitionally the family name
     - Fix a lone recognized trailing suffix with no comma being routed to ``first``/``last``: ``"Johnson PhD"`` and ``"Mr. Johnson PhD"`` now keep the suffix in ``suffix``
     - Fix a split ``"Ph. D."`` credential being read as two tokens; it now classifies as one suffix, replacing v1's ``fix_phd`` hook. This now holds wherever the credential sits: 1.x healed the pair only when it trailed, so ``"Ph. D. John Smith"`` parsed as title ``Ph.`` / given ``D.`` with the real given name pushed to ``middle``; it now reads given ``John``, family ``Smith``, suffix ``Ph. D.``
+    - Fix ``chargé d'affaires``: shipped as one unmatchable ``TITLES`` entry since it was added, it is now two chainable entries (``chargé``, ``d'affaires``), so the title is recognized; the unaccented spelling ``charge`` ships too, like ``attaché``/``attache``
+    - Remove seven multi-word ``SUFFIX_ACRONYMS`` entries (``leed ap``, ``nicet i``–``nicet iv``, ``psm i``, ``psm ii``) that could never match in any release; splitting them would swallow real names ("John Leed", "Smith, A.P."), so they are dropped instead
+    - Add a ``UserWarning`` when a multi-word entry is stored in a per-word ``Lexicon`` field or as a ``capitalization_exceptions`` key -- such entries can never match
     - Parse an input with no alphanumeric character to an empty name in both APIs. 1.x kept pure punctuation as a name part, so ``"."`` gave ``first="."`` and ``bool()`` was ``True``; 2.0 empties it, keeping ``bool(parse(x))`` an honest "did I get a name?" test. The check is Unicode-aware, so names in any script are unaffected; only inputs that are entirely punctuation or symbols (``"."``, ``"- -"``) change. Junk embedded in a name with real content -- the stray dot in ``"John . Smith"`` -- is still kept, since that parse is already truthy
     - Fold a leading never-given particle into the family name. Note that ``Lexicon.particles_ambiguous`` is the **complement** of v1's ``non_first_name_prefixes``, not a rename -- it lists the particles that *may* double as a given name, where v1 listed the ones that may not. Copying a v1 customization across without inverting it silently reverses the behavior; see :doc:`migrate`
     - Add ``ma`` and ``do`` to ``suffix_acronyms_ambiguous``, the set of post-nominals that are also ordinary surnames. An entry there is read as a credential when the name can spare it — written with periods (``"John Smith M.A."``), or when removing it still leaves a given *and* a family name (``"John Smith MA"`` → suffix ``MA``). With only two pieces to go around, the surname reading wins instead: ``"Jack Ma"`` and ``"Anh Do"`` keep their family names. As a side effect, a parenthesized or quoted ``"(MA)"``/``"(DO)"`` now falls through to nickname parsing rather than escaping to ``suffix``, since inside delimiters the nickname reading is the plausible one
@@ -81,6 +88,7 @@ Release Log
 
     - Reimplement ``HumanName`` as a facade over the 2.0 pipeline, and ``Constants`` as a shim resolving to a ``(Lexicon, Policy)`` snapshot with a shared parser cache. Fields, aggregates, mutation through ``name.C.titles.add(...)``, rendering defaults, ``capitalize()``, ``matches()``, ``comparison_key()``, iteration, ``as_dict()`` and pickling are all preserved, and ``nameparser.parser`` and ``nameparser.config`` remain importable. The compatibility layer ships through 2.x and is removed in 3.0
     - Note that ``CONSTANTS.capitalize_name`` and ``force_mixed_case_capitalization`` are still honored through the facade, but the 2.0 API never capitalizes during ``parse()`` -- call ``capitalized()`` when you want it
+    - Add ``Role`` members (and their string values ``given``/``family``) as valid ``HumanName`` subscript keys: ``hn[Role.GIVEN]`` returns ``hn.first``
 
     **Command line**
 
@@ -98,6 +106,16 @@ Release Log
     tooling, not part of the installed package) -- see its README to
     reproduce the comparison against your own names.
 
+    **Changed since 2.0.0rc1** (for anyone who tested the release candidate)
+
+    - ``Role`` became a ``StrEnum``; ``str(Role.GIVEN)`` is now ``"given"``
+    - ``ParsedName.tokens_for()`` raises ``ValueError`` for unknown roles instead of returning no tokens; it also accepts role-name strings
+    - ``ParsedName.as_dict()``'s ``include_empty`` is keyword-only
+    - ``HumanName`` subscripting accepts ``Role`` members
+    - The eight multi-word vocabulary entries that could never match were repaired (``chargé d'affaires`` split; seven credential acronyms removed), and storing a new multi-word entry now warns; those eight are dropped silently, not warned about, when a restored ``Constants`` pickle carries all eight of them (the pre-2.0 signature)
+    - ``PolicyPatch``'s repr shows only the fields a patch sets, instead of all nine with UNSET sentinels
+    - New since rc1: ``STABLE_TAGS``, ``Policy.patched()``, ``Parser.matches()``, ``Parser.capitalized()``, and ``Parser.revise()`` -- see the API section above
+
 * 1.4.0 - July 12, 2026
 
     - Add ``Constants.copy()``, a detached deep copy that preserves the source instance's current customizations (unlike ``Constants()``, which always starts from library defaults) -- useful as ``CONSTANTS.copy()`` for a private snapshot of the shared config (#260)

docs/release_log.rst

@@ -188,6 +188,11 @@ Fixing case
     >>> str(parse("juan de la vega").capitalized())
     'Juan de la Vega'
 
+The no-argument form above uses the DEFAULT lexicon; for a name parsed
+with a custom :class:`~nameparser.Parser`, call
+:meth:`Parser.capitalized() <nameparser.Parser.capitalized>` so the
+parser's own vocabulary decides the exceptions.
+
 Nicknames and maiden names
 ----------------------------
 
@@ -297,7 +302,10 @@ Comparing names
 ``==`` is strict value equality — two :class:`~nameparser.ParsedName`
 instances are equal only if every field matches exactly. For "is this
 the same name, allowing for order and case?" use ``matches()`` or
-``comparison_key()`` instead.
+``comparison_key()`` instead. ``matches()`` parses a str argument with
+the DEFAULT parser; to compare against a string using a custom
+:class:`~nameparser.Parser`'s vocabulary, call
+:meth:`Parser.matches() <nameparser.Parser.matches>`.
 
 .. doctest::
 
@@ -309,16 +317,18 @@ the same name, allowing for order and case?" use ``matches()`` or
 When the parser had to guess
 -----------------------------
 
-Some names have no single correct reading. ``"Van Johnson"`` could be
-the given name ``Van``, or the family-name particle ``van``. 2.0 takes
-the more likely reading and *records* the choice on ``ambiguities``
-rather than deciding silently:
+Some names have no single correct reading. A leading ``Van`` could be
+a given name — it really is for the actor Van Johnson — or the start
+of a family name, as it is for President Van Buren. Both are the same
+shape, so no rule can tell them apart. 2.0 takes the more likely
+reading and *records* the choice on ``ambiguities`` rather than
+deciding silently:
 
 .. doctest::
 
-    >>> name = parse("Van Johnson")
+    >>> name = parse("Van Buren")
     >>> name.given, name.family
-    ('Van', 'Johnson')
+    ('Van', 'Buren')
     >>> for a in name.ambiguities:
     ...     print(a.kind.value, "-", a.detail)
     particle-or-given - leading 'Van' may be a family-name particle; read as a given name
@@ -385,12 +395,17 @@ so you can always get back to the text a field came from:
     >>> [t.text for t in name.tokens_for(Role.FAMILY)]
     ['de', 'la', 'Vega']
 
+A role's string name works too: ``name.tokens_for("family")``.
+
 Correcting a parse
 --------------------
 
 :class:`~nameparser.ParsedName` is immutable, so a correction is a new
-value: ``replace()`` returns a copy with the given fields changed and
-everything else carried over.
+value: ``replace()`` returns a copy with the given fields changed.
+Untouched fields keep their tokens (and ``original`` is preserved),
+with one deliberate exception: an ambiguity that pointed into a
+replaced field is dropped — correcting the field that was flagged
+clears the flag, while correcting an unrelated field keeps it.
 
 .. doctest::
 
@@ -400,6 +415,50 @@ everything else carried over.
     'Dr. Juan de la Vega'
     >>> name.title
     ''
+    >>> flagged = parse("Van Buren")
+    >>> flagged.replace(given="Martin").ambiguities
+    ()
+    >>> [a.kind.value for a in flagged.replace(family="Harrison").ambiguities]
+    ['particle-or-given']
+
+``replace()`` splits values on whitespace into plain, untagged
+tokens — the vocabulary knowledge a parse would have about the new
+text is not there. The views that depend on tags degrade: the parser
+no longer knows ``de la`` are particles, so ``family_particles``
+empties and the particles start contributing initials.
+
+.. doctest::
+
+    >>> name.family_particles
+    'de la'
+    >>> replaced = name.replace(family="de la Vega Smith")
+    >>> replaced.family_particles
+    ''
+    >>> replaced.initials()
+    'J. d. l. V. S.'
+
+:meth:`Parser.revise() <nameparser.Parser.revise>` is the same
+operation with each value classified by the parser's vocabulary, so
+the correction behaves like a fresh parse of the corrected name
+(which also means delimiters and marker words in the value are
+consumed as they would be in a parse):
+
+.. doctest::
+
+    >>> from nameparser import Parser
+    >>> parser = Parser()
+    >>> revised = parser.revise(name, family="de la Vega Smith")
+    >>> revised.family_particles
+    'de la'
+    >>> revised.initials()
+    'J. V. S.'
+
+``revise()`` has two siblings on :class:`~nameparser.Parser`:
+:meth:`Parser.matches() <nameparser.Parser.matches>` and
+:meth:`Parser.capitalized() <nameparser.Parser.capitalized>`. Those
+two matter when you have built a custom parser — the
+:class:`~nameparser.ParsedName` methods of the same names fall back
+to the *default* configuration for str or omitted arguments.
 
 Command line
 ------------

docs/usage.rst

@@ -20,6 +20,7 @@
     PolicyPatch,
 )
 from nameparser._types import (
+    STABLE_TAGS,
     Ambiguity,
     AmbiguityKind,
     ParsedName,
@@ -33,6 +34,7 @@
     "HumanName",
     # v2 core
     "Span", "Role", "Token", "Ambiguity", "AmbiguityKind", "ParsedName",
+    "STABLE_TAGS",
     "Lexicon", "Policy", "PolicyPatch", "PatronymicRule", "UNSET",
     "GIVEN_FIRST", "FAMILY_FIRST", "FAMILY_FIRST_GIVEN_LAST",
     "DEFAULT_NICKNAME_DELIMITERS", "Locale",