Commit e6a6b69
committed
Correct five accuracy nits around the peel (#308)
- _script_segment's module header Reads: line omitted the suffix
vocabulary the stage now reads through _is_post_nominal ->
_vocab.is_suffix_strict; _segment.py sets the convention for naming
it. Produces: likewise omitted the peel's _PEELED_TAG.
- The same docstring's gate list ("gated by the FAMILY comma and by
间隔号 but NOT by segment_scripts") read as exhaustive and omitted
the ASCII bail -- the gate that actually decides a caller-configured
LATIN tail. The bail's own comment had it right; the header had not
caught up.
- config/suffixes.py said "Two more are in NEITHER set" and then named
three strings (君, 선생, 교수). The list above it counts entries.
- AGENTS.md's shim-transformation roster was one short: maiden_delimiters
minus nickname_delimiters carries the same v1-reachability comment
and is pinned by test_snapshot_overlap_keeps_v1_nickname_precedence.
Verified: with parens in both v1 buckets, 1.4.0 and HEAD both parse
nickname, while Policy alone gives maiden. It sits on the Policy half
of _snapshot(), which is why a Lexicon-only sweep missed it.
- test_properties.py's "neither derivation guarantees a fire" reads as
"sometimes fires". Measured over the 250 examples: peel 2, surname
half 0. Give the number and the structural reason -- a non-hangul
surname makes `w + "민준"` a mixed-script token whose
effective_script is None, so it is never an activated site; the one
hangul token drawn ('남궁민준') landed in an example whose policy had
HANGUL out of segment_scripts.1 parent b8a9252 commit e6a6b69
4 files changed
Lines changed: 31 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
134 | | - | |
| 134 | + | |
135 | 135 | | |
136 | 136 | | |
137 | 137 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
11 | 12 | | |
12 | | - | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
13 | 16 | | |
14 | 17 | | |
15 | 18 | | |
| |||
25 | 28 | | |
26 | 29 | | |
27 | 30 | | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
33 | 40 | | |
34 | 41 | | |
35 | 42 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
145 | 145 | | |
146 | 146 | | |
147 | 147 | | |
148 | | - | |
| 148 | + | |
149 | 149 | | |
150 | 150 | | |
151 | 151 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
332 | 332 | | |
333 | 333 | | |
334 | 334 | | |
335 | | - | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
336 | 346 | | |
337 | 347 | | |
338 | 348 | | |
| |||
0 commit comments