perf(search): the grapheme-snap fast path skips Cyrillic and CJK — every boundary reaches the segmenter

[IgorShevchik]

Background

Follow-up from the review of #371. The fix there snaps highlight and truncation boundaries to grapheme clusters via Intl.Segmenter, guarded by a cheap screen so ordinary text never pays for segmentation.

The screen is a threshold:

// src/runtime/utils/search.ts
const CLUSTER_CONTINUATION_FLOOR = 0x0300

Nothing below U+0300 can continue a cluster, so a boundary between two such characters is a cluster boundary by construction and the segmenter is skipped.

Problem

Cyrillic is U+0430. So is every other non-Latin script — CJK, Devanagari, Hebrew, Greek. All of them sit above the floor, so the screen never fires and every boundary goes through Segments.containing().

For a product localised into Russian, the screen therefore covers markup and Latin identifiers, not the body text users actually search.

Measured at 979 characters, against the pre-fix cost (isolated processes, median of nine):

content	cost
ASCII	+1%
Cyrillic	+2.9 µs
CJK	+2.5 µs

Not a regression anyone will notice at resultLimit: 12 — roughly 0.1 ms per keystroke — but it is the one case the optimisation was written for and does not serve. test/bench/search.bench.ts has a cyrillic case pinning it.

Direction

Replace the threshold with a test for what a character can actually do: a boundary is safe unless the character at it can continue a cluster (\p{M}, ZWJ, variation selectors, emoji modifiers, regional indicators, a low surrogate) or the one before it is Prepend, ZWJ, or CR before LF.

A prototype of that screen measured 0.072 µs against 2.69 µs for the segmenter — a 37× saving on Cyrillic — and gave zero unsound answers over 81 boundaries across 18 fixtures (Cyrillic, Latin with diacritics, CJK, Hangul, Devanagari, flags, ZWJ families, skin-tone modifiers, keycaps, CRLF, Arabic prepend).

Why this is filed rather than done

81 boundaries is far too thin to justify a soundness claim. The screen must never answer "safe" where UAX #29 says there is no boundary; one false "safe" silently reintroduces #364, and the failure mode is a plausible-looking wrong character that nobody reports.

Anyone taking this on should first build an exhaustive differential harness against Intl.Segmenter — every code unit pair in the affected ranges, plus randomised multi-code-point sequences — and treat a single disagreement as disqualifying. The prototype above is a starting point, not a candidate.

Alternative worth considering first

Cache the segmenter view across the three highlight() calls a single item makes (label, suffix, description operate on different values, but a group's items are processed together). Cheaper to get right than a hand-rolled Unicode predicate, though it saves less.

Note

src/runtime/utils/search.ts is a ported file — see the .sync/PORTING.md §2 invariant before touching it.

[IgorShevchik]

The premise here is right, but "raise the floor" is not available as a fix, and it is worth recording why before someone tries it.

The floor is already maximal

Brute-forced the whole space — for every code point in U+0001–U+10FFFF, does it join a preceding letter under Intl.Segmenter:

minimum code point that can continue a cluster:  U+0300
CLUSTER_CONTINUATION_FLOOR:                      U+0300

Confirmed independently against Unicode 17.0.0's GraphemeBreakProperty.txt: below U+0300 the only assigned break values are CR, LF and Control. The first Extend entry in the entire file is 0300..036F; Prepend starts at 0600, SpacingMark at 0903, ZWJ is the lone 200D, Hangul jamo start at 1100, regional indicators at 1F1E6.

So U+0300 is not a conservative round number chosen for comfort. It is the largest value a single scalar < FLOOR comparison can use and still be correct — raise it by one and the screen starts admitting U+0300 itself, a real combining mark, as if it were safe.

What that means for the issue

The complaint stands: for a product localised into Russian, the screen buys the body text nothing. The instant either neighbour is Cyrillic (≥ U+0400) or CJK, the gate current < FLOOR && previous < FLOOR fails and every boundary goes to the segmenter.

But the remedy is not a bigger constant. It would have to be a structurally different check — a small table of disjoint safe ranges, or a real property lookup — because the safe set is not an interval. Cyrillic is safe except for the combining marks interleaved through it, and the same is true of every other block above the floor.

That is a materially bigger change than the issue currently implies, and it needs its own measurement: the current screen costs one comparison and skips the segmenter for ASCII and Latin-1. A range table costs more per boundary and would have to earn that back on Cyrillic-heavy input specifically.

Not constrained by #412

For anyone reading these together: #412 adds CR/LF fixtures below the floor. They pin the carve-out, not the floor's height, and would pass identically wherever the floor sat — CR and LF are far below any plausible value.

Found while reviewing #412; recorded here rather than acted on, since it changes what this issue is asking for rather than answering it.

---

Generated by Claude Code