SIGSEGV on parse of 5-byte input '{𱡀' (U+31860 after open brace) on Linux

[jdidion]

Repro

printf '{\xF0\xB1\xA1\x80' | xxd
# 00000000: 7bf0 b1a1 80                             {....

Five bytes: open brace { (0x7B) followed by the 4-byte UTF-8 encoding of U+31860 (CJK Unified Ideograph Extension G).

Parsing this input in Rust via tree-sitter + tree-sitter-bash = 0.25.1 on Linux x86_64 (Ubuntu-latest GitHub Actions) crashes the process with SIGSEGV (exit code 139). The same bytes parse cleanly on macOS aarch64 as a normal tree.root_node().has_error() == true state.

Minimal Rust reproducer

use tree_sitter::Parser;

fn main() {
    let mut p = Parser::new();
    p.set_language(&tree_sitter_bash::LANGUAGE.into()).unwrap();
    let input = "{\u{31860}";  // 5 bytes
    let _ = p.parse(input, None);
    println!("did not crash");
}

Output on Linux: process exits with signal 11 (SIGSEGV) before println! runs. Same crate versions (tree-sitter = 0.25, tree-sitter-bash = 0.25.1) on macOS succeed and print did not crash.

Discovery path

Found by property-based fuzzing of a classifier that wraps tree_sitter::Parser::parse. The fuzzer generated arbitrary UTF-8 inputs up to 2000 chars; ~1 in every 60 inputs tripped a Linux-only SIGSEGV. Byte-grained bisect of one captured 2863-byte crasher narrowed the triggering bytes to positions 2485-2489 — exactly { + U+31860. A separate classifier sweep confirmed:

Input	Linux result
{ + U+31860	SIGSEGV
{ + U+10000 (Gothic A)	clean error state
{ + U+1F600 (😀)	clean error state
{ + U+FFFD (BMP)	clean error state
{ + U+4E00 (CJK BMP)	clean error state
( + U+31860	clean error state
[ + U+31860	clean error state
" + U+31860	clean error state
+ U+31860	parses as word (Ok)
a + U+31860	parses as word (Ok)
U+31860 alone	parses as word (Ok)

So the crash is very narrow: specifically the pair { + U+31860 (adjacent, no separator). Not brace + any astral codepoint, not U+31860 in any context — both required.

Hypothesis

Given the specificity, my guess is a lookup into a generated parser table (or external scanner state) that overruns a bounds check only on this specific token pair. The input is 5 bytes; nothing upstream has had a chance to accumulate state. The macOS/Linux divergence is probably ASLR/memory layout luck — macOS happens to put a readable page where Linux has nothing mapped.

Barbican context

This is a bug in a classifier shipped as a safety layer for AI-generated shell commands; a SIGSEGV in the classifier is a crash in the safety floor. I've worked around it locally with a 10-line pre-flight scan that denies {<U+31860> before calling tree_sitter::Parser::parse, but that doesn't scale if there are other dangerous pairings in the parser table, so a proper upstream fix is the real resolution.

Happy to test a patched branch if useful.

[jdidion]

Update after deeper bisect: the crash is not a single-codepoint pathology, it's a whole-row pathology.

The captured reproducer's trailing byte was 0x80 (codepoint U+31840, not U+31860 as my original report said — my mistake; CJK Ext G glyphs render nearly identically and I mis-eyeballed the hex). Classifier-sweep probes with the same 3-byte prefix F0 B1 A1 but a different 4th byte (0xA0, i.e. U+31860) also SIGSEGV on Linux. Spot-checks across the row are consistent.

So the minimal-case shape is:

'{' + <any codepoint in U+31840..U+3187F>

That's the 64-codepoint range whose UTF-8 encoding starts with F0 B1 A1. Still specifically { as the preceding byte — paren/bracket/quote/space/letter+U+3184X all parse cleanly.

Working hypothesis (take with salt): the generated parser's compound-statement error-recovery path indexes a state table by the lead byte of the unknown token and doesn't bounds-check when that byte happens to land in a reserved region of the table. The whole CJK Ext G row F0 B1 A1 ?? seems to hit the same slot.

Our local mitigation (https://github.com/jdidion/barbican/pull/33) widened to the 3-byte-prefix match accordingly.

Happy to produce more data points if useful — the row boundary (does U+3187F/U+31880 crash? does U+3183F?) would be easy to confirm via the same forked-probe harness.

[jdidion]

Further update from continued fuzzing over the past ~10 days.

TL;DR

The pathology is block-level, not row-level. Whatever bad table index was being hit lines up with the top 2 bytes of the 4-byte UTF-8 encoding, not the top 3. Our preflight now denies the whole F0 B0 and F0 B1 lead pairs — 8192 codepoints wholesale — when byte-adjacent to {.

Evidence across captured crash classes

Each class below was captured in CI (Ubuntu-latest, tree-sitter 0.25 + tree-sitter-bash 0.25.1, forked classify-probe subprocess per proptest case). Minimal reproducer is always { + the astral codepoint shown.

Class	Codepoint	UTF-8 prefix (first 3 bytes)	Lead pair	CI context
1	U+31860	F0 B1 A1	F0 B1	Initial report, above
2	U+31BC3	F0 B1 AF	F0 B1	1.3.3 (Barbican PR #39)
3	U+31F88	F0 B1 BE	F0 B1	1.3.4 (Barbican PR #42)
5	U+316FF	F0 B1 9B	F0 B1	1.3.6 (Barbican PR #47)
6	U+30225	F0 B0 88	F0 B0	1.3.7 fuzz lane (Barbican #49)
7	U+314CD	F0 B1 93	F0 B1	1.3.8 fuzz lane (Barbican #50)

(Class 4 was a state-accumulation artifact specific to the in-process proptest harness; it didn't reproduce under subprocess isolation, so we stopped tracking it.)

The block-level bisect that promoted F0 B0 and F0 B1

For class 6 (first codepoint with lead byte 0xB0 — a completely different block from the 0xB1 row I reported originally), we ran a 10-probe forked-subprocess sweep over the block:

openbrace_plus_30000_ext_g_start   F0 B0 80 80   SIGSEGV
openbrace_plus_30100_row_above     F0 B0 84 80   SIGSEGV
openbrace_plus_301FF_just_below    F0 B0 87 BF   SIGSEGV
openbrace_plus_30200_row_start     F0 B0 88 80   SIGSEGV
openbrace_plus_30225_minimal       F0 B0 88 A5   SIGSEGV  (triggering capture)
openbrace_plus_3023F_row_end       F0 B0 88 BF   SIGSEGV
openbrace_plus_30240_just_above    F0 B0 89 80   SIGSEGV
openbrace_plus_30300_row_well_above F0 B0 8C 80  SIGSEGV
openbrace_plus_30FFF               F0 B0 BF BF   SIGSEGV
solo_30225_no_prefix (no `{`)      F0 B0 88 A5   parses cleanly (exit-0)

9 probes spanning 5 different rows in the F0 B0 lead pair, all SIGSEGV. The solo-codepoint (no { prefix) still parses cleanly, so the { adjacency remains the trigger.

For class 7: after we widened the preflight to deny all of F0 B0, the next CI run surfaced a crasher whose first { + astral pair was { + U+314CD (prefix F0 B1 93) — a row that was not in any of the 4 previously-pinned F0 B1 XX rows. With 5 non-adjacent rows under F0 B1 now confirmed crashing (U+314C0, U+316C0, U+31840, U+31BC0, U+31F80), the whole lead pair is suspect — extending the block-level mitigation to F0 B1 cleared the next bisect cycle.

Current mitigation

Barbican's preflight (https://github.com/jdidion/barbican/blob/main/crates/barbican/src/tables.rs) now uses a 2-byte lead-pair deny list covering F0 B0 and F0 B1 — 8192 codepoints. All of CJK Extensions G and H fall inside this range. We keep a 3-byte row-level table too but it's empty; if a future lead pair turns out to crash only specific rows, the narrower entries go there.

Open questions (happy to probe if useful)

• Does F0 B2 (U+32000..U+32FFF) crash too? We haven't surfaced a class in that range yet; our fuzz inputs just haven't landed there.
• Does F0 B3 (U+33000..U+33FFF)?
• Are the lower lead pairs F0 9F / F0 90 / F0 91 safe? The class-7 capture contained { + astral pairs in all three, but the presence-order suggests the F0 B1 one was the trigger; we haven't done a pointed bisect of those lower ranges yet.

Updated hypothesis

The generated parser's error-recovery path is indexing a state/transition table using (at least) the first two UTF-8 continuation bytes, not just the lead. The F0 B0..B1 range hits an uninitialized or out-of-bounds slot. The { prefix specifically is load-bearing — other prefix bytes ((, [, ", space, letter) all parse the same codepoints cleanly, which points at the brace-start state's transition table specifically.

Let me know if a patch direction would help, or if you'd like us to produce any specific additional data points.

[jdidion]

Final update. Two more findings that changed the mitigation shape again:

Class 8 — crash is NOT limited to CJK Ext G/H

After widening the preflight to cover F0 B0 + F0 B1 lead pairs (the two blocks we'd bisected), the next CI proptest run found a 3540-byte crasher whose first { + astral pair was:

char=84  U+1F8C1  (UTF-8: F0 9F A3 81)  — an emoji/symbols codepoint

The capture contained 10 { + astral pairs spanning SIX different UTF-8 lead pairs:

Lead pair	Codepoint examples
F0 9F	U+1F8C1, U+1F7F0, U+1F1FF (SMP emoji / symbols)
F0 9E	U+1E7EA, U+1EE39
F0 9D	U+1D72F, U+1D7C0 (mathematical alphanumeric)
F3 A0	U+E0144 (tag-space, supplementary special purpose)
F0 9B	U+1BC94 (Duployan)
F0 90	U+10102 (Aegean numbers)

All F0 9F / F0 9E / F0 9D / F0 9B / F0 90 land in the SMP, nowhere near CJK Extensions G/H. F3 A0 is in plane 14 (supplementary special-purpose). The upstream bug is not specific to any codepoint block — it's triggered by the 4-byte-UTF-8 continuation path generically.

Class 9 — trigger is NOT byte-adjacent to {

The classifier probe {5 + U+31F88 (6 bytes total) SIGSEGVs under classify-probe. Byte sequence:

7B 35 F0 B1 BE 88
`{` `5` U+31F88 (F0 B1 BE 88)

The { is followed by ASCII 5, then the 4-byte codepoint — { is NOT byte-adjacent to the astral codepoint. My original report claimed adjacency was required; that was wrong. The parser enters a broken state after any { and the broken state persists across intermediate ASCII bytes before landing at the 4-byte codepoint.

Current mitigation

Collapsed both tables. The check is now:

// If the input contains any `{` followed ANYWHERE later by a 4-byte
// UTF-8 lead byte (0xF0..=0xF7), deny.
let mut seen_openbrace = false;
for &b in bytes {
    if b == b'{' { seen_openbrace = true; continue; }
    if seen_openbrace && (0xF0..=0xF7).contains(&b) {
        return Err(ParseError::Malformed);
    }
}
Ok(())

Eight lines, no tables. Linear scan with a bool flag. Subsumes every prior preflight entry.

Verification

After this widening, the best-effort linux-fuzz-repro CI lane ran to completion across all 8192 proptest cases with zero crashes — the first time that's happened since the lane shipped in 1.3.0. Barbican 1.3.8 will ship with this mitigation.

Updated hypothesis

The generated parser's error-recovery path appears to enter an uninitialized state after the { token, and any subsequent 4-byte UTF-8 codepoint dereferences an OOB pointer in that state. The state is persistent across intermediate tokens until something resets it (a close-brace probably, but we haven't probed that directly).

This is genuinely a tree-sitter-bash-specific bug, not a tree-sitter core bug — tree-sitter itself parses these bytes cleanly for other grammars. Happy to provide a patch direction or help test a fix.

Reference

Mitigation landed in jdidion/barbican#50 (merging now), shipped in Barbican 1.3.8. Source: https://github.com/jdidion/barbican/blob/main/crates/barbican/src/parser.rs (preflight_known_crashers).