v1: rewrite glob for a much simpler and _correct_ engine - #73
Open
gobwas wants to merge 9 commits into
Open
Conversation
Replace the tree-of-specialized-matchers design (compiler/, match/,
syntax/ast/, util/ -- ~6000 lines) with a compact backtracking walk over
a small matcher tree, inspired by Russ Cox's research.swtch.com/glob:
- stars save restart points on a LIFO stack; a separator-free star
discards the restart points born inside its own alternative (the
Russ Cox single-restart idea, scoped by alternatives boundaries);
- alternatives save checkpoints addressed by tree path; resume walks
re-enter the tree directly through the saved path;
- terminal sub-sequences specialize into O(1) shaped matchers
(prefix/suffix/prefix_suffix/contains), patterns without checkpoints
skip the backtracking state entirely, and stateful patterns are
pre-filtered by min-length/required-suffix checks;
- match state is pooled; checkpoint paths come from a bulk-freed
arena: matching does not allocate.
Compile is hardened: malformed patterns (unclosed braces/classes,
invalid UTF-8, embedded NUL) return *SyntaxError with an offset instead
of hanging or panicking, and single-alternative braces `{ab*}` now mean
the pattern `ab*` (fuzzer caught a bug there, see below).
Correctness is differentially fuzzed against an independent glob->regexp
translator (FuzzMatchRegexp), which validates the lexer, the parser, the
rewrites and the engine at once; the corpus pins two bugs it found.
Benchmarks vs v0.2.3: compile 25-40% faster with half the allocations;
literal/prefix/suffix/wildcard matching at parity, `a*a*a*a*b` faster;
alternative-heavy patterns within 2-7x; zero allocations per Match.
Explored-but-unwired optimizations are parked with their findings in
expand.go and factor.go.
Fixes: #33, #34, #36, #41, #43, #48, #50, #51, #54, #56, #57, #59, #61, #62, #66
Move the engine and the matchers to match.go, Pattern and SyntaxError to glob.go; document the remaining exported symbols. Align the class grammar in Compile/readme with the code. Fixes: a valid U+FFFD was taken for an invalid byte both by the lexer (`[�]` did not compile) and by the class matchers (`[!a]` did not match it); formatRunes sorted a matcher's Sep in place; a trailing backslash is now a SyntaxError instead of being dropped. Remove the unused frame.String, syntax.Char and internal/debug/writer.go.
The regexps in BenchmarkCompareGlobAndRegexp now carry the `s` flag and escaped dots, so they are the exact equivalents of the globs; refresh both tables from a single run. Explain that `**` is not the shells' globstar and add an Escaping section (backslashes in Go string literals). Fixes: #21, #35, #38, #58, #65
This was referenced Aug 17, 2026
Open
Owner
Author
|
Hey folks, since you have sent a fix or reported an issue against this library over the time, I'm kindly asking you to review (at least partially) #73 -- a full rewrite of the glob matching engine (towards much simpler design and code!). The commit messages and the PR description have the details. If you have a moment, the most useful things to check are: does it fix what you hit (the "Closes" list), and does anything in your usage break. Reviews via the PR, or just a comment, both welcome. @calmh @erikgeiser @amitmishra11 @Yanhu007 @HNO3Miracle @jamiecobbett @LukeShu @torkel-hoge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR replaces the tree-of-specialized-matchers design worth of 6K lines with a compact backtracking walk over a
small matcher tree, inspired by research.swtch.com/glob.
Closes #21, #33, #34, #35, #36, #38, #39, #41, #43, #48, #49, #50, #51,
#54, #56, #57, #58, #59, #60, #61, #62, #65, #66.
Superseded PRs: #30, #31, #40, #55, #64, #68, #69, #70, #71.