fix(fuzz): drop glob_fuzz inputs that contain banned debug shapes#1621
Merged
Conversation
Inputs are inlined into ls/case/[[ ]] commands; an unmatched path is
echoed verbatim by the shell, so a user-supplied 'Span {' tripped
TM-INF-022 even though no internal Debug formatter ran. Filter at the
fuzz-input layer keeps the leak detector strict on real internals.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
bashkit | 40019de | Commit Preview URL Branch Preview URL |
May 09 2026, 09:08 AM |
This was referenced May 10, 2026
chaliy
added a commit
that referenced
this pull request
May 10, 2026
…pe check (#1623) ## Summary When a fuzz/proptest target inlines arbitrary input bytes into a shell script, bash and ls produce error messages that quote the input verbatim: ``` bash: <cmd>: command not found bash: <path>: No such file or directory ls: cannot access '<path>': No such file or directory ``` These echoes can accidentally form a banned substring even when no internal Debug formatter ran. Two real cases: - arithmetic_fuzz / glob_fuzz: input contained `/.rustup/toolchains/` literally; bash echoed it back. Filtered at the input layer in #1621 and #1622. - **glob_fuzz (new, dispatched run on this branch):** input ended with `Tok"`. Bash treated `Tok:` as a command and rendered `bash: Tok:: command not found` — the trailing `:` chrome from bash's error format glued onto the input's final `:` to form the banned parser-token shape `Tok::`. Pre-filtering the input for `Tok::` doesn't catch this — the substring only forms after bash's formatter runs. ## Fix Structural fix in `assert_fuzz_invariants`: strip lines that match a recognized real-shell error template before running the banned-shape check. - Byte-length cap (`MAX_STDERR_BYTES`) still runs on the **unfiltered** stderr — flood regressions still caught. - Host-canary check (TM-INF-013) still runs on the unfiltered stderr — env-leak regressions still caught. - The strict `assert_no_leak` path (used by per-builtin tests) is unchanged — non-fuzz tests must not produce shell echoes in the first place. Recognized templates are conservative: prefix `bash: ` or `ls: ` plus a known suffix. Lines that look similar but don't match the exact template stay in stderr, so real Debug leaks that happen to coexist with shell errors still trip the assertion. ## Test plan - [x] 8 unit tests in `testing::tests` cover both directions — strip known shell echoes, keep internal panic/Debug lines, keep partial matches and lines from other tools. - [x] `cargo test -p bashkit --lib testing::` green - [x] `cargo test -p bashkit --test proptest_security --all-features` green (18 cases) - [x] `cargo clippy -p bashkit --lib --tests -- -D warnings` clean - [x] `cargo fmt --check` clean - [x] Threat-model TM-INF-022 section updated to document the carve-out - [ ] Manually dispatch `Fuzz Testing` on this branch and confirm `glob_fuzz` job is green
yonas
pushed a commit
to yonasBSD/bashkit
that referenced
this pull request
May 11, 2026
…es (everruns#1622) ## Summary The nightly fuzz job on main is red. `arithmetic_fuzz` inlines fuzz input directly into `echo \$(({input}))`. With unbalanced parens the arithmetic expansion closes early and the remainder is parsed as commands; bash then echoes the unknown command verbatim (`bash: /.rustup/toolchains/gww: No such file or directory`). That is real-shell stderr, not a TM-INF-022 leak. Last run: input bytes contained `/.rustup/toolchains/` literally and the fuzz harness's leak detector tripped on the banned host-path shape — but no internal Debug formatter ran. Fix: at the fuzz-input layer, drop inputs that already contain any `UNIVERSAL_BANNED` substring. Mirrors everruns#1621 for `glob_fuzz`. ## Test plan - [x] `cargo check` on the fuzz crate compiles cleanly - [ ] Manually dispatch `Fuzz Testing` on this branch and confirm `arithmetic_fuzz` job is green
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The nightly fuzz job on main is red.
glob_fuzzinlines fuzz input directlyinto
ls /tmp/{input},case ... in {input}), and[[ ... == {input} ]],so any unmatched path is echoed verbatim by the shell/
lsin stderr.Last run: input
==(Span {(;producedls: cannot access '/tmp/==(Span {(;',which contains the banned
Span {shape — but no internal Debug formatter ran,this is real-shell-style stderr.
Fix: at the fuzz-input layer, drop inputs that already contain any
UNIVERSAL_BANNEDsubstring. This keeps TM-INF-022 detection strict on realbuiltin internals while removing the false-positive class.
Test plan
cargo checkon the fuzz crate compiles cleanlyFuzz Testingon this branch and confirmglob_fuzzjob is greenGenerated by Claude Code