fix(fuzz): drop arithmetic_fuzz inputs that contain banned debug shapes#1622
Merged
Conversation
The nightly fuzz job on main is red. `arithmetic_fuzz` inlines fuzz input
directly into `echo $(({input}))`. With unbalanced parens the arithmetic
expansion closes early and the remainder is parsed as commands; bash then
echoes the unknown command verbatim ("bash: /.rustup/toolchains/gww: No
such file or directory"). That is real-shell stderr, not a TM-INF-022
leak.
Last run: input bytes contained `/.rustup/toolchains/` literally and the
fuzz harness's leak detector tripped on the banned host-path shape.
Fix: at the fuzz-input layer, drop inputs that already contain any
`UNIVERSAL_BANNED` substring. This keeps TM-INF-022 detection strict on
real builtin internals while removing the false-positive class. Mirrors
the same fix applied to `glob_fuzz` in #1621.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
bashkit | 1fad283 | Commit Preview URL Branch Preview URL |
May 10 2026, 09:07 AM |
7 tasks
chaliy
added a commit
that referenced
this pull request
May 10, 2026
…pe check (#1623) ## Summary When a fuzz/proptest target inlines arbitrary input bytes into a shell script, bash and ls produce error messages that quote the input verbatim: ``` bash: <cmd>: command not found bash: <path>: No such file or directory ls: cannot access '<path>': No such file or directory ``` These echoes can accidentally form a banned substring even when no internal Debug formatter ran. Two real cases: - arithmetic_fuzz / glob_fuzz: input contained `/.rustup/toolchains/` literally; bash echoed it back. Filtered at the input layer in #1621 and #1622. - **glob_fuzz (new, dispatched run on this branch):** input ended with `Tok"`. Bash treated `Tok:` as a command and rendered `bash: Tok:: command not found` — the trailing `:` chrome from bash's error format glued onto the input's final `:` to form the banned parser-token shape `Tok::`. Pre-filtering the input for `Tok::` doesn't catch this — the substring only forms after bash's formatter runs. ## Fix Structural fix in `assert_fuzz_invariants`: strip lines that match a recognized real-shell error template before running the banned-shape check. - Byte-length cap (`MAX_STDERR_BYTES`) still runs on the **unfiltered** stderr — flood regressions still caught. - Host-canary check (TM-INF-013) still runs on the unfiltered stderr — env-leak regressions still caught. - The strict `assert_no_leak` path (used by per-builtin tests) is unchanged — non-fuzz tests must not produce shell echoes in the first place. Recognized templates are conservative: prefix `bash: ` or `ls: ` plus a known suffix. Lines that look similar but don't match the exact template stay in stderr, so real Debug leaks that happen to coexist with shell errors still trip the assertion. ## Test plan - [x] 8 unit tests in `testing::tests` cover both directions — strip known shell echoes, keep internal panic/Debug lines, keep partial matches and lines from other tools. - [x] `cargo test -p bashkit --lib testing::` green - [x] `cargo test -p bashkit --test proptest_security --all-features` green (18 cases) - [x] `cargo clippy -p bashkit --lib --tests -- -D warnings` clean - [x] `cargo fmt --check` clean - [x] Threat-model TM-INF-022 section updated to document the carve-out - [ ] Manually dispatch `Fuzz Testing` on this branch and confirm `glob_fuzz` job is green
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The nightly fuzz job on main is red.
arithmetic_fuzzinlines fuzz input directly intoecho \$(({input})). With unbalanced parens the arithmetic expansion closes early and the remainder is parsed as commands; bash then echoes the unknown command verbatim (bash: /.rustup/toolchains/gww: No such file or directory). That is real-shell stderr, not a TM-INF-022 leak.Last run: input bytes contained
/.rustup/toolchains/literally and the fuzz harness's leak detector tripped on the banned host-path shape — but no internal Debug formatter ran.Fix: at the fuzz-input layer, drop inputs that already contain any
UNIVERSAL_BANNEDsubstring. Mirrors #1621 forglob_fuzz.Test plan
cargo checkon the fuzz crate compiles cleanlyFuzz Testingon this branch and confirmarithmetic_fuzzjob is green