Skip to content

fix(fuzz): strip real-shell error lines from stderr before banned-shape check#1623

Merged
chaliy merged 1 commit into
mainfrom
claude/fuzz-strip-real-shell-errors
May 10, 2026
Merged

fix(fuzz): strip real-shell error lines from stderr before banned-shape check#1623
chaliy merged 1 commit into
mainfrom
claude/fuzz-strip-real-shell-errors

Conversation

@chaliy
Copy link
Copy Markdown
Contributor

@chaliy chaliy commented May 10, 2026

Summary

When a fuzz/proptest target inlines arbitrary input bytes into a shell script, bash and ls produce error messages that quote the input verbatim:

bash: <cmd>: command not found
bash: <path>: No such file or directory
ls:   cannot access '<path>': No such file or directory

These echoes can accidentally form a banned substring even when no internal Debug formatter ran. Two real cases:

Fix

Structural fix in assert_fuzz_invariants: strip lines that match a recognized real-shell error template before running the banned-shape check.

  • Byte-length cap (MAX_STDERR_BYTES) still runs on the unfiltered stderr — flood regressions still caught.
  • Host-canary check (TM-INF-013) still runs on the unfiltered stderr — env-leak regressions still caught.
  • The strict assert_no_leak path (used by per-builtin tests) is unchanged — non-fuzz tests must not produce shell echoes in the first place.

Recognized templates are conservative: prefix bash: or ls: plus a known suffix. Lines that look similar but don't match the exact template stay in stderr, so real Debug leaks that happen to coexist with shell errors still trip the assertion.

Test plan

  • 8 unit tests in testing::tests cover both directions — strip known shell echoes, keep internal panic/Debug lines, keep partial matches and lines from other tools.
  • cargo test -p bashkit --lib testing:: green
  • cargo test -p bashkit --test proptest_security --all-features green (18 cases)
  • cargo clippy -p bashkit --lib --tests -- -D warnings clean
  • cargo fmt --check clean
  • Threat-model TM-INF-022 section updated to document the carve-out
  • Manually dispatch Fuzz Testing on this branch and confirm glob_fuzz job is green

…pe check

When a fuzz/proptest target inlines arbitrary input bytes into a shell
script, bash and ls produce error messages that quote the input verbatim:

    bash: <cmd>: command not found
    bash: <path>: No such file or directory
    ls:   cannot access '<path>': No such file or directory

These echoes can accidentally form a banned substring even when no
internal Debug formatter ran. Two real cases:

- arithmetic_fuzz / glob_fuzz: input contained `/.rustup/toolchains/`
  literally; bash echoed it back. Filtered at the input layer in #1621
  and the arithmetic_fuzz follow-up.
- glob_fuzz (new): input ended with `Tok"`. Bash treated `Tok:` as a
  command and rendered `bash: Tok:: command not found` — the trailing
  `:` chrome from the error format glued onto the input's final `:` to
  form the banned parser-token shape `Tok::`. Pre-filtering the input
  for `Tok::` doesn't catch this — the substring only forms after
  bash's formatter runs.

Structural fix: in `assert_fuzz_invariants`, strip lines that match a
recognized real-shell error template before running the banned-shape
check. The byte-length cap (`MAX_STDERR_BYTES`) and the host-canary
check (TM-INF-013) still run on the unfiltered stderr, so flood and
env-leak regressions are still caught. The strict `assert_no_leak`
path (used by per-builtin tests) is unchanged — non-fuzz tests must
not produce shell echoes in the first place.

Recognized templates are conservative: prefix `bash: ` or `ls: ` plus
a known suffix (`: command not found`, `: No such file or directory`,
`: Is a directory`, `: Permission denied`, `: cannot execute …`, plus
the `command not found. Did you mean: ., :, [?` variant). Lines that
look similar but don't match the exact template stay in stderr, so
real Debug leaks that happen to coexist with shell errors still trip
the assertion.

Tests: 8 unit tests in `testing::tests` cover both directions —
strips known shell echoes, keeps internal panic/Debug lines, keeps
partial matches and lines from other tools. Threat-model TM-INF-022
section updated to document the carve-out.
@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit ca39280 Commit Preview URL

Branch Preview URL
May 10 2026, 09:51 AM

@chaliy chaliy merged commit a6a746c into main May 10, 2026
38 checks passed
@chaliy chaliy deleted the claude/fuzz-strip-real-shell-errors branch May 10, 2026 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant