AFL all the way - #13
Merged
Merged
Conversation
The fuzz build no longer needs the OSS-Fuzz Docker image. The local build
only enables -fsanitize=fuzzer + UBSan (no MemorySanitizer), so the
instrumented-libc++ world Docker provided is unnecessary; we link against the
system's libstdc++-built boost, protobuf and abseil instead. Only
libprotobuf-mutator and evmone-standalone are still built from source, staged
into deps/ by scripts/build_ossfuzz.sh.
- New cmake/toolchains/libfuzzer-native.cmake (libc++-free; -O2 -DNDEBUG to
avoid a clang-22 -O1 miscompile of the solidity AST; lld; opt-in -DFUZZ_UBSAN).
- tools/ossfuzz/CMakeLists.txt: resolve libprotobuf-mutator / evmone-standalone
/ system protobuf via variables instead of hard-coded /usr paths and bare
lib names; delete the abiv2_isabelle_ossfuzz target.
- scripts/build_ossfuzz.sh: native build (regenerate protobuf bindings, build
the two deps into deps/, configure + make) — no docker run.
- Delete scripts/docker/ Dockerfiles, publish-images.yml, update_solidity.sh,
AbiV2IsabelleFuzzer.cpp.
- build-fuzz.yml / build-afl.yml: run natively on ubuntu-latest with apt deps.
- Regenerate committed *.pb.{cc,h} with the system protoc.
- Update README, CLAUDE.md, tools/afl/README.md.
Note: the -O2 SEGV fix is applied but not yet end-to-end verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the 'no Docker' framing, the libc++/MSan FAQ, and stray Docker mentions now that the fuzz build is native. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tools/ossfuzz/CMakeLists.txt: collapse the 15+ near-identical harness
definitions behind a single add_ossfuzz_target(name SOURCES.. LIBS.. DEFINES..)
helper and shared SOURCE/LIB groups (528 -> ~270 lines). Also:
- remove the dead commented-out abiv2_proto_ossfuzz block
- drop ${COMPILE_OPTIONS} (never defined anywhere; expanded to nothing)
- drop the per-target target_include_directories for the LPM/evmone roots
(already added globally via include_directories)
- PUBLIC -> PRIVATE compile options on executables
libfuzzer-native.cmake: remove the dead "-O2 -DNDEBUG" from CMAKE_CXX_FLAGS
(Release's -O3 -DNDEBUG is appended after it and wins, so it never took
effect) and replace the inaccurate "-O1 miscompile" comment with the actual
findings on the clang+libstdc++ dynamic_cast SEGV: ruled out linker, type_info
ODR/visibility, corrupt RTTI data, and optimisation level.
No change to effective build behaviour.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The build regenerates tools/ossfuzz/*.pb.{cc,h} on every run (now with the
libc++ protoc in deps/), so committing them just produces large, version-
dependent churn in every PR. git-ignore and untrack them instead; they stay on
disk and are rebuilt by scripts/build_ossfuzz.sh (the IDE/LSP just needs one
build first).
Also refresh CLAUDE.md to match the libc++ fuzz build (clang+libc++; boost,
protobuf+abseil, evmone and libprotobuf-mutator built from source into deps/;
.pb bindings git-ignored).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The examples are added with EXCLUDE_FROM_ALL but CMake still configures them, and examples/libxml2 requires LIBLZMA_LIBRARY which isn't present on the CI runner — failing configure. We don't build the examples, so turn them off (LIB_PROTO_MUTATOR_EXAMPLES=OFF) to avoid the transitive liblzma/libxml2/expat dependency. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The distro libclang_rt.fuzzer.a is compiled against libstdc++ (every clang ships it that way, including apt.llvm.org builds), so linking it into the -stdlib=libc++ harnesses leaves its internal std::__cxx11 string/iostream symbols undefined (ld.lld: undefined symbol: std::ios_base::ios_base(), std::__cxx11::basic_string<...>::reserve, ...). Bumping the clang/Ubuntu version does not help. Build libFuzzer from the matching-version compiler-rt source with -stdlib=libc++ into deps/lib/libFuzzer.a (like boost/protobuf/evmone already are) and link it via -DLIB_FUZZING_ENGINE. The toolchain now only falls back to -fsanitize=fuzzer when no engine is supplied. Revert the build-fuzz runner to ubuntu-latest; the runner version is irrelevant now. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate the libFuzzer/libc++ proto fuzzers onto AFL++ with afl-clang-fast++ and the system libstdc++, keeping libprotobuf-mutator for structure-aware mutation. This drops the entire libc++ ABI cascade: boost/protobuf/abseil come from the system, only LPM is still built from source (a normal libstdc++ build), and the from-source libFuzzer engine is replaced by AFL++'s libAFLDriver.a. The harnesses are unchanged (DEFINE_PROTO_FUZZER stays engine-agnostic, so the OSS-Fuzz/libFuzzer build still works). LPM mutation is brought to AFL via a custom-mutator .so: tools/ossfuzz/lpm_afl_mutator.cc wraps the base protobuf_mutator::Mutator and serializes in text format to match the harness's LoadProtoInput, so the existing corpus is reused as-is and the .so carries no dependency on the engine's LLVMFuzzerMutate symbol. Run with AFL_CUSTOM_MUTATOR_ONLY=1 so byte havoc never corrupts the protobuf. tools/ossfuzz/CMakeLists.txt grows an AFL-mode branch (detected via afl-clang-fast): link the in-tree static evmone (EVMONE_STATIC) built by evmone_external, use the system protobuf, and take the engine from LIB_FUZZING_ENGINE (libAFLDriver.a). Pilot on sol_proto_ossfuzz_evmone: builds clean, no dynamic_cast SEGV, all mutations driven by the LPM .so, ~90% of generated programs compile (the rest are pre-existing protoToSol2 quirks, identical under libFuzzer). Rollout (remaining grammars, build script rewrite, run-script wiring, libc++ teardown, docs) still to come. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Complete the migration started by the pilot. scripts/build_ossfuzz.sh is now the AFL builder: it builds libprotobuf-mutator from source into deps_afl/ against the system protobuf, regenerates the bindings, builds one LPM custom-mutator .so per grammar (sol2/yul/sol/shuffler/recstruct/ roundtrip/abiV2) from lpm_afl_mutator.cc, and builds the fuzzers into build_afl/ with afl-clang-fast++ and AFL++'s libAFLDriver.a. No libc++, no Docker, no from-source boost/protobuf/abseil/libFuzzer. scripts/run_ossfuzz_afl.sh maps a fuzzer to its grammar mutator and runs afl-fuzz with AFL_CUSTOM_MUTATOR_ONLY=1. Validated sol_proto and yul_proto end to end under AFL. READMEs rewritten AFL-first and much shorter (two build trees: build/ + build_afl/, no Docker/libc++): README.md 197->94, tools/ossfuzz 457->71, tools/afl 425->77. All lines <=80 chars, no wide tables. shuffler-fuzzer and vendored READMEs left untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The AFL+LPM migration moved the fuzzers from build_ossfuzz/ to build_afl/ and switched crash files from libFuzzer's "crash-<hash>" to AFL++'s "id:000000,sig:..." naming. The three replay scripts still pointed at the old tree and filtered for "crash-*", so they found zero crashes. Repoint the dumper paths to build_afl/ and add a shared is_crash_file() that accepts both libFuzzer and AFL names while still excluding our generated .sol/.yul/.out/.bt/.seq/.dump.txt artifacts and AFL's crashes/README.txt. The replay machinery is unchanged: the harnesses are engine-agnostic, so PROTO_FUZZER_DUMP_PATH and the build/ debug runners behave identically, and libAFLDriver.a replays a file arg like libFuzzer. run_sol_crashes.py also notes that an AFL crash dir is findings_<fuzzer>/default/crashes (basename "crashes"), so --fuzzer must be passed explicitly there. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The run example pointed at a nonexistent `corpus_sol` placeholder. Use a real seed dir (seeds_sol_proto) and note the seed must be text-format protobuf — a one-byte default-message seed is enough — and that Solidity source corpora (corpus_afl/) are not valid seeds for the proto fuzzers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a --resume flag that runs afl-fuzz with -i- so AFL re-reads its own findings dir and continues from the existing queue (no corpus dir needed). Document it with a short example in the ossfuzz README. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The workflow still described the old libc++/libFuzzer world and would abort immediately — build_ossfuzz.sh now requires AFL++'s afl-clang-fast++ and libAFLDriver.a, plus system protobuf/abseil/boost and ninja/pkg-config. Rewrite the comment, swap the apt packages (drop libc++, add ninja-build, pkg-config, llvm-dev, protobuf-compiler, libprotobuf-dev, libabsl-dev, libboost-all-dev), and add a Build AFL++ step (make source-only NO_NYX=1) before building the fuzzers. Rename the ccache key to ...-afl-. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ccache 4.x (ubuntu-latest) defaults its cache to ~/.cache/ccache, so the cache step's ~/.ccache path never populated — the cache was a silent no-op. Pin CCACHE_DIR=~/.ccache to match. Also drop wget: the AFL build_ossfuzz.sh git-clones LPM and takes everything else from submodules/system. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The libFuzzer-native -> AFL++ pivot left behind stale references to the old design. Update CLAUDE.md's build-model/fuzz sections to describe the actual flow (afl-clang-fast++ + system libstdc++, LPM into deps_afl/, build_afl/ tree, LPM custom-mutator bridge), and note the now-dormant libc++/libFuzzer CMake branch as untested. Also delete the orphaned cmake/toolchains/libfuzzer-native.cmake (added then immediately obsoleted within this branch; nothing references it) and fix two build_ossfuzz/ -> build_afl/ references that the rollout missed in files it didn't otherwise touch (sol_ice_crash_backtrace_categorizer.py, build_instrumented.sh). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
afl-fuzz aborts with "No usable test cases" on an empty/missing input dir even though the LPM custom mutator can grow a corpus from nothing, since the mutator transforms an existing queue entry rather than generating one. Drop in a single non-empty seed in the fresh-run path so afl-fuzz has something to start mutating; --resume is unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update the build-afl/build-fuzz workflows, CLAUDE.md/README, ossfuzz CMakeLists and README for the native AFL++ build, drop the orphaned libc++/libfuzzer toolchain files, and rename the YulArity patch now that it is no longer libc++-specific. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The harness and LPM mutator use text-format protobuf (use_binary=false), not binary, so the previous \x00 auto-seed made the harness flood stderr with "Error parsing text-format ... Program". Write a comment line instead: non-empty (AFL requires it) yet parses to a default message. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
AFL trims new queue entries by truncating raw bytes, bypassing the LPM
custom mutator and handing the harness an invalid (truncated) text-format
proto — flooding stderr with "Error parsing text-format ... Program"
("Expected {, found ''"). We don't implement the custom trim API, so set
AFL_DISABLE_TRIM=1; the grammar mutator keeps inputs minimal regardless.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
afl-cmin tests its -o dir for emptiness with os.rmdir(), which fails with ENOTDIR on a symlink and then reports "exists and is not empty" even when the target is empty. fuzz-afl points corpus_tsgen at the shared corpus via a symlink, so cmin always aborted. Minimize into a real temp dir and copy the result into OUT instead.
scripts/build_ossfuzz.sh and scripts/run_ossfuzz_afl.sh build and run the proto fuzzers that live in tools/ossfuzz/, so co-locate them there (mirroring tools/afl/, which already keeps its scripts next to its C++). The top-level scripts/ dir is now gone. Fixed the ROOTDIR depth in both scripts and updated all path references (CI workflow, CLAUDE.md, READMEs, .gitignore, CMakeLists). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Can we merge this? Then I can make the "local VM test" work in a more general way. I don't want to make a mess of it now, it's useful to debug this change :) BTW, this currently needs to be used with for example. This is because I don't want to merge the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.