-
Notifications
You must be signed in to change notification settings - Fork 0
ENG-4691: rustyroute cargo-fuzz target + OSS-Fuzz submission #14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
jimbofreedman
wants to merge
15
commits into
main
Choose a base branch
from
worktree-ENG-4691
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
b6a58a4
feat: ENG-4691 cargo-fuzz targets for Graph::from_bytes and route
jimbofreedman a1b9b39
ci: ENG-4691 fuzz.yaml quick-pass (nightly cargo-fuzz, 60s load_archive)
jimbofreedman 68c0c5f
docs: ENG-4691 OSS-Fuzz project files + submission runbook
jimbofreedman 7afb1f7
test: ENG-4691 lock fuzz workflow + workspace-exclusion invariants
jimbofreedman 9633ecf
fix: ENG-4691 deliver OSS-Fuzz seed via <target>_seed_corpus.zip
jimbofreedman 028fde5
test: ENG-4691 lock OSS-Fuzz staging files as durable invariants
jimbofreedman 2d1b5cf
docs: ENG-4691 changelog entry for cargo-fuzz + OSS-Fuzz
jimbofreedman fd907a4
fix: ENG-4691 pin gnu target for fuzz CI (ASan vs musl static libc)
jimbofreedman 2a9599a
fix: ENG-4691 bound route_inputs blocked-set length (Copilot review)
jimbofreedman b5e2274
fix: ENG-4691 confine leaked borrow before free in load_archive
jimbofreedman 4356ec2
chore: ENG-4691 drop unused arbitrary derive feature from fuzz crate
jimbofreedman dd90d10
docs: ENG-4691 clarify the auto_ccs TODO is a comment, not a placeholder
jimbofreedman 5c03573
fix: ENG-4691 pin fuzz target triple and $SRC paths in OSS-Fuzz build
jimbofreedman 6790879
chore: ENG-4691 add jimbo@freedman.io as OSS-Fuzz co-maintainer
jimbofreedman 7c6b66f
docs: ENG-4691 reference loader symbols instead of line numbers
jimbofreedman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| name: "fuzz" | ||
|
|
||
| # ENG-4691: cargo-fuzz quick-pass. Feeds arbitrary bytes to | ||
| # Graph::from_bytes (the load_archive target) for 60s per PR so malformed | ||
| # archives can't panic, OOM, or segfault. The deep, continuous variant runs | ||
| # on OSS-Fuzz (see oss-fuzz/). Mirrors ci.yaml's conventions; the sole | ||
| # deviation is the toolchain: cargo-fuzz needs a nightly sanitizer toolchain, | ||
| # so this is the one job in the repo that is not stable-only. | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: [main] | ||
| paths-ignore: | ||
| - '**/*.md' | ||
| - 'docs/**' | ||
| workflow_dispatch: | ||
|
|
||
| concurrency: | ||
| group: fuzz-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| load-archive: | ||
| name: fuzz load_archive | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: dtolnay/rust-toolchain@nightly | ||
| with: | ||
| # ASan is incompatible with a statically linked libc, so the fuzz | ||
| # build must target the dynamically-linked gnu triple. Pin it | ||
| # explicitly (both here and on the cargo-fuzz commands below) — | ||
| # on GitHub's runners cargo-fuzz otherwise defaults to the musl | ||
| # target, which fails with "sanitizer is incompatible with | ||
| # statically linked libc". | ||
| targets: x86_64-unknown-linux-gnu | ||
| - uses: taiki-e/install-action@v2 | ||
| with: | ||
| tool: cargo-fuzz | ||
| - uses: Swatinem/rust-cache@v2 | ||
| with: | ||
| workspaces: fuzz | ||
| shared-key: fuzz | ||
| # Compile BOTH targets so a break in route_inputs is caught even | ||
| # though only load_archive is run in the quick-pass. | ||
| - run: cargo fuzz build --target x86_64-unknown-linux-gnu | ||
| # 60s libFuzzer run seeded from fuzz/corpus/load_archive/. | ||
| - run: cargo fuzz run load_archive --target x86_64-unknown-linux-gnu -- -max_total_time=60 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| /target/ | ||
| /artifacts/ | ||
| /coverage/ | ||
| # Fuzz crate lockfile — mirrors the repo's no-lockfile stance for library and | ||
| # sub-package crates (see root .gitignore); OSS-Fuzz rebuilds from scratch. | ||
| /Cargo.lock | ||
| # libFuzzer-discovered corpus entries are regenerable noise; commit only the | ||
| # named seed(s) under corpus/<target>/seed_*. | ||
| /corpus/*/* | ||
| !/corpus/*/seed_* |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| [package] | ||
| name = "rustyroute-fuzz" | ||
| version = "0.0.0" | ||
| publish = false | ||
| edition = "2021" | ||
|
|
||
| [package.metadata] | ||
| cargo-fuzz = true | ||
|
|
||
| # Empty table => fuzz/ is its OWN workspace root, so it is never absorbed | ||
| # into a parent [workspace] even if one is later added to the root | ||
| # Cargo.toml. The root crate is single-crate today (no [workspace]), so the | ||
| # fuzz package is already excluded; this future-proofs that guarantee. | ||
| [workspace] | ||
|
|
||
| [dependencies] | ||
| libfuzzer-sys = "0.4" | ||
| # No `derive` feature: `RouteInput` implements `Arbitrary` by hand (to bound | ||
| # the blocked-set length), so the derive macro would be dead weight in the | ||
| # fuzz and OSS-Fuzz builds. | ||
| arbitrary = "1" | ||
|
|
||
| [dependencies.rustyroute] | ||
| path = ".." | ||
| # Default features (=> data-50km) expose rustyroute::data::BYTES_50KM for the | ||
| # route_inputs fixed graph. | ||
|
|
||
| [[bin]] | ||
| name = "load_archive" | ||
| path = "fuzz_targets/load_archive.rs" | ||
| test = false | ||
| doc = false | ||
| bench = false | ||
|
|
||
| [[bin]] | ||
| name = "route_inputs" | ||
| path = "fuzz_targets/route_inputs.rs" | ||
| test = false | ||
| doc = false | ||
| bench = false |
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| #![no_main] | ||
| //! ENG-4691: fuzz `rustyroute::Graph::from_bytes` on arbitrary bytes. | ||
| //! | ||
| //! Contract: feeding any byte slice to `from_bytes` must only ever return a | ||
| //! typed `LoadError` (or `Ok`) — never panic, OOM, or segfault. `from_bytes` | ||
| //! already guards its header length before slicing (`validate_header`) and | ||
| //! runs rkyv's *checked* `access`, so this target proves that contract holds | ||
| //! across the whole input space and guards against regressions. | ||
|
|
||
| use libfuzzer_sys::fuzz_target; | ||
|
|
||
| fuzz_target!(|data: &[u8]| { | ||
| // `Graph::from_bytes` takes `&'static [u8]`. Promote a copy of the | ||
| // transient fuzz buffer to `'static`, then reclaim it after use so RSS | ||
| // stays flat across libFuzzer's many in-process iterations. A plain | ||
| // `Box::leak` would accumulate one copy per iteration and — with the | ||
| // large committed seed driving `-max_len` up — climb toward | ||
| // `-rss_limit_mb` and trip a false-positive OOM crash. | ||
| let ptr = Box::into_raw(data.to_vec().into_boxed_slice()); | ||
|
|
||
| // Inner scope confines the `&'static [u8]` borrow (and the `Graph` that | ||
| // holds it) so both are definitely dead before the free below — no shared | ||
| // reference derived from `ptr` is live across `Box::from_raw`. | ||
| { | ||
| // SAFETY: `ptr` is a freshly created boxed slice we have not freed, so | ||
| // dereferencing it to a shared slice is valid. | ||
| let leaked: &'static [u8] = unsafe { &*ptr }; | ||
| let _ = rustyroute::Graph::from_bytes(leaked); | ||
| } | ||
|
|
||
| // SAFETY: a `Graph` holds only `GraphBacking::Static(&'static [u8])` — a | ||
| // borrow, not an owner — and both it and `leaked` went out of scope above, | ||
| // so no reference into `ptr` survives. Reconstructing the `Box` to free it | ||
| // is sound. | ||
| drop(unsafe { Box::from_raw(ptr) }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| #![no_main] | ||
| //! ENG-4691: fuzz `rustyroute::Graph::route` argument composition against a | ||
| //! fixed graph. | ||
| //! | ||
| //! Contract: for a valid graph, any `(from, to, blocked)` composition must | ||
| //! only ever return `Ok(Route)` or a typed `RouteError` — never panic. Coords | ||
| //! are validated inside `Graph::route`, and blocked ids that do | ||
| //! not exist simply never filter anything, so arbitrary inputs are safe by | ||
| //! construction; this target proves it empirically. | ||
|
|
||
| use arbitrary::{Arbitrary, Unstructured}; | ||
| use libfuzzer_sys::fuzz_target; | ||
| use std::collections::HashSet; | ||
| use std::sync::OnceLock; | ||
|
|
||
| /// Structured fuzz input. The f64 endpoints span NaN/±inf and the full range, | ||
| /// exercising coord validation; libFuzzer's coverage feedback learns in-range | ||
| /// coordinates over time to reach the Dijkstra path. Both endpoints are fuzzed | ||
| /// independently (self-route and cross-node paths). | ||
| #[derive(Debug)] | ||
| struct RouteInput { | ||
| from: (f64, f64), | ||
| to: (f64, f64), | ||
| blocked: Vec<u32>, | ||
| } | ||
|
|
||
| /// Hand-written `Arbitrary` (rather than derive) so the `blocked` set length is | ||
| /// bounded. `route` treats unknown edge ids as no-op filters, so blocked-set | ||
| /// *size* has no bearing on routing correctness — but a derived unbounded | ||
| /// `Vec<u32>` would grow with the fuzzer's input size (especially under | ||
| /// OSS-Fuzz), producing large `Vec`/`HashSet` allocations that manifest as | ||
| /// slow units or OOMs unrelated to `Graph::route`. The count is read from a | ||
| /// single `u8`, capping the set at 255 elements. | ||
| impl<'a> Arbitrary<'a> for RouteInput { | ||
| fn arbitrary(u: &mut Unstructured<'a>) -> arbitrary::Result<Self> { | ||
| let from = <(f64, f64)>::arbitrary(u)?; | ||
| let to = <(f64, f64)>::arbitrary(u)?; | ||
| let count = u8::arbitrary(u)? as usize; // ≤ 255 — bounds memory use | ||
| let mut blocked = Vec::with_capacity(count); | ||
| for _ in 0..count { | ||
| blocked.push(u32::arbitrary(u)?); | ||
| } | ||
| Ok(RouteInput { from, to, blocked }) | ||
| } | ||
| } | ||
|
|
||
| /// The fixed graph, built once from the baked 50 km archive (available via the | ||
| /// path dep's default `data-50km` feature). | ||
| fn graph() -> &'static rustyroute::Graph { | ||
| static G: OnceLock<rustyroute::Graph> = OnceLock::new(); | ||
| G.get_or_init(|| { | ||
| rustyroute::Graph::from_bytes(rustyroute::data::BYTES_50KM) | ||
| .expect("baked 50km archive is valid") | ||
| }) | ||
| } | ||
|
|
||
| fuzz_target!(|input: RouteInput| { | ||
| let blocked: HashSet<u32> = input.blocked.into_iter().collect(); | ||
| let _ = graph().route(input.from, input.to, &blocked); | ||
| }); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # OSS-Fuzz submission for rustyroute (ENG-4691) | ||
|
|
||
| This directory stages the files that OSS-Fuzz needs for continuous fuzzing of | ||
| `rustyroute`. They live here so they are versioned alongside the fuzz targets | ||
| they build, but the **actual submission is a pull request to the external | ||
| [`google/oss-fuzz`](https://github.com/google/oss-fuzz) repository** — it | ||
| cannot be merged from this repo. | ||
|
|
||
| ``` | ||
| oss-fuzz/projects/rustyroute/ | ||
| project.yaml # engine/sanitizer config + maintainer contacts | ||
| Dockerfile # clones this repo into the OSS-Fuzz base-builder-rust image | ||
| build.sh # cargo fuzz build -O; copies target binaries + seed to $OUT | ||
| ``` | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - `github.com/spotship/rustyroute` must be **public** (OSS-Fuzz only fuzzes | ||
| public projects). The Dockerfile clones over HTTPS. | ||
| - OSS-Fuzz requires **two maintainer email addresses** associated with the | ||
| project. Satisfied: `project.yaml` `auto_ccs` lists `jimbo@spot-ship.com` | ||
| and `jimbo@freedman.io`. | ||
|
|
||
| ## Submission steps | ||
|
|
||
| Steps 1–2 are already done; the submission itself (steps 3–6) is a deliberate | ||
| human follow-up and has **not** been started. | ||
|
|
||
| 1. ~~Confirm the co-maintainer email and add it to `auto_ccs`.~~ Done — both | ||
| maintainer addresses are in `project.yaml`. | ||
| 2. ~~Verify the repo is public.~~ Done — `spotship/rustyroute` is public. | ||
| 3. Fork `google/oss-fuzz`. Copy this `projects/rustyroute/` directory to | ||
| `projects/rustyroute/` in the fork (drop the `oss-fuzz/` prefix — in | ||
| google/oss-fuzz the path is `projects/rustyroute/`). | ||
| 4. Validate locally against the OSS-Fuzz tooling: | ||
| ```sh | ||
| python infra/helper.py build_image rustyroute | ||
| python infra/helper.py build_fuzzers rustyroute | ||
| python infra/helper.py check_build rustyroute | ||
| ``` | ||
| 5. Open the pull request to `google/oss-fuzz`. Approval typically takes | ||
| **1–3 weeks**; an OSS-Fuzz maintainer must merge it. | ||
| 6. Once the PR is open, paste its link into ClickUp ticket **ENG-4691** as the | ||
| tracking link. | ||
|
|
||
| ## Notes | ||
|
|
||
| - The CI quick-pass (`.github/workflows/fuzz.yaml`) runs `load_archive` for 60s | ||
| per PR. OSS-Fuzz runs the deep, continuous variant of both targets. | ||
| - `build.sh` builds both `load_archive` and `route_inputs`; keep the target | ||
| list in sync with `fuzz/Cargo.toml`. | ||
| - The committed seed is delivered to OSS-Fuzz as | ||
| `$OUT/load_archive_seed_corpus.zip` — OSS-Fuzz only ingests seed corpora | ||
| from `<target>_seed_corpus.zip`, not from loose files copied into `$OUT`. | ||
| `base-builder-rust` provides `zip`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| FROM gcr.io/oss-fuzz-base/base-builder-rust | ||
| # Absolute $SRC paths: build.sh does `cd "$SRC/rustyroute"`, so spell the same | ||
| # location here instead of leaning on the base image's default working dir | ||
| # being $SRC. | ||
| RUN git clone --depth 1 https://github.com/spotship/rustyroute $SRC/rustyroute | ||
| WORKDIR $SRC/rustyroute | ||
| COPY build.sh $SRC/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| #!/bin/bash -eu | ||
| # ENG-4691: OSS-Fuzz build script for rustyroute. Runs inside the | ||
| # gcr.io/oss-fuzz-base/base-builder-rust image (nightly + cargo-fuzz + clang | ||
| # preinstalled). Builds every fuzz target and copies the binaries — plus the | ||
| # committed seed corpus — into $OUT for the OSS-Fuzz runners. | ||
|
|
||
| cd "$SRC/rustyroute" | ||
|
|
||
| # Pin the target triple rather than relying on cargo-fuzz's default. That | ||
| # default is the *host* triple, which is not reliably gnu — it resolved to musl | ||
| # on GitHub's runners, and ASan (project.yaml `sanitizers: address`) is | ||
| # incompatible with a statically linked libc. See the same pin in | ||
| # .github/workflows/fuzz.yaml. Deriving the output dir from the same variable | ||
| # keeps the build and the copy below from ever disagreeing. | ||
| FUZZ_TARGET_TRIPLE="x86_64-unknown-linux-gnu" | ||
|
|
||
| # cargo-fuzz auto-locates the fuzz/ package from the crate root. | ||
| cargo fuzz build -O --target "$FUZZ_TARGET_TRIPLE" | ||
|
|
||
| FUZZ_TARGET_OUTPUT_DIR="fuzz/target/$FUZZ_TARGET_TRIPLE/release" | ||
| for target in load_archive route_inputs; do | ||
| cp "$FUZZ_TARGET_OUTPUT_DIR/$target" "$OUT/" | ||
| done | ||
|
|
||
| # Ship the committed seed corpus so OSS-Fuzz starts with coverage. OSS-Fuzz | ||
| # only ingests seeds from $OUT/<target>_seed_corpus.zip (loose files in $OUT | ||
| # are ignored), so package the load_archive seed(s) into that zip. Only | ||
| # load_archive has a committed seed; route_inputs relies on coverage-guided | ||
| # discovery. | ||
| if compgen -G "fuzz/corpus/load_archive/*" > /dev/null; then | ||
| zip -j "$OUT/load_archive_seed_corpus.zip" fuzz/corpus/load_archive/* | ||
| fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| homepage: "https://github.com/spotship/rustyroute" | ||
| main_repo: "https://github.com/spotship/rustyroute" | ||
| language: rust | ||
| primary_contact: "jimbo@spot-ship.com" | ||
| auto_ccs: | ||
| - "jimbo@spot-ship.com" | ||
| - "jimbo@freedman.io" | ||
| fuzzing_engines: | ||
| - libfuzzer | ||
| sanitizers: | ||
| - address |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.