Skip to content

fix: cross-crate DEP_<LINKS>_INCLUDE ${out_dir} token left unresolved (#163)#164

Open
esnunes wants to merge 3 commits into
hermeticbuild:mainfrom
esnunes:fix/cross-crate-dep-env-out-dir
Open

fix: cross-crate DEP_<LINKS>_INCLUDE ${out_dir} token left unresolved (#163)#164
esnunes wants to merge 3 commits into
hermeticbuild:mainfrom
esnunes:fix/cross-crate-dep-env-out-dir

Conversation

@esnunes

@esnunes esnunes commented Jun 25, 2026

Copy link
Copy Markdown

Problem

A -sys crate that exposes an OUT_DIR-relative include directory via the cargo links/metadata convention (e.g. cargo:include=$OUT_DIR/include) propagated DEP_<LINKS>_INCLUDE to a dependent crate's build script with the producing crate's OUT_DIR rewritten to a literal, unresolved ${out_dir} token. The dependent's C/C++ compile then pointed -I at <execroot>/${out_dir}/include and failed.

Concretely, librocksdb-sys (LZ4 backend) could not find lz4.h:

rocksdb/util/compression.h:49:10: fatal error: 'lz4.h' file not found

Fixes #163.

Root cause

In the rules_rust build-script runner (cargo/private/cargo_build_script_runner/lib.rs), outputs_to_dep_env used redact_paths, which rewrites the producing crate's OUT_DIR to the generic ${out_dir} token. That token is only resolved by the rustc action's process_wrapper (via --out-dir), for the crate that owns the build script. A cross-crate .depenv file is instead consumed by a dependent crate's build-script runner (bin.rs), which substitutes only ${pwd} — so ${out_dir} survived verbatim.

Fix

outputs_to_dep_env now redacts only the exec root (${pwd}) and keeps the producing crate's real out_dir-relative path. That output-directory tree is already a declared input of the dependent build-script action at that same exec-root-relative path, so ${pwd}/<out_dir>/… resolves correctly. Build-script actions don't advertise supports-path-mapping, so --experimental_output_paths=strip won't rewrite the path either.

Because rules_rust is provisioned as a pinned http_archive (not vendored), the fix ships as an in-repo patch applied through the existing rules_rust.patch extension tag (same mechanism as rs/private/pyo3/patches/).

Tests (in the patch)

  • Unit: dep_env_out_dir_is_preserved_not_tokenized (+ a Windows-path variant) assert the exact redacted string.
  • End-to-end: extends test/cargo_build_script/metadata_dep_env — a producer build script emits cargo:include=$OUT_DIR/include (with a real header); the consumer reads DEP_PRODUCER_INCLUDE and a new metadata_dep_env_include_test asserts it resolves to an existing path.

Verification

  • Runner unit tests + the metadata_dep_env e2e suite pass.
  • Negative control: the e2e include test fails without the lib.rs fix (proving it's a real regression guard).
  • The exact issue reproduction (rocksdb 0.24.0, features bindgen-runtime + lz4) now builds to completion and the binary runs (writes/reads a key) — previously it failed instantly on lz4.h.

Also included

A repo-root mise.toml (separate chore commit) providing a host cargo/rustc and the bazelisk launcher plus test/test-e2e/gazelle tasks. It's incidental dev tooling used to reproduce/verify this fix — happy to split it into its own PR if preferred.

esnunes and others added 3 commits June 25, 2026 02:00
…icbuild#163)

A `-sys` crate that exposes an OUT_DIR-relative include dir via the
`links`/metadata convention (e.g. `cargo:include=$OUT_DIR/include`)
propagated `DEP_<LINKS>_INCLUDE` to a dependent build script with the
producing crate's OUT_DIR rewritten to a literal `${out_dir}` token. That
token is only resolved by the rustc action's process_wrapper for the crate
that owns the build script; a cross-crate `.depenv` is consumed by a
*dependent* crate's build-script runner (`bin.rs`), which resolves only
`${pwd}`. The token survived unresolved, so e.g. librocksdb-sys could not
find `lz4.h`.

Fix `outputs_to_dep_env` in the rules_rust build-script runner to redact
only the exec root (`${pwd}`) and keep the producing crate's real
out_dir-relative path; that tree is already a declared input of the
dependent action at the same path. Shipped as an in-repo patch applied via
the existing `rules_rust.patch` extension (rules_rust is a pinned
http_archive), with unit and end-to-end regression tests.

Fixes hermeticbuild#163.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Provide a host `cargo`/`rustc` (1.94.1) and the bazelisk launcher
(honoring `.bazelversion`), plus `test` / `test-e2e` / `gazelle` tasks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dzbarsky

Copy link
Copy Markdown
Member

Thanks for your PR! I'm about to go on a trip so I might not get to this one for a week or so - these things are a bit tricky and I want to make sure we're giving it our due diligence

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cross-crate DEP_<LINKS>_INCLUDE propagation broken: ${out_dir} token left unresolved in dependent build script (rocksdb + lz4 repro)

2 participants