fix: cross-crate DEP_<LINKS>_INCLUDE ${out_dir} token left unresolved (#163)#164
Open
esnunes wants to merge 3 commits into
Open
fix: cross-crate DEP_<LINKS>_INCLUDE ${out_dir} token left unresolved (#163)#164esnunes wants to merge 3 commits into
esnunes wants to merge 3 commits into
Conversation
…icbuild#163) A `-sys` crate that exposes an OUT_DIR-relative include dir via the `links`/metadata convention (e.g. `cargo:include=$OUT_DIR/include`) propagated `DEP_<LINKS>_INCLUDE` to a dependent build script with the producing crate's OUT_DIR rewritten to a literal `${out_dir}` token. That token is only resolved by the rustc action's process_wrapper for the crate that owns the build script; a cross-crate `.depenv` is consumed by a *dependent* crate's build-script runner (`bin.rs`), which resolves only `${pwd}`. The token survived unresolved, so e.g. librocksdb-sys could not find `lz4.h`. Fix `outputs_to_dep_env` in the rules_rust build-script runner to redact only the exec root (`${pwd}`) and keep the producing crate's real out_dir-relative path; that tree is already a declared input of the dependent action at the same path. Shipped as an in-repo patch applied via the existing `rules_rust.patch` extension (rules_rust is a pinned http_archive), with unit and end-to-end regression tests. Fixes hermeticbuild#163. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Provide a host `cargo`/`rustc` (1.94.1) and the bazelisk launcher (honoring `.bazelversion`), plus `test` / `test-e2e` / `gazelle` tasks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Member
|
Thanks for your PR! I'm about to go on a trip so I might not get to this one for a week or so - these things are a bit tricky and I want to make sure we're giving it our due diligence |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A
-syscrate that exposes anOUT_DIR-relative include directory via the cargolinks/metadata convention (e.g.cargo:include=$OUT_DIR/include) propagatedDEP_<LINKS>_INCLUDEto a dependent crate's build script with the producing crate'sOUT_DIRrewritten to a literal, unresolved${out_dir}token. The dependent's C/C++ compile then pointed-Iat<execroot>/${out_dir}/includeand failed.Concretely,
librocksdb-sys(LZ4 backend) could not findlz4.h:Fixes #163.
Root cause
In the rules_rust build-script runner (
cargo/private/cargo_build_script_runner/lib.rs),outputs_to_dep_envusedredact_paths, which rewrites the producing crate'sOUT_DIRto the generic${out_dir}token. That token is only resolved by the rustc action'sprocess_wrapper(via--out-dir), for the crate that owns the build script. A cross-crate.depenvfile is instead consumed by a dependent crate's build-script runner (bin.rs), which substitutes only${pwd}— so${out_dir}survived verbatim.Fix
outputs_to_dep_envnow redacts only the exec root (${pwd}) and keeps the producing crate's realout_dir-relative path. That output-directory tree is already a declared input of the dependent build-script action at that same exec-root-relative path, so${pwd}/<out_dir>/…resolves correctly. Build-script actions don't advertisesupports-path-mapping, so--experimental_output_paths=stripwon't rewrite the path either.Because rules_rust is provisioned as a pinned
http_archive(not vendored), the fix ships as an in-repo patch applied through the existingrules_rust.patchextension tag (same mechanism asrs/private/pyo3/patches/).Tests (in the patch)
dep_env_out_dir_is_preserved_not_tokenized(+ a Windows-path variant) assert the exact redacted string.test/cargo_build_script/metadata_dep_env— a producer build script emitscargo:include=$OUT_DIR/include(with a real header); the consumer readsDEP_PRODUCER_INCLUDEand a newmetadata_dep_env_include_testasserts it resolves to an existing path.Verification
metadata_dep_enve2e suite pass.lib.rsfix (proving it's a real regression guard).rocksdb 0.24.0, featuresbindgen-runtime+lz4) now builds to completion and the binary runs (writes/reads a key) — previously it failed instantly onlz4.h.Also included
A repo-root
mise.toml(separatechorecommit) providing a hostcargo/rustcand thebazelisklauncher plustest/test-e2e/gazelletasks. It's incidental dev tooling used to reproduce/verify this fix — happy to split it into its own PR if preferred.