bench-incremental: add nix-dynamic tools (experimental builder)#128
Open
aldoborrero wants to merge 5 commits intomainfrom
Open
bench-incremental: add nix-dynamic tools (experimental builder)#128aldoborrero wants to merge 5 commits intomainfrom
aldoborrero wants to merge 5 commits intomainfrom
Conversation
Adds nix-dynamic and nix-dynamic-nocgo as buildTool entries that build via buildGoApplicationExperimental (recursive-nix + dynamic-derivations + ca-derivations). The harness probes each tool once at startup and drops a tool with a SKIP notice if its probe build fails — so the dynamic tools degrade gracefully on stores without recursive-nix while the rest of the run continues. writeNixExpr now takes the template as an argument so the dag and experimental templates share the same path. :house: Remote-Dev: homespace
The recursive-nix inner daemon serves the same store the outer build runs against. With NIX_REMOTE=local?root=..., the inner go2nix resolve registers FOD drvs whose inputDrvs aren't all present in the rooted store, so AddToStore fails with "path '...' is not valid". The host store has them (it's where the bench resolved nixpkgs/go2nix-cli from). Dynamic builds are still per-fixture-path so concurrent runs don't interfere; the dag tools keep using the rooted store for isolation. :house: Remote-Dev: homespace
The default (NumCPU*2) launched 128 parallel FOD builds inside the recursive-nix sandbox on a 64-core host; one got OOM-killed. Honour the cores budget the outer build was given. :house: Remote-Dev: homespace
5 tasks
Benchmark Regression Check
Baseline: |
Bazel's cc_configure repository rule searches PATH for gcc during workspace loading. The bench-incremental wrapper only had nix and bazel on PATH, so the auto-configuration failed with: Cannot find gcc or CC Add pkgs.stdenv.cc (the wrapped compiler with correct Nix store include/library paths and binutils) so cc_configure can detect the host C toolchain. The recorded absolute paths work correctly with --incompatible_strict_action_env since build actions use those paths directly, not PATH lookups.
Bazel creates convenience symlinks (bazel-bin, bazel-out, etc.) in the workspace directory during builds. Since the bazel tool shares fixtureCopy with the nix tools, these symlinks change the store path when nix-dynamic re-evaluates the fixture, forcing a full wrapper rebuild that fails with "path is not valid" in the recursive-nix inner daemon. Redirect symlinks to the output base via --symlink_prefix so bazel leaves the shared fixture copy unmodified.
Member
|
There still appears to be a non-deterministic failure happening regardless of the combination of tools when using the torture fixture: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #124 (rebased onto main with #125/#127 merged; force-push avoided by opening fresh).
Adds
nix-dynamic/nix-dynamic-nocgotools to bench-incremental for measuring the experimental builder (buildGoApplicationExperimental, dynamic-derivations + recursive-nix + ca-derivations) against dag mode.dynExprTemplatecallingbuildGoApplicationExperimentalvia apkgs.runCommandwrapperbuildToolinterface gainsSkipOnFail() bool; dynamic tools set it so a missing recursive-nix store SKIPs cleanly instead of aborting the runlocal?root=...store) — the recursive-nix inner daemon needs the FOD inputDrvs visiblenix/dynamic/default.nix: cap--nix-jobsat$NIX_BUILD_CORESso the inner FOD builds don't OOM at NumCPU×2 parallelismVerified:
--fixture torture --tools nix-dynamic --runs 1 --scenario no_change→OK nix-dynamic / 1.06s. Full torture comparison (nix-ca vs nix-dynamic): dynamic wins no_change ~1.5× (saves thederivationStricteval floor); dag wins all touch scenarios ~1.8× (no iface cutoff in dynamic — deep is 18 drvs vs 2 — and the wrapper drv rerunsgo list+drv-gen ~10s/touch).