Skip to content

bench-incremental: add nix-dynamic tools (experimental builder)#120

Closed
aldoborrero wants to merge 3 commits intomainfrom
aldo/bench-incremental-dynamic
Closed

bench-incremental: add nix-dynamic tools (experimental builder)#120
aldoborrero wants to merge 3 commits intomainfrom
aldo/bench-incremental-dynamic

Conversation

@aldoborrero
Copy link
Copy Markdown
Member

Summary

Adds nix-dynamic and nix-dynamic-nocgo to bench-incremental so the touch-and-rebuild benchmark can compare default-mode (dag) against buildGoApplicationExperimental (recursive-nix + dynamic-derivations + ca-derivations) on the same fixture and rooted local store.

  • Second expr template wraps app.target in runCommand so the existing nix-instantiatenix-store --realise flow in nixTool.build works unchanged.
  • New probe step builds each requested tool once before the run; dynamic tools that fail are dropped with a SKIP notice (mirrors benchmark-build's ca-derivations skip), dag tools still fail fast.
  • writeNixExpr now takes the template string so both templates share one Sprintf.
  • --tools help and docs/src/benchmarking.md updated, with the local-only / recursive-nix system-feature caveat.

Sample output (light fixture, leaf scenario)

Probing tools...
  OK   nix-nocgo
  OK   nix-dynamic-nocgo
...
| Scenario     | nix-nocgo (s / drvs) | nix-dynamic-nocgo (s / drvs) | Winner        | Speedup |
| leaf-private | 1.24 / 2             | 2.72 / 4                     | **nix-nocgo** | 2.19x   |

The 4 drvs for dynamic on a leaf touch are wrapper + resolved-link + the changed package + the runCommand symlink — fixed per-touch overhead regardless of edit depth, which is exactly what the comparison is meant to surface.

Test plan

  • go build && go vet in go/bench-incremental
  • nix build .#checks.x86_64-linux.{formatting,bench-incremental-help}
  • End-to-end build of generated bench-nix-dynamic-nocgo.nix produces a working static binary
  • Probe SKIP path fires when the dynamic build can't run; mixed --tools nix-nocgo,nix-dynamic-nocgo continues with the survivor

Adds nix-dynamic and nix-dynamic-nocgo to the tool set so the touch-a-file
incremental benchmark can compare default-mode (dag) against
buildGoApplicationExperimental (recursive-nix + dynamic-derivations +
ca-derivations) on the same fixture and rooted store.

The dynamic expr wraps app.target in a runCommand so the existing
instantiate→realise flow in nixTool.build needs no special-casing. A
once-per-run probe step builds each requested tool; dynamic tools are
dropped with a SKIP notice if the probe fails (mirrors benchmark-build's
ca-derivations skip), while dag tools keep failing fast.

writeNixExpr now takes the template as a parameter so both templates
share one Sprintf.

:house: Remote-Dev: homespace
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 15, 2026

Benchmark Regression Check

Scenario Tool Base (s) Current (s) Change Drvs (base) Drvs (curr) Status
no_change nix-ca-nocgo 0.67 0.68 +0.5% 0 0 ok
no_change nix-nocgo 0.67 0.68 +1.9% 0 0 ok
leaf-private nix-ca-nocgo 1.80 1.91 +6.2% 2 2 ok
leaf-private nix-nocgo 1.45 1.47 +1.0% 2 2 ok
mid-private nix-ca-nocgo 1.93 1.89 -2.1% 2 2 ok
mid-private nix-nocgo 1.51 1.54 +1.7% 4 4 ok
deep-private nix-ca-nocgo 2.17 2.09 -3.7% 3 3 ok
deep-private nix-nocgo 1.70 1.75 +2.4% 8 8 ok

Baseline: main | Current: ce8647bedb38a39148f7870e909d3df2ae00f80a

@aldoborrero aldoborrero requested a review from brianmcgee April 15, 2026 15:11
- benchmarking.md: "silently drops" → "drops with a SKIP notice" (the
  probe prints the reason).
- nix-dynamic-nocgo: pass CGO_ENABLED as "0" to match the dag-mode
  templates. dynamic/default.nix already toString-coerces so the int
  worked, but keep the four call sites uniform.

:house: Remote-Dev: homespace
@aldoborrero
Copy link
Copy Markdown
Member Author

Closing — #121 refactored bench-incremental to a buildTool interface and this conflicts in five places. The dynamic-mode comparison is recursive-nix-gated (local-only) anyway; easier to redo against the new interface if/when wanted than carry the rebase.

@aldoborrero aldoborrero deleted the aldo/bench-incremental-dynamic branch April 16, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant