Enzyme 2nd order fix by michel2323 · Pull Request #322 · JuliaSmoothOptimizers/ADNLPModels.jl

michel2323 · 2025-01-30T19:22:00Z

No description provided.

Copilot

Pull request overview

Updates the Enzyme-based AD backends and test harness to improve/enable second-order (Hessian / Hvprod) functionality and run Enzyme tests in CI.

Changes:

Refactors Enzyme backend implementations to use preallocated buffers and new Enzyme autodiff call patterns for Jacobian/Hessian/Hvprod.
Introduces and tests a new predefined backend entry (:enzyme_backend) and adjusts which backend symbol is used in test suites.
Enables the Enzyme CPU test step in Buildkite and adds Enzyme + ADNLPModels to the test project deps.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
`src/enzyme.jl`	Major refactor of Enzyme AD backend structs/buffers and Hvprod/Jprod/Jtprod/Hessian implementations.
`test/enzyme.jl`	Adds `:enzyme_backend` predefined backend for tests and updates sparse Hessian test invocations.
`test/sparse_hessian.jl`	Adjusts sparse Hessian tests (signature now includes `info`; edits Rosenbrock test setup).
`test/nlp/nlpmodelstest.jl`	Skips view-subarray tests for `:enzyme_backend` as well as `:enzyme`.
`test/nls/nlpmodelstest.jl`	Same skip adjustment for NLS tests.
`test/Project.toml`	Adds `ADNLPModels` and `Enzyme` to the test project dependencies.
`.buildkite/pipeline.yml`	Re-enables the Enzyme CPU test step in CI.
`src/ADNLPModels.jl`	Leaves commented debug code in `get_lag`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-13T18:03:08Z

 function sparse_hessian(backend, info, kw)
  @testset "Basic Hessian derivative with backend=$(backend) -- $info -- T=$(T)" for T in (
-    Float32,
    Float64,


The Float32 test case was removed, so this test no longer exercises sparse Hessian behavior under single precision. If Float32 is still expected to work for the non-Enzyme backends, please keep Float32 in this loop (or gate it only for the backends/types that cannot support it) to avoid losing coverage.

Suggested change

Float64,

Float64,

Float32,

Copilot · 2026-02-13T18:03:09Z


    # Test also the implementation of the backends
    b = nlp.adbackend.hessian_backend
+    @show b


@show b will print backend internals during normal test runs and can make CI logs noisy. Please remove this debug output (or guard it behind a debug flag).

Suggested change

@show b

Copilot · 2026-02-13T18:03:09Z

+      # res = 0
+      # n = length(x)
+      # for i in 1:(n-1)
+      #   res += 100 * (x[i + 1] - x[i]^2)^2 + (x[i] - 1)^2
+      # end
+      # res


Large blocks of commented-out implementation code inside the test function make it harder to maintain and review. Please remove the commented-out loop version (or convert it into a dedicated alternative implementation used by a test if it’s still needed).

Suggested change

# res = 0

# n = length(x)

# for i in 1:(n-1)

# res += 100 * (x[i + 1] - x[i]^2)^2 + (x[i] - 1)^2

# end

# res

Copilot · 2026-02-13T18:03:09Z

+      # x -> sum(100 * (x[i + 1] - x[i]^2)^2 + (x[i] - 1)^2 for i = 1:(n - 1)),
+      # x -> sum(100 * (x[i + 1] - x[i]^2)^2 + (x[i] - 1)^2 for i = 1:3),
+      # x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2,


The commented-out alternative objective definitions around the ADNLPModel construction add noise and make it unclear which definition is intended. Please remove these commented lines unless they are needed for a specific, active test case.

Suggested change

# x -> sum(100 * (x[i + 1] - x[i]^2)^2 + (x[i] - 1)^2 for i = 1:(n - 1)),

# x -> sum(100 * (x[i + 1] - x[i]^2)^2 + (x[i] - 1)^2 for i = 1:3),

# x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2,

Copilot · 2026-02-13T18:03:09Z

+const test_enzyme = true
+


const test_enzyme = true appears to be unused in the test suite (no other references). If it’s not required for conditional logic in the included files, consider removing it to avoid dead test configuration state.

Suggested change

const test_enzyme = true

Copilot · 2026-02-13T18:03:10Z

-        Enzyme.Const(Enzyme.gradient!),
-        Enzyme.Const(Enzyme.Reverse),
+      copyto!(b.xbuf, x)
+      _hvp!(Enzyme.DuplicatedNoNeed(b.grad, Hv), b.f, b.xbuf, v)


In the :obj Hvprod! method, Float64[] is passed as the y argument. This hard-codes Float64 and can introduce unnecessary specialization/type-instability (and may break if callers use Float32/BigFloat/etc.). Prefer passing an empty vector matching eltype(x) (e.g., similar(x, 0) or a preallocated empty buffer stored on the backend).

Suggested change

_hvp!(Enzyme.DuplicatedNoNeed(b.grad, Hv), b.f, b.xbuf, v)

_hvp!(Enzyme.DuplicatedNoNeed(b.grad, Hv), b.ℓ, b.xbuf, v, similar(x, 0), obj_weight, b.cx)

Copilot · 2026-02-13T18:03:10Z

      hess = zeros(T, n, n)
      fill!(b.seed, zero(T))
      for i = 1:n
        b.seed[i] = one(T)
-        Enzyme.hvp!(b.Hv, Enzyme.Const(f), x, b.seed)
+        grad = Enzyme.make_zero(x)
+        _hvp!(Enzyme.DuplicatedNoNeed(grad, b.Hv), f, x, b.seed)
        view(hess, :, i) .= b.Hv


hessian(::EnzymeReverseADHessian, ...) allocates grad = Enzyme.make_zero(x) inside the for i = 1:n loop, which adds O(n) allocations per Hessian call. Consider allocating/reusing this buffer once (e.g., store it on the backend struct) to reduce allocation overhead, especially for larger n.

Copilot · 2026-02-13T18:03:10Z

 [deps]
+Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9"
 ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"


ADNLPModels and Enzyme were added to [deps], but there are no corresponding [compat] entries. Since this test project already pins compat for other deps, adding compat bounds here would help keep CI reproducible and avoid breakages from upstream Enzyme releases.

Copilot · 2026-02-13T18:03:10Z

+    # n = 4
+    x0 = ones(T, 4)
+    function f(x)


n = 4 was removed/commented out here, but the test file still uses n later (in the matrix_free Extended Rosenbrock model). As written this will cause UndefVarError: n not defined when that later block executes; either reintroduce n or replace those generator bounds with length(x).

codecov · 2026-02-13T20:04:56Z

Codecov Report

❌ Patch coverage is 0% with 98 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.75%. Comparing base (fc3ab2f) to head (cf5f009).
⚠️ Report is 36 commits behind head on main.

Files with missing lines	Patch %	Lines
src/enzyme.jl	0.00%	98 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #322       +/-   ##
===========================================
- Coverage   92.23%   76.75%   -15.48%     
===========================================
  Files          13       13               
  Lines        1597     1876      +279     
===========================================
- Hits         1473     1440       -33     
- Misses        124      436      +312

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

This was referenced Feb 1, 2025

Manual sparse pattern for Jacobian and Hessian control-toolbox/CTDirect.jl#348

Merged

Use show_time for sparse AD backends #326

Merged

michel2323 force-pushed the ms/enzyme branch from c9af69b to e5777a9 Compare February 13, 2026 16:03

michel2323 added 2 commits February 13, 2026 11:29

Enzyme support

ae34199

Resolve SubArray issue with buffer

3e3acc0

michel2323 force-pushed the ms/enzyme branch from eb903fa to 3e3acc0 Compare February 13, 2026 17:29

michel2323 added 2 commits February 13, 2026 11:51

jtprod fix

8a8d0cf

Fix Hvprod

f049d4e

michel2323 marked this pull request as ready for review February 13, 2026 17:57

michel2323 requested review from amontoison and Copilot February 13, 2026 17:57

Copilot started reviewing on behalf of michel2323 February 13, 2026 17:57 View session

michel2323 added 2 commits February 13, 2026 12:02

Fix

2b9afb8

Cleanup

caac03f

Copilot AI reviewed Feb 13, 2026

View reviewed changes

More fixes

7f46cdb

Apply suggestions from code review

cf5f009

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

amontoison approved these changes Feb 14, 2026

View reviewed changes

amontoison merged commit adaffab into JuliaSmoothOptimizers:main Feb 14, 2026
28 of 30 checks passed

amontoison deleted the ms/enzyme branch February 14, 2026 01:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enzyme 2nd order fix#322

Enzyme 2nd order fix#322
amontoison merged 8 commits into
JuliaSmoothOptimizers:mainfrom
michel2323:ms/enzyme

michel2323 commented Jan 30, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

codecov Bot commented Feb 13, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	# x -> sum(100 * (x[i + 1] - x[i]^2)^2 + (x[i] - 1)^2 for i = 1:(n - 1)),
	# x -> sum(100 * (x[i + 1] - x[i]^2)^2 + (x[i] - 1)^2 for i = 1:3),
	# x -> 100 * (x[2] - x[1]^2)^2 + (x[1] - 1)^2,

	_hvp!(Enzyme.DuplicatedNoNeed(b.grad, Hv), b.f, b.xbuf, v)
	_hvp!(Enzyme.DuplicatedNoNeed(b.grad, Hv), b.ℓ, b.xbuf, v, similar(x, 0), obj_weight, b.cx)

Conversation

michel2323 commented Jan 30, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Feb 13, 2026 •

edited

Loading