How to pass preconditioner?

[cossio]

I found very little documentation about this.

https://nlopt.readthedocs.io/en/stable/NLopt_Reference/#preconditioning-with-approximate-hessians

But that only explains the C interface, and says that only LD_CCSAQ supports preconditioners.

How is the interface in Julia? Is it possible to pass a preconditioner to LBFGS? Can I pass a sparse Hessian approximation (e.g., a diagonal matrix)?

See also https://discourse.julialang.org/t/how-to-pass-a-preconditioner-to-nlopt/16361

[giadasp]

Dear Cossio,
Did you solve your problem?
I'm interested in this issue as well.
Do you use JuMP or NLopt directly?
I would like to give gradients to my objective function using the JuMP interface language but I'm not able to do it. Do you know how to do it?

Thank you for your attention

Giada

[cossio]

@giadasp I did not figure out how to pass a preconditioner to NLopt. But Optim has a documented API for this purpose, https://github.com/JuliaNLSolvers/Optim.jl,

[odow]

#223 exposes the necessary C API for this
https://github.com/JuliaOpt/NLopt.jl/blob/6fd18707b633978bfb3070b8fa514adcd9fe5289/src/libnlopt.jl#L143-L149
but I don't have an example of exactly what this should take.

[odow]

Here's an example:

using NLopt, Test
begin
    function nlopt_scalar_callback(n, p_x, p_grad, f_data)
        data = unsafe_pointer_to_objref(f_data)::Preconditioner
        x = unsafe_wrap(Array, p_x, (n,))
        if p_grad !== C_NULL
            return data.obj_callback(x, unsafe_wrap(Array, p_grad, (n,)))
        end
        return data.obj_callback(x, Cdouble[])
    end 
    function nlopt_preconditioner_callback(n, p_x, p_v, p_vpre, f_data)
        data = unsafe_pointer_to_objref(f_data)::Preconditioner
        x = unsafe_wrap(Array, p_x, (n,))
        v = unsafe_wrap(Array, p_v, (n,))
        vpre = unsafe_wrap(Array, p_vpre, (n,))
        data.preconditioner(vpre, x, v)
        return
    end
    mutable struct Preconditioner
        obj_callback::Function
        preconditioner::Function
    end
    Base.unsafe_convert(::Type{Ptr{Cvoid}}, p::Preconditioner) = pointer_from_objref(p)
    function precond_max_objective!(opt::NLopt.Opt, obj_fn, precond_fn)::NLopt.Result
        preconditioner = Preconditioner(obj_fn, precond_fn)
        c_scalar_callback = @cfunction(
            nlopt_scalar_callback,
            Cdouble, 
            (Cuint, Ptr{Cdouble}, Ptr{Cdouble}, Ptr{Cvoid}),
        )
        c_precond_callback = @cfunction(
            nlopt_preconditioner_callback, 
            Cvoid, 
            (Cuint, Ptr{Cdouble}, Ptr{Cdouble}, Ptr{Cdouble}, Ptr{Cvoid}),
        )    
        return NLopt.nlopt_set_precond_max_objective(
            opt,
            c_scalar_callback,
            c_precond_callback,
            preconditioner,
        )
    end
end
opt = Opt(:LD_CCSAQ, 2)
lower_bounds!(opt, 0.0)
upper_bounds!(opt, 2.0)
xtol_rel!(opt, 1e-4)
function my_objective_fn(x, grad)
    if length(grad) > 0
        grad[1] = 1.0
        grad[2] = 0.5 / sqrt(x[2])
    end
    return x[1] + sqrt(x[2])
end
function my_preconditioner(vpre, x, v)
    vpre[1] = 0.0
    vpre[2] = v[2] / (2 * sqrt(x[2]))
    return
end
precond_max_objective!(opt, my_objective_fn, my_preconditioner)
min_f, min_x, ret = optimize(opt, [1.0, 1.0])
@test min_f ≈ 2.0 + sqrt(2.0)
@test min_x ≈ [2.0, 2.0]
@test ret == :XTOL_REACHED

Given the docs

Currently, support for preconditioners in NLopt is somewhat experimental, and is only used in the NLOPT_LD_CCSAQ algorithm

I don't know if we need to provide proper support for this.

Although MOI has support for Hessian vector products, so in theory we could hook this up automatically through JuMP.

[odow]

Closing this as won't fix. My post above shows that it's possible to implement this from Julia. Since it is somewhat niche, I don't know that we need to document it. If anyone has a strong opinion to the contrary, please comment below and I can re-open this issue.