LD_MMA unexpected solution for simple problem

[OskarBergmann]

The following code is supposed to minimise sqrt(x[2]) s.t. x[1]>2, x[2]>a, initial guess [1.234, 5.678]. (There are infinitely many solutions).

With a = 8, a correct solution is found - FTOL_REACHED
With a = 6, the solution is wrong (x[2]opt = 6.07, when it should be 6) - MAXEVAL_REACHED
With a = 5, the solution is wrong (x[2]opt = 5.678, when it should be 5) - FTOL_REACHED

Note: when I exchange LD_MMA with LD_SLSQP, the correct solution is always found

using NLopt
import ForwardDiff

function autodiff(f::Function)
    function nlopt_fn(x::Vector, grad::Vector)
        if length(grad) > 0 # modify grad in place if it is passed
           ForwardDiff.gradient!(grad, f, x)
        end
        return f(x) # return f(x) in all cases
    end
end # return the function nlopt_fn

function my_objective_fn(x::Vector, grad::Vector)
           if length(grad) > 0
               grad[1] = 0
               grad[2] = 0.5 / sqrt(x[2])
           end
           return sqrt(x[2])
end

function con1(x::Vector)
    return 2-x[1]
end

function con2(x::Vector,a)
    return a-x[2]
end


opt = NLopt.Opt(:LD_MMA, 2)
NLopt.lower_bounds!(opt, [-Inf, 0.5])
NLopt.xtol_rel!(opt, 1e-8)
NLopt.ftol_abs!(opt, 1e-8)
NLopt.min_objective!(opt, my_objective_fn)
NLopt.maxeval!(opt, 500)

NLopt.inequality_constraint!(opt, autodiff(con1), 1e-8)
NLopt.inequality_constraint!(opt, autodiff(con2), 1e-8)

NLopt.inequality_constraint!(opt, autodiff(con1), 1e-8)
a = 5.5
NLopt.inequality_constraint!(opt, autodiff(x -> con2(x,a)), 1e-8)

min_f, min_x, ret = NLopt.optimize(opt, [1.234, 5.678])

[odow]

It's helpful to show some logging:

julia> using NLopt

julia> function example_270(a)
           function my_objective_fn(x::Vector, grad::Vector)
               @show x
               if length(grad) > 0
                   grad[1] = 0.0
                   grad[2] = 0.5 / sqrt(x[2])
               end
               return sqrt(x[2])
           end
           function con1(x::Vector, grad)
               if length(grad) > 0
                   grad[1] = -1.0
                   grad[2] = 0.0
               end
               return 2 - x[1]
           end
           function con2(x::Vector, grad)
               if length(grad) > 0
                   grad[1] = 0.0
                   grad[2] = -1.0
               end
               return a - x[2]
           end
           opt = NLopt.Opt(:LD_MMA, 2)
           NLopt.lower_bounds!(opt, [-Inf, 0.5])
           NLopt.xtol_rel!(opt, 1e-8)
           NLopt.ftol_abs!(opt, 1e-8)
           NLopt.min_objective!(opt, my_objective_fn)
           NLopt.maxeval!(opt, 500)
           NLopt.inequality_constraint!(opt, con1, 1e-8)
           NLopt.inequality_constraint!(opt, con2, 1e-8)
           min_f, min_x, ret = NLopt.optimize(opt, [1.234, 5.678])
       end
example_270 (generic function with 1 method)

julia> example_270(5)
x = [1.234, 5.678]
x = [1.615966011250105, 5.678]
(2.382855429941145, [1.615966011250105, 5.678], :FTOL_REACHED)

julia> example_270(6)
x = [1.234, 5.678]
x = [1.5019491924311228, 5.945949192431122]
x = [2.229909055324816, 6.071077205214923]
... lines omitted...
x = [2.7996583166744052e6, 2.799662157842555e6]
... lines omitted...
x = [2.0291857093269866e39, 2.0291857093269866e39]
(2.463955601307565, [2.229909055324816, 6.071077205214923], :MAXEVAL_REACHED)

julia> example_270(8)
x = [1.234, 5.678]
x = [1.5019491924311228, 5.945949192431122]
x = [2.143691622935539, 6.587691622935538]
x = [2.143691622935539, 7.667691622935538]
x = [2.143691622935539, 8.099776403148532]
x = [2.143691622935539, 8.005446782331946]
x = [2.143691622935539, 8.000024431966695]
x = [2.143691622935539, 7.999999961628959]
x = [2.143691622935539, 8.000000007823648]
(2.828427117963084, [2.143691622935539, 7.999999961628959], :XTOL_REACHED)

This seems to be working as expected. In the first case, it steps only in the x[1] direction, so the objective hasn't changed between iterations and FTOL is hit.

In the second case, the algorithm diverges but it returns the best solution found to date.

In the third case, it works as expected.

[stevengj]

The algorithm is not guaranteed to converge unless you give it a feasible starting point. (This is true of many local-optimization algorithms.)

[OskarBergmann]

Thank you for the clarification.

[odow]

Closing because I don't think there's any thing to do here.

If you have more questions (fellow Os[ck]ar), come and chat over on our Discourse forum: https://discourse.julialang.org/c/domain/opt/13