fmincon with trust-region-reflective algorithm converges but interior point doesn't

Hello all optimization experts ...

I'm trying to do non-linear fitting of a large model to some data under boundary and linear inequality constraints on some of the model variables (in the order of 10^3-10^4 variables). I have analytic gradients and hessians, and have been playing with the parameters a lot, but still cannot manage to get the interior-point algorithm in fmincon to converge.

trust-region-reflective works fine, but with it I can only enforce boundary, not the inequality constraints. I also tried the interior point algorithm without the inequality constraints and get the same behavior: after a number of good-looking optimization steps, the cost function decreases only very little, but the step-size (of the i.p.-algorithm) oscillates between consecutive steps (increase - decrease - increase - decrease). This does not happen in the trust-region algorithm. Has anybody maybe experienced similar behavior or can point me to where to look for what's causing this?

thanks a lot!

[Steve Grikschat]

What exactly does I-P return? Does it declare the problem infeasible? Does it time out? It might help others to post some output snippets.

Is the solution of the problem with only bounds on the variables near the bounds? If so, that might help explain why interior-point takes small steps towards the end.

From the name, you can tell that the interior-point algorithm wants to remain on the interior of the bounded region. If the solution is near a boundary, it will approach slowly and may never fully make it there. By construction, the inner workings of the I-P algorithm are unstable near constraint boundaries.

The above may also explain the lack of convergence for the inequality constrained formulation.

[Ernst Schwartz]

ahh of course! thank you so much, that was exactly the hint I needed. I need to look at the specifications of my problem a bit more closely, but it really seems in fact that the optimiser getting too close to the bounds was the problem. Thank you so much!

Answer 1

[Steve Grikschat]

What exactly does I-P return? Does it declare the problem infeasible? Does it time out? It might help others to post some output snippets.

Is the solution of the problem with only bounds on the variables near the bounds? If so, that might help explain why interior-point takes small steps towards the end.

From the name, you can tell that the interior-point algorithm wants to remain on the interior of the bounded region. If the solution is near a boundary, it will approach slowly and may never fully make it there. By construction, the inner workings of the I-P algorithm are unstable near constraint boundaries.

The above may also explain the lack of convergence for the inequality constrained formulation.