supplied Hessian of fmincon

Hi,

I have a question and would be happy if anyone can answer. I have a simple optimization problem with only 2 variables and 2 equality constraints. When I compare the Hessian of the Lagrangian function computed by hand with that supplied by fmincon, I observe a huge difference. I know that fmincon uses the SQP algorithm and the BFGS formula to compute the Hessian and this Hessian is an approximation to the Hessian of Lagrangian. However, since this is an approximation, it seems to be close to the accurate value.

Please see the difference :

Hessian provided by fmincon =

    [2.2313    8.7213
     8.7213   60.3726]
Hessian solved by hand  =
   [-0.3666        0
       0          -0.0046]

Please note that the optimal value for x and obj function are same just different in Hessian. I got the same thing for another problem. I am wondering what this difference is for?

Thanks in advance, Mary

[Andrew Newell]

@Mary, how can we know the reason if you don't tell us what the function is and how you are calculating the Hessian by hand?

[Mary dkh]

Andrew!

You are right, but I suppose this difference has a general reason which happens to all kind of optimization problems.

However, my problems is as follow:

X0 = [50; 50]; %Initialize

a = [3;5]

b = [7;11]

c = [15;19]

obj: f = (a1*X(1)^2 + a2*X(2)^2);

constraints:

g1 = b1 *X(1)^2 + b2 *X(2)^2 - 300;

g2 = c1 *X(1) + c2 *X(2) - 100;

I wrote the problem in MATLAB and by using fmincon, I got the following optimum solutions:

X =

5.1924e-002

5.2222e+000

I also have computed my hessian of lagrangian function with respect to X and lambda(lambda is lagrangian multipliers for equality constraints) and write in a general format in Matlab after calling fmincon as following:

Hessian = [ [ [2*a1-2*b1*LM(1)] [0] [-2*b1*X(1)] [-c1] ];

[ 0] [2*a2-2*b2*LM(1)] [-2*b2*X(2)] [-c2] ];

[ [-2*b1*X(1)] [-2*b2*X(2)] [0] [0] ];

[ [-c1] [-c2] [0] [0] ]; ];

so by applying the optimal solution to this Hessian matrix and compare that with the one obtained by fmincon we observe that difference.

Please note that we should compare only Hessian(1:2,1:2) with that of fmincon.

Now can anyone tell me what this difference is for?

Thanks

Answer 1

[Steve Grikschat]

As you stated, fmincon provides a BFGS approximation to the Hessian of the Lagrangian. However, the BFGS matrix is a positive definite matrix based on the local curvature of the problem.

But note, it's always positive definite and your Hessian (from the looks of it) is not necessarily so. This matrix should not be used for any comparisons or computations outside of the optimization.

[Mary dkh]

Thanks for the answer. However, by the term of "approximation" I did not expect that large differennce between two hessians.

[Steve Grikschat]

That's understandable. Maybe it should be called an "iteration matrix" or some other term to avoid confusion.