how to further reduce residual error in an unconstrained levenberg-marquardt optimisation

I am running an optimisation function using unconstrained levenberg-marquardt. The algorithm converges and gives a residual of 412.192 in the 11th iteration. Is there a way in which I can futher reduce this residual error?? Many Thanks.

 if true
                                           First-Order                    Norm of 
 Iteration  Func-count    Residual       optimality      Lambda           step
     0           3         35865.3        2.85e+04         0.01
     1           6         1450.89        2.76e+03        0.001        137.848
     2           9         415.676             229       0.0001        458.723
     3          12         412.195             2.5        1e-05        4.34834
     4          15         412.192           0.221        1e-06        1.42228
     5          18         412.192          0.0141        1e-07      0.0867978
     6          21         412.192        0.000922        1e-08      0.0057553
     7          24         412.192         4.6e-05        1e-09    0.000382006
     8          27         412.192        7.44e-06        1e-10    2.42769e-05
     9          35         412.192        3.43e-05        1e-05     4.1357e-06
    10          41         412.192        0.000123         0.01    1.21969e-05
    11          51         412.192        0.000117       100000    1.66032e-11
  end

Answer 1

[Alan Weiss]

The Lambda parameter climbs to a very high value at the end. I wonder if your function is smooth.

Did you try starting at various start points? Did you formulate your problem correctly, passing in the components of your objective vector, or did you erroneously pass in the sum of squares? Without more details it is hard to know what to advise you to do.

Alan Weiss

MATLAB mathematical toolbox documentation

[Anuj Anand]

Here is the function. i also tried plotting the range of values over which I expect the correct solution (i.e. k(1)=[3000 6000] and k(2)=[0.01 5]) and plotted this function (see attached). It probably does not look like if there are more solution. But perhaps, you can suggest better. Many Thansk

 if true
    function [ a , b , sse ] = measureMT( signal , alpha , Rin )
Rin = 0.00314;
signal=[168.818176,168,167.969696,168.030304,168.212128,349.363647,349.84848,349.757568,349.878784,350.363647,433.818176,433.060608,432.909088,433.545441,432.727264,319.84848,318.727264,319.727264,318.84848,319.121216];
alpha =[2,2,2,2,2,5,5,5,5,5,10,10,10,10,10,18.59634,18.59634,18.59634,18.59634,18.59634];
format long
ahat = zeros(2,2); sse = zeros(1,2);
options = optimoptions(@lsqcurvefit,... 'Algorithm','levenberg-marquardt','Display','iter',... 'FunValCheck','on','ScaleProblem','Jacobian',... 'TolFun',1e-10,'TolX',1e-10,'MaxFunEvals',1e10,'MaxIter',1e10);
for i=1:1:2 x0 = [5000/i i/3.92] ; [ahat(:,i), sse(i)] = ... lsqcurvefit(@(x,xdata) cal(x,alpha,Rin),x0,alpha,signal,[],[],options);
      figure, plot(alpha,signal,'o'), hold on
      plot(alpha,cal(ahat,alpha,Rin),'-r')
  end
  a=ahat(1,:);   b=ahat(2,:); 
function F = cal(k,xdata,Rin) % MODEL 
   F = k(1)*sin(xdata*pi/180)*(1-exp(-Rin*k(2)))./(1-(cos(xdata*pi/180)*exp(-Rin*k(2)))); 
end
end
  end

Answer 2

[Anuj Anand]

Dear Alan, Many Thanks again. I did try various start points, but it still gives very large residual. However, i cannot do too much with it as i have to use un-constrained levenberg-marquardt. So the initial condition that I am giving is not very far from the solution anyways (i think). I dont understand what do you mean by if i am formulating my problem correctly? I mean there is an non-linear equation which act as a model defined in function 'cal' which I have copy pasted from a paper so it is hard to fo wrong with it. Many Thanks for your help again.

function [ a , b , sse ] = measureMT( signal , alpha , Rin )

Rin = 0.00314;

signal=[168.818176,168,167.969696,168.030304,168.212128,349.363647,349.84848,349.757568,349.878784,350.363647,433.818176,433.060608,432.909088,433.545441,432.727264,319.84848,318.727264,319.727264,318.84848,319.121216];

alpha =[2,2,2,2,2,5,5,5,5,5,10,10,10,10,10,18.59634,18.59634,18.59634,18.59634,18.59634];

format long

ahat = zeros(2,2); sse = zeros(1,2);

options = optimoptions(@lsqcurvefit,... 'Algorithm','levenberg-marquardt','Display','iter',... 'FunValCheck','on','ScaleProblem','Jacobian',... 'TolFun',1e-10,'TolX',1e-10,'MaxFunEvals',1e10,'MaxIter',1e10);

for i=1:1:2 x0 = [5000/i i/3.92] ; [ahat(:,i), sse(i)] = ... lsqcurvefit(@(x,xdata) cal(x,alpha,Rin),x0,alpha,signal,[],[],options);

      figure, plot(alpha,signal,'o'), hold on
      plot(alpha,cal(ahat,alpha,Rin),'-r')
  end
  a=ahat(1,:);   b=ahat(2,:); 

function F = cal(k,xdata,Rin) % MODEL 

   F = k(1)*sin(xdata*pi/180)*(1-exp(-Rin*k(2)))./(1-(cos(xdata*pi/180)*exp(-Rin*k(2)))); 

end

end

[Matt J]

If you are responding to Alan's answer, you should do so in the Comment boxes underneath that answer.

As for improving the initial guess, since you only have 2 unknown parameters, you could probably just make a SURF plot of norm(F) over some suitable region of k(1), k(2). Then you could view graphically whether there are more globally optimal solutions than the one you have now.

[Anuj Anand]

Many Thanks for your suggestions. I will follow them and see what I get.

[Anuj Anand]

Hi, i tried these range of values for k(1)=[3000 6000] and k(2)=[0.01 5] and obtained the figure attached. I doubt if there are more global solution. What do you suggest?

[Matt J]

Your attachment didn't make it, but if the figure shows that you are at the global minimum, doesn't that mean the problem is resolved? Your original concern was that you weren't sure if you had fully minimized the function.

[Anuj Anand]

Many Thanks again for your replly. I just wanted to be sure and that is why I am asking. I am not very experiences with it and I was surprised to know that if my solution has reached global minima then why should this residual error be so high?