Stopping fminsearch immediately?

So I have fminsearch running a custom function func, with outputs A, B, and C: [A, B, C] = func(x, y, z).

I used the function fminsearch to optimize the parameters x, y, and z. My problem is that it takes too long as it gets stuck on invalid trial values for x, y, and z. I was able to determine that this happens when the trial results A, B, and C are very large. While I know that fminsearch would self-correct and that it would just re-assign values of x, y, and z, I find this very slow and I wanted to just exit fminsearch.

Therefore, I wanted it such that fminsearch would stop when A >= 1000, B >= 500, and C >= 500. I placed this on func(x, y, z):

if A >= 1000 || B >= 500 || C >= 500
    A = nan
    B = nan
    C = nan
    error('Poor x, y, and z values. Re-iterating.')
end

Sadly, while the function would exit, fminsearch would still try to continue solving the problem. Is there a way to stop fminsearch from doing this, or to accomplish my requirement that fminsearch would stop when it calculates A >= 1000 || B >= 500 || C >= 500?

Thank you very much.

Answer 1

[Walter Roberson]

Use the options to invoke an outputFcn https://www.mathworks.com/help/matlab/math/output-functions.html

Also you can specify MaxFunEvals and MaxIter

[Allan Paolo]

Hi, I've tried something and now I'm at a state of confusion and a total loss :(

I don't even know if I am doing this correctly; I need more examples, probably.

This is what I have now:

For my outputfcn:

function stop = output_opt(x, A, B, C)
stop = false;
if (A >= 1000 || B >= 500 || C >= 500)
    return
end

For my function, func:

function [Z, A, B, C] = func(x)
A = x(1) .^ 2 + 2 .* x(2) .^ 2 - x(3) .^ 2
B = x(1) + x(2) + x(3) .^  2
C = -x(1) .^ 2 - 3 .* x(2) .^ 2 + x(3)
Z = sqrt(A .^ 2 + B .^ 2 + C .^ 2)
end

So I ran the code in fminsearch. What I did is I intended to trigger the conditions such that fminsearch would be cancelled via return(? not sure), thus the 1000 values.

options = optimset('OutputFcn', @output_opt)
[x, Z] = fminsearch(@(x)func(x),[1000, 1000, 1000], options)

What happened is that it printed the error: Undefined operator '>=' for input arguments of type 'struct'.

[Allan Paolo]

Hi, I made progress.

Apparently A is a struct that contains information from the iteration. So I just re-ran func(x) in outputfcn.

What I did is this:

function stop = output_opt(x, optimValues, state)
stop = false;

[Z, A, B, C] = func(x)
if (A >= 1000 || B >= 500 || C >= 500)
    error('stop!')
end
end

It does stop the program by chucking out an error since A = 2000000, B = 1002000, and C = -3999000. So on paper, the solution 'worked'.

My issue is, is there another way to extract A, B, and C from the iteration in determining Z, without re-evaluating func(), then use these values to evaluate OutputFcn?

As far as I understand, fminsearch calculates the value of Z using trial values first, then evaluates outputfcn as a constraint. But to calculate Z, it needs to calculate A, B, and C first, from func(). In my real code, func() is very, very expensive to evaluate, so I am trying to avoid evaluating func() twice.

I was thinking that maybe there is a way to extract A, B, and C from the first iteration (the calculation of Z). Is there a means for me to do this?

As a side question, I tried getting rid of optimValues and state but it also chucks out an error. Do I just ignore these things?

[Walter Roberson]

format long g
options = optimset('OutputFcn', @output_opt, 'display', 'iter');
[x, Z] = fminsearch(@func,[25, 15, 15], options)

 
 Iteration   Func-count     min f(x)         Procedure
     0            1          1563.31         
     1            4          1554.38         initial simplex
     2            6          1474.93         expand
     3            8          1308.78         expand
     4            9          1308.78         reflect
     5           11          1079.27         expand
     6           13          852.175         expand
     7           15          693.181         expand
     8           16          693.181         reflect
     9           17          693.181         reflect
    10           19          682.423         contract outside

x = 1×3

4.53703703703702 10.5833333333333 22.6944444444444

A = 
         -270.439214677641

B = 
          530.158179012346

C = 
         -333.911093964335

Optimization terminated prematurely by user.

x = 1×3

4.53703703703702 10.5833333333333 22.6944444444444

Z = 
          682.423389313118

function Z = func(x)
A = x(1) .^ 2 + 2 .* x(2) .^ 2 - x(3) .^ 2;
B = x(1) + x(2) + x(3) .^  2;
C = -x(1) .^ 2 - 3 .* x(2) .^ 2 + x(3);
Z = sqrt(A .^ 2 + B .^ 2 + C .^ 2);
end

function stop = output_opt(x, optimValues, state)
    A = x(1) .^ 2 + 2 .* x(2) .^ 2 - x(3) .^ 2;
    B = x(1) + x(2) + x(3) .^  2;
    C = -x(1) .^ 2 - 3 .* x(2) .^ 2 + x(3);
    stop = (A >= 1000 || B >= 500 || C >= 500);
    if stop
       x, A, B, C
    end
end

You cannot pass anything from func to the output function

[Walter Roberson]

My issue is, is there another way to extract A, B, and C from the iteration in determining Z, without re-evaluating func(), then use these values to evaluate OutputFcn?

Create a function that calculates A, B, C from x. memoize() the function. Call the function from both places. https://www.mathworks.com/help/matlab/ref/memoize.html

I was thinking that maybe there is a way to extract A, B, and C from the first iteration (the calculation of Z). Is there a means for me to do this?

NO.

As a side question, I tried getting rid of optimValues and state but it also chucks out an error. Do I just ignore these things?

Yes, just ignore them, possibly using the ~ parameter in the function definition.

Note: when ask to stop, the return value from fminsearch() will be the x that led to the stop -- the first x that was out of range, rather than the last x that was in-range.

To get the last x that was in-range, use a shared variable that you update each time stop is false, and after the function returns, fetch the shared variable.

[Allan Paolo]

So probably, this is the best that I can muster:

function stop = output_opt(x, ~, ~)
[~, A, B, C] = func(x)

stop = (A >= 1000 || B >= 500 || C >= 500)
if stop
end
end

Thank you very much!

[Walter Roberson]

ABC_memo = memoize(@ABC);
ABC_memo.CacheSize = 2;

options = optimset('OutputFcn', @(x,~,~) output_opt(x,ABC_memo), 'display', 'iter');
[x, Z] = fminsearch(@(x)func(x,ABC_memo),[25, 15, 15], options)

 
 Iteration   Func-count     min f(x)         Procedure
     0            1          1563.31         
     1            4          1554.38         initial simplex
     2            6          1474.93         expand
     3            8          1308.78         expand
     4            9          1308.78         reflect
     5           11          1079.27         expand
     6           13          852.175         expand
     7           15          693.181         expand
     8           16          693.181         reflect
     9           17          693.181         reflect
    10           19          682.423         contract outside
Optimization terminated prematurely by user.

x = 1×3

4.5370 10.5833 22.6944

Z = 682.4234

function Z = func(x, ABC_memo)
  [A, B, C] = ABC_memo(x);
  Z = sqrt(A .^ 2 + B .^ 2 + C .^ 2);
end

function stop = output_opt(x, ABC_memo)
[A, B, C] = ABC_memo(x);
stop = (A >= 1000 || B >= 500 || C >= 500);
end

function [A,B,C] = ABC(x)
    A = x(1) .^ 2 + 2 .* x(2) .^ 2 - x(3) .^ 2;
    B = x(1) + x(2) + x(3) .^  2;
    C = -x(1) .^ 2 - 3 .* x(2) .^ 2 + x(3);
end

[Allan Paolo]

Hi, sorry for bumping again, but I have this issue.

Say for example I now have func() requiring two inputs (though I am only optimizing with respect to x, and I do not want to optimize y):

options = optimset('OutputFcn', @output_opt);
y = 200;
[x, Z] = fminsearch(@(x)func(x, y),[1000, 1000, 1000], options)

The function looks something like this now:

function [Z, A, B, C] = func(x, y)
A = x(1) .^ 2 + 2 .* x(2) .^ 2 - x(3) .^ 2 + y;
B = x(1) + x(2) + x(3) .^  2 + y;
C = -x(1) .^ 2 - 3 .* x(2) .^ 2 + x(3) + y;
Z = sqrt(A .^ 2 + B .^ 2 + C .^ 2);
end

What I did is:

function stop = output_opt(x, y, ~, ~)
[~, A, B, C] = func(x, y)

stop = (A >= 1000 || B >= 500 || C >= 500)
if stop
end
end

As I predicted, this would cause an error as the second item in the input is reserved for optimValues (the struct error again).

Is there a method for me to be able to input y, without involving memoize? After that, I'll try doing your advise regarding memoize.

Thanks!

[Walter Roberson]

y = 200;
options = optimset('OutputFcn', @(x,optimValues,state)output_opt(x,optimValues,state,y);
[x, Z] = fminsearch(@(x)func(x, y),[1000, 1000, 1000], options)

[Walter Roberson]

[~, A, B, C] = func(x, y)

That requires recalculating everything that was calculated in func(). Using memoize() would avoid that. memoize() acts as a cache.

if stop
end

That is not needed.