Controlling stepsize of fmincon

Hi

Im am trying to shape optimize a heating surface using MATLAB fmincon and a CFD software. My problem is that the computational costs of a function evaluation (ie a CFD analysis) is very high and thus I want to make a rater rough search of my solution space.

When I iniate fmincon it seems that some ver small steps (in the order of 10^-5 of a mm) are done in order to determine a search direction. This is a waste of function evalutaion since this does not changes the results at all.

My question: Is their any way to avoid these useless function evalution and start out by making large steps?

[Matt J]

If you know that the small steps are done in order to determine a search direction, then why would you consider them useless or wasteful? Isn't determining a search direction important? How can the algorithm make any progress without knowing in which direction to step? Rather than saying that they are "useless", I think what you really mean to say is that they are "costly".

Answer 1

[Richard Alcock]

Take a look at the documentation for fmincon: http://www.mathworks.com/help/toolbox/optim/ug/fmincon.html#f854721 and the options available for the optimization functions: http://www.mathworks.com/help/toolbox/optim/ug/f19175.html

Reading those, I think you want to set the DiffMinChange option to something larger than it's default.

DiffMinChange Minimum change in variables for finite-difference gradients (a positive scalar). The default is 0

[Pablo]

DiffMinChange not define de mínimum step size

[Sean de Wolski]

@Pablo, all though it does not define a minimum step size it does encourage fmincon() to look further away where the gradient might be steeper. This will cause it to take bigger steps.

Answer 2

[Radomir]

Hello,

I understand that this topic is no longer up to date, but I believe that it could still help to somebody.

Solving similar problem (also CFD software ivolved) I used FiniteDifferenceStepSize as an optimset parametr to avoid small parameters increase.

Radomir

[Punnag Chatterjee]

Can you share part of the code where you define define the option = optimset(.....)? Also general ideas/tips on how to select an appropriate FiniteDifferenceStepSize ?

[AM]

Hello,

I am using fmincon with an expensive objective function (Chemical software involved) and I was wondering if you had more information about how how to choose FiniteDifferenceStepSize to use it as an optimiset parameter.

Thank you

[Matt J]

Adjusting FiniteDifferenceStepSize is not a way to decrease computation (in spite of what was said above). To avoid the cost of finite difference computations, you should provide your own gradient calculation like in this example.

[AM]

I am using an external software to determine the points given by the model and then comparing them to experimental points so I do not have an analytical expression of my objective function. This is why I cannot provide the gradient, and my code takes a long time to run.

Is there any way to reduce the number of function evaluations?

[Matt J]

You can only reduce the number of finite differencing function evaluations by somehow reducing the number of unknown parameters being estimated. That way fewer partial derivatives need to be computed, and hence fewer function evaluations required.

However, I don't quite understand why "external software" = "no analytical expression" nor why "no analytical expression" = "can't provide gradient" . Are you saying you don't know any mathematical description for what the external software is computing? If you do know it, there should be a way to furnish a gradient computation for it.

[AM]

Are you saying you don't know any mathematical description for what the external software is computing?

Rather than that I should have said that I do not know how to provide a gradient with the information I have.

What the external software does is solve mass balances in a batch reactor. So it gives me

F=[f1;f2;…;fi;…;fn_comp]

where fi (a scalar) is the concentration of component i at a specified time t and n_comp is the total number of components.

My decision vector x contains m decision variables (stoichiometric coefficients and kinetic parameters). I give these variables to the external software in a text file, which Matlab writes and it gives me the different fi.

Then my objective function is the sum of the following vector:

vect= [(C1-f1)^2; (C2-f2)^2; … (Ci-fi)^2; … ; (Cn_comp-fn_comp)^2] 

where Ci is the experimental concentration of component i.

objfun=sum(vect)

What I meant is that I do not have an analytical expression of F and I thought that to provide a gradient of objfun I would need it so that I could determine dFi/dxj for i=1:n_comp and j=1:m where Fi would be the analytical expresssion of the function allowing to calculate fi. With this I could determine the gradient G of the objective function

G=[dobjfun/dx1; … ;dobjfun/dxm]

I will look more into this. I’m sorry if I am not clear enough in my explanation of the problem and my notations (trying to explain is always difficult for me).

Thank you for the clarification regarding the number of function evaluations

[Matt J]

What I meant is that I do not have an analytical expression of F

That does make things more difficult. But what I really was trying to understand is if you know the algorithm used to compute F. Could you write your own version of it if you had to? If for example, F is not computed by an explicit formula, but instead your external software somehow obtains F as the solution of an implicit equation,

phi(x,F)=0

then the gradient of F could still be obtained using the implict function theorem.

[AM]

I will look into this, thank you for your help Matt!

Answer 3

[Matt J]

My question: Is their any way to avoid these useless function evalution and start out by making large steps?

The additional function evaluations are caused by finite difference calculations that fmincon (and other Optimization Toolbox solvers) use to compute derivatives of the objective and constraints, which in turn are used to compute search directions. You can avoid finite differencing, and often save computation, by providing analytic gradient calculations and/or analytic Hessian calculations for algorithms that use them.

However, if you must rely on the default finite difference calculations, then it is usually a bad idea to increase the differencing stepsize in the hope of taking larger steps. The finite differencing controls the direction of search in each iteration, not the distance moved. Typically, reducing the step size will just degrade the finite difference approximation, causing fmincon to choose an inferior search direction. A non-accurate gradient calculation would also prevent fmincon from recognizing when a point of zero-gradient has been reached, and that it is time to stop iterating.