Optimising using a combination of linear equality constraints and no constraints

Hi,

I am currently trying to optimise at 18 variable input. 13 of my variables don't require any constraints on them except for ub and lb. 5 of the variables are summed to 1. I tried using simulannealbnd and normalising these weightings to each other. I have tried using patternsearch but patternsearch only seemed to consider the variables in the linear equality.

Does any one have any advise on what optimiser to use?

% set upper and lower bounds for time (s), PID and Error:
LB=[0.15,...%time delay
    0,0,0,0,0,...%noise
    0,0,0,0,0,0,0,...%gains
    0,0,0,0,0]; %weights to sum to 1
UB=[0.15,...%time delay
    1/10000,1/10000,1/1000,1/1000,1/1000,...%noise
    10,10,500,1500,500,2,2,...%gains
    1,1,1,1,1];%weights to sum to 1
INPUT=(LB+UB)/2; %set initial guess in the middle

Aeq=[zeros(1,13),ones(1,5)];

ObjectiveFunction=@(x) NN_v1_2_Parfor(INPUT,setup);

[x,fval,exitFlag,output] = patternsearch(ObjectiveFunction,INPUT,[],[],Aeq,1,LB,UB);

Answer 1

[Alan Weiss]

I don't know whether your objective function is smooth or not. If it is smooth (differentiable), I would suggest that you use fmincon as your solver. If it is not smooth, then use patternsearch with the following option change:

options = optimoptions('patternsearch','PollMethod','GSSPositiveBasis2N');

Be sure to pass options in your solver call.

I also suggest that you remove variable 1 from the problem entirely, so your problem is 17 variables. And I suggest that you rescale your variables so that all are of the same order. Your objective function internally should divide the first five variables by 1000, and the upper bound for the first five variables should be 1. Similarly, the new variables 8, 9, and 10 should internally be multiplied by 500, 1500, and 500 respectively, and the corresponding upper bounds should all be 1.

Alan Weiss

MATLAB mathematical toolbox documentation

[Adam Handley]

The optimiser seems to be finding a solution much faster than when I was using simulanealbnd on 5 variables and when I rerun my function using the optimised parameters the solution is not one that I would expect to see if it was optimised correctly.

I accept that this may be an issue with my code elsewhere but as the time taken is not as long I am inclined to believe that this is not optimising all the 17 variables correctly.

Adam

[Alan Weiss]

patternsearch is much, much more efficient than simulannealbnd. I'm not the least surprised that it ends up being much, much faster. The only issue is that, if you have a nonconvex problem and want a global solution, simulannealbnd is much, much more likely to escape from local solutions that are not global solutions. If you want a global solution then I suggest that you rerun patternsearch from a variety of start points.

And again, if your objective function is smooth, I suggest that you use fmincon instead as the local solver. It is much faster than patternsearch for finding a local solution.

Alan Weiss

MATLAB mathematical toolbox documentation

[Adam Handley]

My objective function is the sum of t-statistics of 10 trials of the simulation for 4 variables. I am not sure if a t-statistic would be described as a smooth function.

[Alan Weiss]

Is your objective function deterministic? I mean, if you run the objective function twice with the same input parameters, do you necessarily get the same result? If not, then any Global Optimization Toolbox solver may not be reliable, because these solvers (as well as all Optimization Toolbox solvers) have an inherent assumption of a deterministic objective function.

Sometimes you can make a simulation deterministic by resetting the random stream before each objective function call, something like

rng default

as the first line of your objective function code.

Alan Weiss

MATLAB mathematical toolbox documentation

[Adam Handley]

The objective function won't be deterministic as the simulation as random noise added at various stages throughout the simulation. The basis of the simulation is that the same simulation won't be achieved each time. The parameters found should be able to hold up in a stochastic environment. I don't know how this fairs in relation to the optimisers.

[Alan Weiss]

Your problem needs to be better defined. Are you trying to minimize the mean of something? Then you need to take enough samples to have a good estimate of the mean. This might imply that your simulation should be run many, many times for each parameter, and the whole computation will then be slow.

I suspect that you need to think hard about what exactly you are attempting to do. What is the real objective function? The theoretical, deterministic thing that you are trying to optimize? Is there any way to calculate it, or can you only obtain samples from a distribution?

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[Adam Handley]

I am trying to minimise the difference between an inverted pendulum model and experimental collected data. The experimental data no one experiment is the same and to accurately have the simulation match the experiment the simulation cannot run exactly the same each time. Therefore I have adopted the approach of optimising based on the t-statistic from a set number of optimisations in order to determine if the simulation can replicate the sample of experimentally measured data. As I only have 10 sets of experimental data I have only been running 10 simulations.