Problem in finding the global minima of a regression model

Hi,

I have created a regression model using the Matlab Regression Learner App in order to fit the best curve to a series of parameters. I succeed in creating a trainedModel (1x1 struct); however I do not really know what I have to do in order to proceed.

I have exported the model in the workspace and tried to use surrogateopt to find the global minima. The problem is that, being really new to Matlab, I cannot understand why the function does not accept the trainedModel.predictFcn as a valid function.

Could you please give me a sort of guidance I need to follow, as detailed as possible, in order to achieve my objective.

Thanks in advance.

Giuseppe.

Answer 1

[Alan Weiss]

If you have a trained regression model then I think that there is nothing left to minimize. The training is, in fact, the solution of the minimum, where the software minimizes the sum of squared differences between the data and the model predictions. This is the best curve you refer to.

Or perhaps I misunderstand what you are trying to do. If so, please ask again with more detail.

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Matera]

Once i get the trained model, and export it into the workspace, I want to find the global minima of the fitted function using for example a genetci algorithm.

My problem is basically that, once I got the trainedModel (1x1 struct) in the workspace I do not know how to proceed, not to minimise the function, but to use and optimise it in terms of finding particular points in the function (minima/maxima).

If I am not wrong the global minima can be found using either surrogateopt or Particle Swarm optimisation.

Best regards.

Giuseppe.

[Alan Weiss]

To get the value of the model at a specified point, use the predict method. So, for example, your objective function could be

fun = @(x)predict(mdl,x)

where mdl is your trained model. Optimize by calling a solver:

sol = fminsearch(fun,x0)
% or
sol = ga(fun,nvar)

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Matera]

I tried both the solutions you suggested but the following error messages appeared:

• when I used fminsearch:

Check for incorrect argument data type or missing argument in call to function 'predict'.

Error in untitled4>@(x)predict(objconstr,x) (line 1)

fun = @(x)predict(objconstr, x)

Error in fminsearch (line 201)

fv(:,1) = funfcn(x,varargin{:});

Error in untitled4 (line 3)

sol = fminsearch(fun,x0)

• when I used ga:

Check for incorrect argument data type or missing argument in call to function 'predict'.

Error in untitled4>@(x)predict(objconstr,x) (line 1)

fun = @(x)predict(objconstr, x)

Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11)

fcn_handle = @(x) fcn(x,FcnArgs{:});

Error in makeState (line 58)

firstMemberScore = FitnessFcn(state.Population(initScoreProvided+1,:));

Error in gaunc (line 47)

state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);

Error in ga (line 411)

[x,fval,exitFlag,output,population,scores] = gaunc(FitnessFcn,nvars, ...

Error in untitled4 (line 2)

sol = ga(fun,5)

Best Regards

Giuseppe.

[Alan Weiss]

I clearly do not understand what your trained model is. Is it an object of class NonLinearModel? If not, what is it? Your screen shots are too small for me to read.

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Matera]

Apologies for not specifing it. What I have done from the beginning was:

• take a set of data (parameters + predictors) and upload it to matlab
• create a new session in the matlab regression learner app and train all the model
• export the model (the first option in the export model tree) that gave me the least RMSE

This is what I have done so far. I have a trained model and given a set of new values for the parameters I can predict the result using yfit = trainedModel.predictFcn(T). So far so good, I can input a new set of data and it will return me the predictions.

However, my aim is not that, I would like to study the function and get the global mimima. I have tryied the code below but i keep getting error messages.

beta0 = [];

mdl = fitnlm(DATA, objconstr, beta0);

fun = @(x)predict(mdl, x);

sol = ga(fun,5);

where Data is the table with the initial variables and preditors, objconstr is the function handle taken from the trainedModel and Beta0 should be the coefficients of the function. The problem now seems to be the Beta0, as we do not have an actual function with coefficient.

I hope this clarify the problem.

Thanks a lot for the time and help you are giving me.

Best regards.

Giuseppe.

[Alan Weiss]

My apologies, I am completely unfamiliar with the regression learner app, and made some assumptions about how it works that I now see are wrong. One issue is that you can use a table of data for the app, but Optimization Toolbox function work only with matrices, not tables.

I suggest that, if you want to work with Optimization Toolbox functions, you export your regression in a way that the exported regression accepts matrix data, not table data. This means (I believe) that your training data should be in matrix form, not table form.

Then I believe that using

fun = @(x)trainedModel.predictFcn(x)

should be a valid function for optimization solvers.

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Matera]

I tried your way and it partially worked. I firt converted the table into a matrix using table2array(), that upload the array into the regression learner and exported the model.

Now there is no error message; however, it does not seem to work proprerly.

If I use:

fun = @(x)trainedModel1.predictFcn(x)

x0 = [0.789, 1, 0.234, 0.8, 098754]

sol = fminsearch(fun,x0)

where x0 is just an initial guess for the parameters I get:

fun =

function_handle with value:

@(x)trainedModel1.predictFcn(x)

x0 =

1.0e+04 *

0.0001 0.0001 0.0000 0.0001 9.8754

sol =

1.0e+04 *

0.0001 0.0001 0.0000 0.0001 9.8754

while if I use:

fun = @(x)trainedModel1.predictFcn(x)

sol = ga(fun,5);

I get:

fun =

function_handle with value:

@(x)trainedModel1.predictFcn(x)

Optimization terminated: average change in the fitness value less than options.FunctionTolerance.

Unfortunatley, both the methods do not work.

Do you have any suggestion about what the problem could be?

Thanks for your help.

Best regards.

Giuseppe.

[Alan Weiss]

I suggest that you use a plot function or iterative display to see what is going on.

options = optimset('Display','iter');
[x,fval] = fminsearch(fun,x0,options)
% or
opts = optimoptions("ga","PlotFcn",@gaplotbestf);
[x,fval] = ga(fun,5,[],[],[],[],[],[],[],opts)

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Matera]

Finally, it worked! However, I only tried with one variable so I might need to check if it works with 5 variables as well.

Also, is there a way to plot the actual function in 5D?

Thank you very much for the help you gave me and your time.

Best regards.

Giuseppe.

[Giuseppe Matera]

Something like the example here: https://uk.mathworks.com/help/gads/ga.html#d123e46843 where the function ps_example is plotted, but using 5 dimensions.

Best regards.

Giuseppe.

[Alan Weiss]

To plot in 4 or 5 dimensions is tough. I am not familiar with a general technique. You could try using parallelplot, but I'm not sure that this will satisfy you.

Alan Weiss

MATLAB mathematical toolbox documentation

[Giuseppe Matera]

I will definitly have a look at it.

Thanks again for all your help and time.

Best regards.

Giuseppe.