How can I compare three linear models (fitlme)?

I have three different models with different random effects and funtions:

modelspec = 'Latency ~ 1 + TestNumber + (1|AnimalID)';

lmeinter = fitlme(data, modelspec);

modelspec = 'Latency ~ 1 + TestNumber + (1 + TestNumber|AnimalID)';

lmelin = fitlme(data,modelspec);

formula = 'Latency ~ 1+ TestNumber + TestNumber^2+ (1 + TestNumber|AnimalID)';

lmequad = fitlme(data, formula);

Now I would like to compare the performance of the different models. They are all nested. Using the "compare" function (Likelihiid ration test), I can compare two, but not three models. Is there a way to run an anova on models?

[Harald]

Hi,

how about comparing two models, and then comparing the better model of the two to the third one?

Best wishes,

Harald

[Star Strider]

One option could be to get the residuals from each regression (as column vectors) and then use the friedman function to see if any are different. Another option might be the multcompare function (there are also other versions of the function, so consider them to see which is most appropriate for what you want to do).

Answer 1

[Shivansh]

Hi David!

I understand that you want to compare three different linear mixed-effects models.

One approach to do this is by pairwise comparison of the models. You can do this using the ‘compare’ function using the following code.

% Compare the models pairwise
comparison1 = compare(lmeinter, lmelin);
comparison2 = compare(lmeinter, lmequad);
comparison3 = compare(lmelin, lmequad);

% Display the results
disp(comparison1);
disp(comparison2);
disp(comparison3);

If you want to compare more than two models simultaneously, I am not sure about how you want to use anova but you can use the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC). Both AIC and BIC are measures of the goodness of fit of an estimated statistical model and can be used for model selection.

Lower values of AIC or BIC indicate better fitting models. You can implement it by adding the following code:

AIC = [lmeinter.ModelCriterion.AIC lmelin.ModelCriterion.AIC lmequad.ModelCriterion.AIC];
BIC = [lmeinter.ModelCriterion.BIC lmelin.ModelCriterion.BIC lmequad.ModelCriterion.BIC];
disp('AIC:');
disp(AIC);
disp('BIC:');
disp(BIC);

You can refer to the following documentation links for more information:

1. compare: https://www.mathworks.com/help/stats/linearmixedmodel.compare.html.
2. AIC and BIC: https://www.mathworks.com/help/ident/ref/idmodel.aic.html.
3. Information Criterion for model selection: https://www.mathworks.com/help/econ/information-criteria.html.

Hope it helps!

[David Obert]

Thank you for the answer! I combined your answer with the comment comment of Star Strider. I use the compare function as well as the AIC (as you proposed). Additionally, I calculated the residual sum of square. So I have three measures, all being consistent and demonstrating that lmequad is the best model.