Building random forest with cross-validation

Hi,

i have been wondering for some time now how random forests (Bagging, or AdaBoost, doesn't matter) are built when using cross-validation.

Let's see we're using 5-fold cross validation to train random forests on 5 different training sets and therefore test on 5 different test sets.

How does the 'final' random forest look like when we are basically building 5 random forests (one for each fold of the cross validation). How are these forests combined into a final model?

I have never understood this step and I really hope someone can help me with this!

thanks in advance,

Steven

Answer 1

[Ayush Aniket]

The models used in cross validation (5, as you mentioned) are not directly combined into a single final model.

The goal of cross-validation is to evaluate the model's performance across different data splits, ensuring it generalizes well. After cross-validation, you discard the five individual models.You train a new random forest on the entire dataset using the best hyperparameters found during cross-validation. This final model is used for predictions.

Refer the following example to read the workflow of training a final model after evaluating and finding hyperparameters through cross-validation: https://www.mathworks.com/help/stats/classificationsvm.crossval.html#mw_e9fd437d-4125-4b9e-b87c-f370bc439a3e