Dimensionality reduction: select 3 random attributes for each tree in fitcensemble

Hi,

I want to use fitcensemble to determine which of my original attributes are the most important.

My idea is to create a large forest of 2000 trees. In each tree, I want to use only 3 attributes, randomnly selected. Also, I have set the maximum number of splits to 2. With this model and the method oobPermutedPredictorImportance() I understand I will get the most important attributes.

I think with my code (below) I fulfill all these conditions but the second one. How can I specify that each tree of the forest contains a small number of attributes (3) and that these attributes randomnly change from tree to tree?

if true
treeTemplate1 = templateTree('MaxNumSplits', 2,'PredictorSelection','allsplits'); 
Mdl1 = fitensemble(X, Y, 'bag', 2000, treeTemplate1, 'type', 'classification');
Imp1 = oobPermutedPredictorImportance(Mdl1);
end

[Ela Markovic]

Replying to this question from 2017 as it is still relevant and I am dealing with the same problem.

I found in MathWorks documentation the following statement:

"By default, the number of predictors to select at random for each split is equal to the square root of the number of predictors for classification, and one third of the number of predictors for regression."

How to set a different value than default is the question.

Answer 1

[Ela Markovic]

I managed to find the answer on how to limit the number of features (or atributes) that each specific tree chooses.

You need the following Name-Value pair:

'NumVariablesToSample', 3

Your template tree is then:

treeTemplate1 = templateTree('MaxNumSplits', 2,'PredictorSelection',...
    'allsplits','NumVariablesToSample', 3);