Viewing the tree used for prediction by the method of "RUSBoost' in fitensemble

I have a question in regard to viewing the Tree from the fitensemble function. I am using 'RUSBoost' as the method. I can see that there are 1000 trees in the cell called Trained since I set nlearn to be a 1000. So these 1000 trees are the weak learners if I am not mistaken. But then where is the strong learner that was gotten using these weak learners? In other words, where is the tree that is actually used for prediction? How can I see that tree?

Thank you for your help.

Answer 1

[Ilya]

The strong learner is the ensemble. An ensemble is a collection of trees. It predicts by averaging predictions from individual trees. This average is weighted. You can get the weights from the TrainedWeights property of the ensemble object.

[artsci4]

Thank you for your answer llya.

So as I understand it, it is not possible to draw a single tree that shows how the ensemble makes decisions.

[Ilya]

If an ensemble could be reduced to a single decision tree with univariate splits, there would be no point in growing the ensemble. If you are willing to consider non-univariate splits such as, for instance, splits on various functions of two or more predictor variables, you can use that flexibility to represent any complex decision boundary in a multivariate domain (consider gene expression programming, for example). Ensemble learning does not do that. An ensemble is a non-parametric model, and here is the usual feature of non-parametric models - it is hard to visualize them in the multivariate space.

[artsci4]

I have another question related to this. Among all the machine learning algorithms provided in the MATLAB toolboxes, is fitensemble the only algorithm that allows the input of a cost function?

[Ilya]

Please open new threads for new questions.

ClassificationDiscriminant, ClassificationTree and ClassificationKNN accept the cost matrix as well.