How can I prune the weak learners in an ensemble learner?

조회 수: 5 (최근 30일)
Andrew
Andrew 2014년 10월 4일
편집: Jan 2017년 8월 10일
The fitensemble method doesn't seem to provide a method for actually pruning the weak learners that comprise it. I understand accuracy may not be improved by doing so, but in my usage space is at a premium, so I'd like to reduce the number of nodes in the trees used as my weak learners -- or at least be able to experiment with the effect that has. And I'm interested in reducing the number of nodes not just by adjusting the "leafiness" via the MinLeaf and MinParent settings, but also use pruning.
Note that enabling "Prune" in the templateTree used as my weak learner does not seem to actually do any pruning, but merely computes the PruneList. However, not only do the CompactClassificationTrees that are my weak learners (inexplicably) lack a prune method, even though they have a PruneList, but also, they are protected members of the ensemble and thus not adjustable anyway.
So how can I prune my weak learners after creating an ensemble?

채택된 답변

Andrew
Andrew 2014년 10월 4일
I'm answering my own question. (Actually, I had found the answer before asking, after much fiddling, but I thought it might be useful for others.)
NOTE: this is a somewhat disgusting hack. I'd love to know if there is a better way!
Though the "Trained" property of my ensemble is set-protected, and the constituent CompactClassificationTrees don't seem to possess a publicly accessible "prune" method, the hidden "Impl" property of each is publicly set-able and allows me to do what I want. So after using a template tree with pruning on to create an initial ensemble, I was able to loop over the resulting set of trained trees and prune each like the following. (The key is the two usage of "Impl" to access public implementations I could adjust.)
tTree = templateTree( ... , 'Prune', 'on');
ensembleCTrees = fitensemble(xdata, labels, 'AdaBoostM2', 10, tTree);
prunedEnsemble = ensembleCTrees;
for iTree = 1:maxTrees
prunedEnsemble.Impl.Trained{iTree}.Impl = prune( ...
prunedEnsemble.Impl.Trained{iTree}.Impl, 'Level', 20);
end
Then, I could try different settings of the prune level and compare prediction performance between the original and pruned ensembles.

추가 답변 (1개)

Daniel Vieira
Daniel Vieira 2017년 8월 10일
I'm having a similar problem, it appears this doesn't work anymore on 2017a.
  댓글 수: 1
Jan
Jan 2017년 8월 10일
편집: Jan 2017년 8월 10일
@Daniel: Please do not highjack an existing thread by posting a new question as a pseudo-answer. Open a new thread and add a link to this one, if this is useful. Add an explanation, what "doesn't work" mean. It is easier to solve a problem than to guess, what the problem is. Then delete your answer here. Thanks.

댓글을 달려면 로그인하십시오.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by