Entropy

Helps find the optimal decision tree

Chun Zheng

Entropy is used to help create an optimized decision tree. I created an entropy function called getBestEnt so that given the information it has received, it will return the highest information gain and the index of the best feature to use for the decision tree.

Here's an example:
hair=[1 1 2 3 2 2 2 1];
entropyF(class,hair)
ans =
0.5000
eyes=[1 1 1 1 2 1 2 2];
entropyF(class,eyes)
ans =
0.6068
height=[1 2 1 1 1 2 2 1];
entropyF(class,height)
ans =
0.9512
entropy(class)
ans =
0.9544
allFeat=[eyes hair height];
[big ind]=getBestEnt(class, allFeat)
big =
0.9544
ind

1
note: big stands for best information gain
The ind determines the 1nd feature(eyes) as the best feature