Threshold For Predictor Importance

Suppose you have a set of predictors X1, X2, X3, X4,.....Xn, and a Response variable Y. If after computing predictor importance for X1--Xn against Y, how does one determine a threshold that is statistical; not just an arbitrary cut off?

Answer 1

[Gaurav Garg]

Hi,

To determine threshold in case of decision trees, Information Gain is calculated for each of the nodes. The node with the highest Information Gain is selected as the decision node, and the same process is repeated for the next level nodes as well.

The way Information Gain is calculated can differ from categorical data to numerical data.

For more information, you can look at the following links -

1. Splitting categorical variables
2. Viewing decision trees

Please note that Information gain is computed statistically. You can also easily compute Information Gain on your dataset and then compare it with the decision tree returned by the model.

[BP205]

Sorry for the late response. Thanks for the answer