An Information-Gain-based Feature Ranking Function for XGBoost

XGBoost (short for Extreme Gradient Boosting) is a relatively new classification technique in machine learning which has won more and more popularity because of its exceptional performance in multiple competitions hosted on Kaggle.com. A lesser known benefit of using XGBoost is that the tree ensemble model can rank features for high-dimensional data sets. The official implementation of XGBoost (Python) provides only one feature scoring function called get_fscore. What it does is that, it computes feature scores by counting how many times a feature appears in the splits and rank the features according to the splits. It is simple, and it is straightforward, but I believe we should not ignore another metric which is critical to the decision tree method.