Model-based feature importance – Towards Data Science

In an earlier post, I discussed a model agnostic feature selection technique called forward feature selection which basically extracted the most important features required for the optimal value of chosen KPI. It had one caveat though -- large time complexity. In order to circumvent that issue feature importance can directly be obtained from the model being trained. In this post, I will consider 2 classification and 1 regression algorithms to explain model-based feature importance in detail. An inherently binary classification algorithm, it tries to find the best hyperplane in k-dimensional space that separates the 2 classes, minimizing logistic loss.