Alternative Feature Selection Methods in Machine Learning - KDnuggets

You've probably done your online searches on "Feature Selection", and you've probably found tons of articles describing the three umbrella terms that group selection methodologies, i.e., "Filter Methods", "Wrapper Methods" and "Embedded Methods". Under the "Filter Methods", we find statistical tests that select features based on their distributions. These methods are computationally very fast, but in practice they do not render good features for our models. In addition, when we have big datasets, p-values for statistical tests tend to be very small, highlighting as significant tiny differences in distributions, that may not be really important. The "Wrapper Methods" category includes greedy algorithms that will try every possible feature combination based on a step forward, step backward, or exhaustive search.