A Practical Introduction to Sequential Feature Selection

Sequential feature selection is a supervised approach to feature selection. It makes use of a supervised model and it can be used to remove useless features from a large dataset or to select useful features by adding them sequentially. This is a forward approach because we start with 1 feature and then we add other features. There's a backward approach as well, that starts from all the features and removes the less relevant ones according to the same maximization criteria. Since, at each step, we check the performance of the model with the same dataset with the addition of each remaining feature (one by one), it's a greedy approach. The algorithm stops when the desired number of features is reached or if the performance doesn't increase above a certain threshold.