Evolving Ensemble Fuzzy Classifier

Abstract-- The concept of ensemble learning offers a promising avenue in learning from data streams under complex environments because it addresses the bias and variance dilemma better than its single-model counterpart and features a reconfigurable structure, which is well-suited to the given context. While various extensions of ensemble learning for mining nonstationary data streams can be found in the literature, most of them are crafted under a static base-classifier and revisits preceding samples in the sliding window for a retraining step. This feature causes computationally prohibitive complexity and is not flexible enough to cope with rapidly changing environments. Their complexities are often demanding because it involves a large collection of offline classifiers due to the absence of structural complexities reduction mechanisms and lack of an online feature selection mechanism. A novel evolving ensemble classifier, namely Parsimonious Ensemble (pENsemble), is proposed in this paper. A dynamic online feature selection scenario is integrated into the pENsemble. This method allows for dynamic selection and deselection of input features on the fly. The efficacy of the pENsemble has been numerically demonstrated through rigorous numerical studies with dynamic and evolving data streams where it delivers the most encouraging performance in attaining a tradeoff between accuracy and complexity. I. INTRODUCTION The data-intensive era where data are collected continuously in a fast rate under dynamic and evolving environments opens a new research direction to process data streams efficiently [1], [2]. Unlike a classical paradigm in machine learning where a dataset is utilised to construct hypothesis and is executed over multiple passes, data streams requires a strictly online learning framework with a low memory requirement and even if possible with no memory at all - one-pass learning mode. Another challenging trait of data streams lies in the non-stationary characteristics [3] where the data does not follow static and predictable distributions and contains a variety of concept drifts [4], [5]. These facts make a retraining phase when incorporating a new sample to an old dataset impossible to be performed because it leads to the socalled catastrophic forgetting [6] of previously valid knowledge and is not scalable when dealing with massive data streams. Evolving Intelligent System (EIS) provides a unique solution for data stream mining because a strictly one-pass learning procedure involved here has delivered great success to cope with time-critical applications where data streams are generated at a very fast sampling rate [7]. Furthermore, EIS adopts an open structure where its components can be automatically generated, pruned, merged and recalled on the fly [8], [9] and can be well-suited to a given problem.