Goto

Collaborating Authors

 trie-based model


Memory Efficient Tries for Sequential Pattern Mining

arXiv.org Artificial Intelligence

Sequential Pattern Mining (SPM) is a prominent topic in unsupervised learning that aims at finding frequent patterns of events in sequential datasets. Frequent patterns have a wide range of applications and are used, for example, to develop novel association rules, aid supervised learners in prediction tasks, and design effective recommender systems. While supervised learning algorithms have enjoyed great success in using large-size datasets for better prediction accuracy, unsupervised algorithms such as SPM are still faced with challenges in scalability and memory requirement. In particular, the two dominant SPM methodologies, Apriori (Agrawal et al., 1994) and prefix-projection (Han et al., 2001), suffer from the explosion of candidate patterns or require to fit in memory the entire large-size training dataset. This memory bottleneck is aggravated by the steady increase of dataset size in recent years, which may contain a larger and richer set of frequent patterns to be investigated. It is thus vital for the success of SPM algorithms that they adapt to their rapidly growing data environment. This paper investigates the role of dataset models in the time and memory efficiency of SPM algorithms.