SARDNET: A Self-Organizing Feature Map for Sequences

A self-organizing neural network for sequence classification called SARDNET is described and analyzed experimentally. SARDNET extends the Kohonen Feature Map architecture with activation retention anddecay in order to create unique distributed response patterns for different sequences. SARDNET yields extremely dense yet descriptive representations of sequential input in very few training iterations.The network has proven successful on mapping arbitrary sequencesof binary and real numbers, as well as phonemic representations of English words. Potential applications include isolated spoken word recognition and cognitive science models of sequence processing. 1 INTRODUCTION While neural networks have proved a good tool for processing static patterns, classifying sequentialinformation has remained a challenging task. The problem involves recognizing patterns in a time series of vectors, which requires forming a good internal representationfor the sequences. Several researchers have proposed extending the self-organizing feature map (Kohonen 1989, 1990), a highly successful static pattern classification method, to sequential information (Kangas 1991; Samarabandu andJakubowicz 1990; Scholtes 1991). Below, three of the most recent of these networks are briefly described. The remainder of the paper focuses on a new architecture designed to overcome the shortcomings of these approaches.