Autoencoders as Pattern Filters

An autoencoder is a feed-forward Deep Neural Network (DNN) consisting of an encoder and a decoder, trained to reproduce its input at the output layer (Figure 1) [1], [2]. Autoencoders are used in a large variety of applications: dimensionality reduction, feature extraction, image denoising, imputing missing data etc. Depending on the dimensionality of the hidden layer we can distinguish two types of autoencoders: Undercomplete: the hidden layer has a lower dimension than the input/output layers. Overcomplete: the hidden layer has a higher dimension than the input/output layers. In general, the dimensionality of the hidden layer should be different than the dimensionality of the input/output layers in order to avoid learning an identity data transformation. Undercomplete autoencoders are typically used in unsupervised learning tasks, such as: dimensionality reduction, feature learning, and generative models. The encoder is generally used to learn a lower dimensional latent representation of the input samples, performing an efficient compression through non-linear transformations. In the same time, the decoder learns how to reconstruct the input samples from this latent compressed representation.