Reconstruction of Hidden Representation for Robust Feature Extraction

L(x, g(f(x))), where f(.) is the encoder function, g(.) is the decoder function and L(.) is the reconstruction error. Recently, they have become one of the most promising approaches to representation learning for estimating the data-generating distribution. Since the appearance of Auto-Encoders, many variants of representation learning algorithms based on Auto-Encoders have been proposed, e.g., Sparse Auto-Encoders [9], [10], Denoising Auto-Encoders (DAEs) [11], Higher Order Contractive Auto-Encoders [12], Variational Auto-Encoders [13], Marginalized Denoising Auto-Encoders [14], Generalized Denoising Auto-Encoders [15], Generative Stochastic Networks [16], MADE [17], Laplacian Auto-Encoders [18], Adversarial Auto-Encoders [19], Ladder Variational Auto-Encoders [20] and so on. In an Auto-Encoder-based algorithm, minimizing the reconstruction error of the input with the encoder and decoder function is a common practice for feature learning. The learned features are usually applied in subsequent tasks such as supervised classification [21]. In the past few years, many research works have shown that the reconstruction of the input with the encoder and decoder function is not only an efficient way for learning feature representation, but its resulting representations also substantially help the performance of the subsequent tasks. In general, a lower value of the reconstruction error of the input has a better feature representation of the input. In an ideal situation, the value of this reconstruction error is equal to 0, i.e., the 2