Data Mapping for Restricted Boltzmann Machine

R estricted Boltzmann machine (RBM) is two - layer neural nets constructed as a probabilistic model and i t s training is to maximiz e a product of probabilities by the contrastive divergence (CD) scheme . In this paper a data mapping is used to describe the relationship between visible and hidden layer s and the training is to minimize a squared error of the reconstructed visible layer by the gradient descent or a finite difference approximation . T his paper presents three new findings: 1) nodes on visible and hidden layers can take real - valued matrix dat a without a probabilistic interpretation; 2) the famous CD1 is a finite difference approximation of gradient descent after ignoring the second - order error; 3) activation can take non - sigmoid function s such as identity, relu and softsign. The data mapping p rovides a unified framework on dimensionality reduction, feature extraction and data representation pioneered and developed by Hinton and his colleagues . As an approximation of gradient descent, the finite difference learning is applicable to both directed and undirected graphs. N umerical results are performed to confirm these new findings on very low dimensionality reduction, matrix data and flexible activation s . Keywords: Restricted Boltzmann machine, data mapping, squared error, contrastive divergence, gradient descent and finite difference .