Fine Tuning Method by using Knowledge Acquisition from Deep Belief Network

Deep Learning is well known to be the representative method of artificial intelligence. The representation learning can discover the good set of features to input patterns and calculate the representation itself. Many kinds of structures and learning methods have been developed to achieve the great success. It is often said that Deep Learning should include the hierarchical model deeply, or the discovering of optimal structure and its parameters of Convolutional Neural Network (CNN) [1] is important. This issue pointed out by many researchers is right definitely, however, the effort to find the optimal structure and the parameters is very expensive and the calculation cost becomes high. To realize high level representation at low calculation cost, the self-organizing mechanism to adjust the structure itself and parameters simultaneously should be required with the statistical learning method. We have developed the structural learning method of Restricted Boltzmann Machine (RBM) [2] by neuron generation/annihilation algorithm [3].