POGD: Gradient Descent with New Stochastic Rules

Gradient descant is a popular optimization in neural networks. Nowadays, neural network has attracted much attention in any fields, there are many different types of the neural networks that has already been developed(Haykin, 2009). The convolutional neural networks is very popular in the classification field(Huang et al., 2017), even though it is a kind of the feedforward neural network(Haykin, 2009). In recent years, the structure of convolutional neural network has been rapidly improved(Smith and Topin, 2016). However, many improvements develop the structural algorithm of convolutional neural network, but not much improve it from optimization.