q-Neurons: Neuron Activations based on Stochastic Jackson's Derivative Operators

The vanilla method to train a Deep Neural Network (DNN) is to use the Stochastic Gradient Descent (SGD) method (a first-order local optimization technique). The gradient of the DNN loss function, represented as a directed computational graph, is calculated using the efficient backpropagation algorithm relying on the chain rule of derivatives (a particular case of automatic differentiation). The ordinary derivative calculus can be encompassed into a more general q-calculus [1, 2] by defining the Jackson's q-derivative (and gradient) as follows: D