Learning representations by forward-propagating errors

In 1986, Rumelhart, Hinton, Williams suggested learning algorithm of the neural network, which is now usually called as back-propagation (BP) Rumelhart et al. [1986]. Since then, deep neural networks became trainable algorithm and is prospered by AlexNet Krizhevsky et al. [2017]. Uncountable researches have been proposed to train more accurate models, analyze model behavior, and enumerous fields. However, there is one profound question: Is the learning rule for neural network unique? It seems that Geoffrey Hinton have contemplated this problem for a long time. In a paper Lillicrap et al. [2020], Hinton and his colleagues approached issues of backpropagation in various perspectives. In 2022, Hinton have suggested a new learning rule named forward-forward algorithm Hinton [2022].