Green's Function Method for Fast On-Line Learning Algorithm of Recurrent Neural Networks

The two well known learning algorithms of recurrent neural networks are the back-propagation (Rumelhart & el al., Werbos) and the forward propa(cid:173) gation (Williams and Zipser). The main drawback of back-propagation is its off-line backward path in time for error cumulation. This violates the on-line requirement in many practical applications. Although the forward propaga(cid:173) tion algorithm can be used in an on-line manner, the annoying drawback is the heavy computation load required to update the high dimensional sensitiv(cid:173) ity matrix (0( fir) operations for each time step). Therefore, to develop a fast forward algorithm is a challenging task.