Analytical Study of the Interplay between Architecture and Predictability

We study model feed forward networks as time series predictors in the stationary limit. The focus is on complex, yet non-chaotic, behavior. The main question we address is whether the asymptotic behavior is governed by the architecture, regardless the details of the weights. We find hierarchies among classes of architectures with respect to the attract or dimension of the long term sequence they are capable of generating; larger number of hidden units can attractors. In the case of a perceptron,generate higher dimensional the stationary solution for general weights, and showwe develop that the flow is typically one dimensional.