Our analysis provides full distribution information on the joint outputs. Furthermore, the9 distribution ofthe cosine similarity explains whymoderately deepand wide ReLU networks can betrained despite10 negative results by mean field (MF) analysis based on correlations. There,14 the normal distribution originates from the MF limit. In contrast, here we understand that the output distribution is15 completely determined bytheempirical covariance matrix ofinputs. This is rather obvious however. Instead, we refer to the rich literature on linear neural networks at23 initialization.

A key quantity of interest is the mutual information and generalizations thereof, including conditional mutual information, multivariate mutual information, total correlation and directed information.

Offline Reinforcement Learning (RL) suffers from the extrapolation error and value overestimation. From a generalization perspective, this issue can be attributed to the over-generalization of value functions or policies towards out-of-distribution (OOD) actions.

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

We wish to thank all of the reviewers for their time and thorough reading of our paper! We appreciate the reviewer's suggestions regarding clarity. We have added the suggested summary sentence "the key We started with binary sentiment classification, but are actively working on more tasks. RNN hidden states onto the top two PCs for two different input sequences that differ only by two tokens (replacing ' The trajectories start out the same as the initial tokens are identical. We have added a footnote noting this in the main text.

Ironically, previous works neglect multi-round interaction, which is indispensable in human communication.

Neural Information Processing Systems http://nips.cc/