While our study identifies clear separations between model hypothesis classes, our best models still have not reached the consistency ceiling of the neural and behavioral benchmarks we have compared against. All models were simultaneously trained across all eight scenarios of the Physion Dynamics Training Set, constituting around 16,000 total training scenarios (2,000 scenes per scenario) [Bear et al., 2021], with a Each C-SWM [Kipf et al., 2020] model was trained on For each stimulus, we compute the proportion of "hit" responses by The Correlation to A verage Human Response is the Pearson's correlation between the model probability-hit vector and the human proportion-hit vector, across stimuli per scenario. OCP Accuracy of humans and models is the average accuracy, across stimuli per scenario. To give the final values of the two quantities, we then compute the weighted mean and s.e.m. of the above per Note that these values are therefore different for each condition, but always the same across all models. All neural predictivities are reported on heldout conditions and their timepoints.

We investigated DNNs as encoding models of visual areas V1, V2, V4, and IT. We examined 32 visual deep neural networks.

Further experimental details are described in Ref. [4]. Each spatiotemporal stimulus spanned over 400 ms corresponding to the retinal integration timescale. Figure 1: (a) The log-likelihood of empirical data for each PMF averaged over cells. Black line is the identity line. The central 20 20 arrays are shown.

Neural Information Processing Systems http://nips.cc/

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization.

Additionally, we provide multimodal predictive models combining video, audio, and physiological inputs.

Encoding models have been used to assess how the human brain represents concepts in language and vision.

Here, we propose a hybrid model, the Hyper-HMM, that simultaneously aligns both temporal and spatial features across brains.