We focus our attention on spatial, structural connectivity at the mesoscale: a coarser scale than that of single neurons or cortical columns but finer than whole brain regions.

A major goal of neuroscience is to understand brain computations during visual processing in naturalistic settings. A dominant approach is to use image-computable deep neural networks trained with different task objectives as a basis for linear encoding models. However, in addition to requiring estimation of a large number of linear encoding parameters, this approach ignores the structure of the feature maps both in the brain and the models. Recently proposed alternatives factor the linear mapping into separate sets of spatial and feature weights, thus finding static receptive fields for units, which is appropriate only for early visual areas. In this work, we employ the attention mechanism used in the transformer architecture to study how retinotopic visual features can be dynamically routed to category-selective areas in high-level visual processing. We show that this computational motif is significantly more powerful than alternative methods in predicting brain activity during natural scene viewing, across different feature basis models and modalities. We also show that this approach is inherently more interpretable as the attention-routing signals for different high-level categorical areas can be easily visualized for any input image. Given its high performance at predicting brain responses to novel images, the model deserves consideration as a candidate mechanistic model of how visual information from retinotopic maps is routed in the human brain based on the relevance of the input content to different category-selective regions.

There is some knowledge from anatomy (Harris, et al, 2019).

This'divide and conquer' strategy captures visual information more accurately.

Last layer of every residual stage (res1, res2, res3, res4) and avgpool Table A.2: Details of the task-optimized DNNs used as baselines 2 A.4 Characterizing the spatial tuning of early visual areas: Polar angle agreement

However, these studies are, by construction, constrained by their a priori hypotheses. Furthermore, beyond the early stages, precise neuronal tuning properties and representational transformations along the ventral visual pathway remain poorly understood.

It is an dimensionality-representation representation of word-word co-occurrence statistics. We used the Flair NLP [1] implementation of BERT embeddings. NWE is the GloV erepresentation, offset by one word in the future. Representations were built using a sliding window of 64 words as a context. HuggingFace [41] implementation of this network to extract feature for these representations.