MindSimulator: Exploring Brain Concept Localization via Synthetic FMRI

Concept-selective regions within the human cerebral cortex exhibit significant activation in response to specific visual stimuli associated with particular concepts. Precisely localizing these regions stands as a crucial long-term goal in neuroscience to grasp essential brain functions and mechanisms. Conventional experiment-driven approaches hinge on manually constructed visual stimulus collections and corresponding brain activity recordings, constraining the support and coverage of concept localization. Additionally, these stimuli often consist of concept objects in unnatural contexts and are potentially biased by subjective preferences, thus prompting concerns about the validity and generalizability of the identified regions. To address these limitations, we propose a data-driven exploration approach. Our proposed MindSimulator leverages advanced generative technologies to learn the probability distribution of brain activity conditioned on concept-oriented visual stimuli. This enables the creation of simulated brain recordings that reflect real neural response patterns. The feasibility opens avenues for exploring novel concept-selective regions and provides prior hypotheses for future neuroscience research. The human brain's visual cortex is decisive in processing and perceiving visual information. Neuroscience researchers have long dedicated themselves to unraveling the brain's visual mechanisms, making impressive strides such as in brain visual encoding (Mitchell et al., 2008), decoding (Gong et al., 2024b), and visual perception (Chen et al., 2020). However, the process of forming visual cognition remains to be explored. Notably, localizing the various functional organizations and activation patterns of the visual cortex that correspond to human conceptual cognition is considered pivotal yet remains a challenging frontier (Huth et al., 2016; Henderson et al., 2023; Luo et al., 2024). Numerous neuroscience studies have illustrated that specific regions of the visual cortex exhibit concept selectivity. When individuals receive visual stimuli related to particular concepts (such as places, bodies, faces, words, colors, and foods), the respective cortical regions exhibit significant activation (Epstein & Kanwisher, 1998; Sergent et al., 1992; Jain et al., 2023; Pennock et al., 2023; Kanwisher et al., 1997; Allen et al., 2022). These regions are termed visual concept-selective regions and play a vital role in advancing the understanding of brain visual cognition.