InIRsystems, theretrievermodule directly influences theperformance ofdownstream tasks, such as retrieval-augmented generation [20, 29, 30] and knowledge-intensive question answering [34, 52]. However, these methods do not explicitly consider targeted optimization for tool usage or the impact on complex multi-stage tasks.

We also implemented the Iterative Null-Space Projection (INLP) method (Ravfogel et al., 2020) to Results using our method are in Table 4. Results using the INLP method are This pattern holds across all of the linguistic properties that we tested. Each language brain region is not necessarily homogeneous in function across all voxels it contains. Bottom plot displays the pretrained BERT vs. removal of all tasks. Like the probing experiments with BERT in the main paper, we also perform experiments with GPT2. We find the results to be similar to BERT, i.e., a rich hierarchy of linguistic signals: initial to middle layers encode surface information, middle layers encode syntax, middle to top layers We verify that the removal of each linguistic property from GPT2 leads to reduced task performance across all layers, as expected.

Language models have been shown to be very effective in predicting brain recordings of subjects experiencing complex language stimuli.

Global information is essential for dense prediction problems, whose goal is to compute adiscrete or continuous label for each pixel in the images. Traditional convolutional layers in neural networks, initially designed for image classification, are restrictive in these problems since the filter size limits their receptive fields. In this work, we propose to replace any traditional convolutional layer with an autoregressivemoving-average (ARMA) layer,anovelmodule with an adjustable receptive field controlled by the learnable autoregressive coefficients.