Strength Low
Prompt Your Brain: Scaffold Prompt Tuning for Efficient Adaptation of fMRI Pre-trained Model
Dong, Zijian, Wu, Yilei, Chen, Zijiao, Zhang, Yichi, Jin, Yueming, Zhou, Juan Helen
We introduce Scaffold Prompt Tuning (ScaPT), a novel prompt-based framework for adapting large-scale functional magnetic resonance imaging (fMRI) pre-trained models to downstream tasks, with high parameter efficiency and improved performance compared to fine-tuning and baselines for prompt tuning. The full fine-tuning updates all pre-trained parameters, which may distort the learned feature space and lead to overfitting with limited training data which is common in fMRI fields. In contrast, we design a hierarchical prompt structure that transfers the knowledge learned from high-resource tasks to low-resource ones. This structure, equipped with a Deeply-conditioned Input-Prompt (DIP) mapping module, allows for efficient adaptation by updating only 2% of the trainable parameters. The framework enhances semantic interpretability through attention mechanisms between inputs and prompts, and it clusters prompts in the latent space in alignment with prior knowledge. Experiments on public resting state fMRI datasets reveal ScaPT outperforms fine-tuning and multitask-based prompt tuning in neurodegenerative diseases diagnosis/prognosis and personality trait prediction, even with fewer than 20 participants. It highlights ScaPT's efficiency in adapting pre-trained fMRI models to low-resource tasks.
Spatio-Temporal Surrogates for Interaction of a Jet with High Explosives: Part I -- Analysis with a Small Sample Size
Kamath, Chandrika, Franzman, Juliette S., Daub, Brian H.
Computer simulations, especially of complex phenomena, can be expensive, requiring high-performance computing resources. Often, to understand a phenomenon, multiple simulations are run, each with a different set of simulation input parameters. These data are then used to create an interpolant, or surrogate, relating the simulation outputs to the corresponding inputs. When the inputs and outputs are scalars, a simple machine learning model can suffice. However, when the simulation outputs are vector valued, available at locations in two or three spatial dimensions, often with a temporal component, creating a surrogate is more challenging. In this report, we use a two-dimensional problem of a jet interacting with high explosives to understand how we can build high-quality surrogates. The characteristics of our data set are unique - the vector-valued outputs from each simulation are available at over two million spatial locations; each simulation is run for a relatively small number of time steps; the size of the computational domain varies with each simulation; and resource constraints limit the number of simulations we can run. We show how we analyze these extremely large data-sets, set the parameters for the algorithms used in the analysis, and use simple ways to improve the accuracy of the spatio-temporal surrogates without substantially increasing the number of simulations required.
SFCNeXt: a simple fully convolutional network for effective brain age estimation with small sample size
Fu, Yu, Huang, Yanyan, Dong, Shunjie, Wang, Yalin, Yu, Tianbai, Niu, Meng, Zhuo, Cheng
Deep neural networks (DNN) have been designed to predict the chronological age of a healthy brain from T1-weighted magnetic resonance images (T1 MRIs), and the predicted brain age could serve as a valuable biomarker for the early detection of development-related or aging-related disorders. Recent DNN models for brain age estimations usually rely too much on large sample sizes and complex network structures for multi-stage feature refinement. However, in clinical application scenarios, researchers usually cannot obtain thousands or tens of thousands of MRIs in each data center for thorough training of these complex models. This paper proposes a simple fully convolutional network (SFCNeXt) for brain age estimation in small-sized cohorts with biased age distributions. The SFCNeXt consists of Single Pathway Encoded ConvNeXt (SPEC) and Hybrid Ranking Loss (HRL), aiming to estimate brain ages in a lightweight way with a sufficient exploration of MRI, age, and ranking features of each batch of subjects. Experimental results demonstrate the superiority and efficiency of our approach.
Elon Musk's Neuralink 'botched experiments' revealed by former employee and internal lab notes
'Botched experiments' by Elon Musk's Neuralink allegedly'kept suffering animals alive for no reason and malpractice caused monkey's brains to hemorrhage' during rushed brain chip testing, a former Neuralink employee and internal lab notes reveal. The billionaire's startup is accused of violating the Animal Welfare Act with its experiments at the University of California, Davis, from 2017 through 2020, which'sacrificed all the animals involved,' a former Neuralink employee, who asked to remain anonymous, told DailyMail.com. One case stood out to them- a monkey sacrificed ahead of schedule due to errors allegedly made during surgery. 'There was no reason to use it,' the former employee, who worked as a necropsy technician, told DailyMail.com. 'BioGlue was not FDA-approved for brain surgery and would never be able to be carried over to human trials.
Combined Pruning for Nested Cross-Validation to Accelerate Automated Hyperparameter Optimization for Embedded Feature Selection in High-Dimensional Data with Very Small Sample Sizes
May, Sigrun, Hartmann, Sven, Klawonn, Frank
Background: Embedded feature selection in high-dimensional data with very small sample sizes requires optimized hyperparameters for the model building process. For this hyperparameter optimization, nested cross-validation must be applied to avoid a biased performance estimation. The resulting repeated training with high-dimensional data leads to very long computation times. Moreover, it is likely to observe a high variance in the individual performance evaluation metrics caused by outliers in tiny validation sets. Therefore, early stopping applying standard pruning algorithms to save time risks discarding promising hyperparameter sets. Result: To speed up feature selection for high-dimensional data with tiny sample size, we adapt the use of a state-of-the-art asynchronous successive halving pruner. In addition, we combine it with two complementary pruning strategies based on domain or prior knowledge. One pruning strategy immediately stops computing trials with semantically meaningless results for the selected hyperparameter combinations. The other is a new extrapolating threshold pruning strategy suitable for nested-cross-validation with a high variance of performance evaluation metrics. In repeated experiments, our combined pruning strategy keeps all promising trials. At the same time, the calculation time is substantially reduced compared to using a state-of-the-art asynchronous successive halving pruner alone. Up to 81.3\% fewer models were trained achieving the same optimization result. Conclusion: The proposed combined pruning strategy accelerates data analysis or enables deeper searches for hyperparameters within the same computation time. This leads to significant savings in time, money and energy consumption, opening the door to advanced, time-consuming analyses.
Yield-predicting AI needs chemists to stop ignoring failed experiments
Machine-learning algorithms that can predict reaction yields have remained elusive because chemists tend to bury low-yielding reactions in their lab notebooks instead of publishing them, researchers say. 'We have this image that failed experiments are bad experiments,' says Felix Strieth-Kalthoff. 'But they contain knowledge, they contain valuable information both for humans and for an AI.' Strieth-Kalthoff from the University of Toronto, Canada, and a team around Frank Glorius from Germany's University of Mรผnster are asking chemists to start including not only their best but also their worst results in their papers. This, as well as unbiased reagent selection and reporting experimental procedures in a standardised format, will allow researchers to finally create yield-prediction algorithms. Retrosynthesis is already using machine-learning models to create shorter, cheaper or non-proprietary synthetic routes. But there have been few attempts at creating programs that predict yields.
Expressions: People around the world pull the same faces in similar social settings
Experts from the US used artificial intelligence to analyse the faces seen in 6 million YouTube videos -- broadly finding similar expressions in similar social settings. However, the team warned, the findings have not confirmed whether emotions themselves are universal, with further research needed to explore this area. Previous, survey-based studies into whether expressions and emotions are universal have been constrained by language barriers and small sample sizes. The study was undertaken by emotion scientist Alan Cowen of the University of California, Berkeley and his colleagues. The researchers used a type of artificial intelligence (AI) referred to as a deep neural network to analyse real-world behaviour in various social contexts across different cultures of the world. English-speaking volunteers in India trained the machine learning algorithm to identify 16 different patterns of facial movement associated with distinct English-language categories of emotion.
Nonparametric Independence Testing for Small Sample Sizes
This paper deals with the problem of nonparametric independence testing, a fundamental decision-theoretic problem that asks if two arbitrary (possibly multivariate) random variables $X,Y$ are independent or not, a question that comes up in many fields like causality and neuroscience. While quantities like correlation of $X,Y$ only test for (univariate) linear independence, natural alternatives like mutual information of $X,Y$ are hard to estimate due to a serious curse of dimensionality. A recent approach, avoiding both issues, estimates norms of an \textit{operator} in Reproducing Kernel Hilbert Spaces (RKHSs). Our main contribution is strong empirical evidence that by employing \textit{shrunk} operators when the sample size is small, one can attain an improvement in power at low false positive rates. We analyze the effects of Stein shrinkage on a popular test statistic called HSIC (Hilbert-Schmidt Independence Criterion). Our observations provide insights into two recently proposed shrinkage estimators, SCOSE and FCOSE - we prove that SCOSE is (essentially) the optimal linear shrinkage method for \textit{estimating} the true operator; however, the non-linearly shrunk FCOSE usually achieves greater improvements in \textit{test power}. This work is important for more powerful nonparametric detection of subtle nonlinear dependencies for small samples.
Nonparametric Independence Testing for Small Sample Sizes
Ramdas, Aaditya (Carnegie Mellon University) | Wehbe, Leila (Carnegie Mellon University)
It is also useful for scientific discovery like in neuroscience, like correlation of X, Y only test for (univariate) to see if a stimulus X (say an image) is independent linear independence, natural alternatives like of the brain activity Y (say fMRI) in a relevant part of mutual information of X, Y are hard to estimate the brain. Since detecting nonlinear correlations is much easier due to a serious curse of dimensionality. A recent than estimating a nonparametric regression function (of approach, avoiding both issues, estimates norms of Y onto X), it can be done at smaller sample sizes, with further an operator in Reproducing Kernel Hilbert Spaces samples collected for estimation only if an effect is detected (RKHSs). Our main contribution is strong empirical by the hypothesis test. For such situations, correlation evidence that by employing shrunk operators only tests for univariate linear independence, while other when the sample size is small, one can attain an improvement statistics like mutual information that do characterize multivariate in power at low false positive rates. We independence are hard to estimate from data, suffering analyze the effects of Stein shrinkage on a popular from a serious curse of dimensionality. A recent popular test statistic called HSIC (Hilbert-Schmidt Independence approach for this problem (and a related two-sample testing Criterion). Our observations provide insights problem) involve the use of quantities defined in reproducing into two recently proposed shrinkage estimators, kernel Hilbert spaces (RKHSs) - see [Gretton et al., 2006; SCOSE and FCOSE - we prove that SCOSE Harchaoui et al., 2007; Gretton et al., 2005b; 2005a].