Goto

Collaborating Authors

 Strength Low


Prompt Your Brain: Scaffold Prompt Tuning for Efficient Adaptation of fMRI Pre-trained Model

Dong, Zijian, Wu, Yilei, Chen, Zijiao, Zhang, Yichi, Jin, Yueming, Zhou, Juan Helen

arXiv.org Artificial Intelligence

We introduce Scaffold Prompt Tuning (ScaPT), a novel prompt-based framework for adapting large-scale functional magnetic resonance imaging (fMRI) pre-trained models to downstream tasks, with high parameter efficiency and improved performance compared to fine-tuning and baselines for prompt tuning. The full fine-tuning updates all pre-trained parameters, which may distort the learned feature space and lead to overfitting with limited training data which is common in fMRI fields. In contrast, we design a hierarchical prompt structure that transfers the knowledge learned from high-resource tasks to low-resource ones. This structure, equipped with a Deeply-conditioned Input-Prompt (DIP) mapping module, allows for efficient adaptation by updating only 2% of the trainable parameters. The framework enhances semantic interpretability through attention mechanisms between inputs and prompts, and it clusters prompts in the latent space in alignment with prior knowledge. Experiments on public resting state fMRI datasets reveal ScaPT outperforms fine-tuning and multitask-based prompt tuning in neurodegenerative diseases diagnosis/prognosis and personality trait prediction, even with fewer than 20 participants. It highlights ScaPT's efficiency in adapting pre-trained fMRI models to low-resource tasks.


Spatio-Temporal Surrogates for Interaction of a Jet with High Explosives: Part I -- Analysis with a Small Sample Size

Kamath, Chandrika, Franzman, Juliette S., Daub, Brian H.

arXiv.org Artificial Intelligence

Computer simulations, especially of complex phenomena, can be expensive, requiring high-performance computing resources. Often, to understand a phenomenon, multiple simulations are run, each with a different set of simulation input parameters. These data are then used to create an interpolant, or surrogate, relating the simulation outputs to the corresponding inputs. When the inputs and outputs are scalars, a simple machine learning model can suffice. However, when the simulation outputs are vector valued, available at locations in two or three spatial dimensions, often with a temporal component, creating a surrogate is more challenging. In this report, we use a two-dimensional problem of a jet interacting with high explosives to understand how we can build high-quality surrogates. The characteristics of our data set are unique - the vector-valued outputs from each simulation are available at over two million spatial locations; each simulation is run for a relatively small number of time steps; the size of the computational domain varies with each simulation; and resource constraints limit the number of simulations we can run. We show how we analyze these extremely large data-sets, set the parameters for the algorithms used in the analysis, and use simple ways to improve the accuracy of the spatio-temporal surrogates without substantially increasing the number of simulations required.


SFCNeXt: a simple fully convolutional network for effective brain age estimation with small sample size

Fu, Yu, Huang, Yanyan, Dong, Shunjie, Wang, Yalin, Yu, Tianbai, Niu, Meng, Zhuo, Cheng

arXiv.org Artificial Intelligence

Deep neural networks (DNN) have been designed to predict the chronological age of a healthy brain from T1-weighted magnetic resonance images (T1 MRIs), and the predicted brain age could serve as a valuable biomarker for the early detection of development-related or aging-related disorders. Recent DNN models for brain age estimations usually rely too much on large sample sizes and complex network structures for multi-stage feature refinement. However, in clinical application scenarios, researchers usually cannot obtain thousands or tens of thousands of MRIs in each data center for thorough training of these complex models. This paper proposes a simple fully convolutional network (SFCNeXt) for brain age estimation in small-sized cohorts with biased age distributions. The SFCNeXt consists of Single Pathway Encoded ConvNeXt (SPEC) and Hybrid Ranking Loss (HRL), aiming to estimate brain ages in a lightweight way with a sufficient exploration of MRI, age, and ranking features of each batch of subjects. Experimental results demonstrate the superiority and efficiency of our approach.


Elon Musk's Neuralink 'botched experiments' revealed by former employee and internal lab notes

Daily Mail - Science & tech

'Botched experiments' by Elon Musk's Neuralink allegedly'kept suffering animals alive for no reason and malpractice caused monkey's brains to hemorrhage' during rushed brain chip testing, a former Neuralink employee and internal lab notes reveal. The billionaire's startup is accused of violating the Animal Welfare Act with its experiments at the University of California, Davis, from 2017 through 2020, which'sacrificed all the animals involved,' a former Neuralink employee, who asked to remain anonymous, told DailyMail.com. One case stood out to them- a monkey sacrificed ahead of schedule due to errors allegedly made during surgery. 'There was no reason to use it,' the former employee, who worked as a necropsy technician, told DailyMail.com. 'BioGlue was not FDA-approved for brain surgery and would never be able to be carried over to human trials.


Combined Pruning for Nested Cross-Validation to Accelerate Automated Hyperparameter Optimization for Embedded Feature Selection in High-Dimensional Data with Very Small Sample Sizes

May, Sigrun, Hartmann, Sven, Klawonn, Frank

arXiv.org Artificial Intelligence

Background: Embedded feature selection in high-dimensional data with very small sample sizes requires optimized hyperparameters for the model building process. For this hyperparameter optimization, nested cross-validation must be applied to avoid a biased performance estimation. The resulting repeated training with high-dimensional data leads to very long computation times. Moreover, it is likely to observe a high variance in the individual performance evaluation metrics caused by outliers in tiny validation sets. Therefore, early stopping applying standard pruning algorithms to save time risks discarding promising hyperparameter sets. Result: To speed up feature selection for high-dimensional data with tiny sample size, we adapt the use of a state-of-the-art asynchronous successive halving pruner. In addition, we combine it with two complementary pruning strategies based on domain or prior knowledge. One pruning strategy immediately stops computing trials with semantically meaningless results for the selected hyperparameter combinations. The other is a new extrapolating threshold pruning strategy suitable for nested-cross-validation with a high variance of performance evaluation metrics. In repeated experiments, our combined pruning strategy keeps all promising trials. At the same time, the calculation time is substantially reduced compared to using a state-of-the-art asynchronous successive halving pruner alone. Up to 81.3\% fewer models were trained achieving the same optimization result. Conclusion: The proposed combined pruning strategy accelerates data analysis or enables deeper searches for hyperparameters within the same computation time. This leads to significant savings in time, money and energy consumption, opening the door to advanced, time-consuming analyses.


Yield-predicting AI needs chemists to stop ignoring failed experiments

#artificialintelligence

Machine-learning algorithms that can predict reaction yields have remained elusive because chemists tend to bury low-yielding reactions in their lab notebooks instead of publishing them, researchers say. 'We have this image that failed experiments are bad experiments,' says Felix Strieth-Kalthoff. 'But they contain knowledge, they contain valuable information both for humans and for an AI.' Strieth-Kalthoff from the University of Toronto, Canada, and a team around Frank Glorius from Germany's University of Münster are asking chemists to start including not only their best but also their worst results in their papers. This, as well as unbiased reagent selection and reporting experimental procedures in a standardised format, will allow researchers to finally create yield-prediction algorithms. Retrosynthesis is already using machine-learning models to create shorter, cheaper or non-proprietary synthetic routes. But there have been few attempts at creating programs that predict yields.


Expressions: People around the world pull the same faces in similar social settings

Daily Mail - Science & tech

Experts from the US used artificial intelligence to analyse the faces seen in 6 million YouTube videos -- broadly finding similar expressions in similar social settings. However, the team warned, the findings have not confirmed whether emotions themselves are universal, with further research needed to explore this area. Previous, survey-based studies into whether expressions and emotions are universal have been constrained by language barriers and small sample sizes. The study was undertaken by emotion scientist Alan Cowen of the University of California, Berkeley and his colleagues. The researchers used a type of artificial intelligence (AI) referred to as a deep neural network to analyse real-world behaviour in various social contexts across different cultures of the world. English-speaking volunteers in India trained the machine learning algorithm to identify 16 different patterns of facial movement associated with distinct English-language categories of emotion.



Bad Data Science and Woody Allen

@machinelearnbot

For breakfast he requests wheat germ, organic honey and tiger's milk - food in 1973 thought to be healthy. The futuristic doctors reply: "You mean there was no deep fat? No steak or cream pies or... hot fudge?" and "Those were thought to be unhealthy... precisely the opposite of what we now know to be true." A recent article in the Wall Street Journal entitled "The Questionable Link Between Saturated Fat and Heart Disease" details scientific malpractice in research about what food is healthy or not. For over fifty years the scientific consensus was that fat - both saturated or not - is a "cause" of obesity, heart disease, and other chronic diseases.


Continuous Correlated Beta Processes

Goetschalckx, Robby (University of Dundee) | Poupart, Pascal (University of Waterloo) | Hoey, Jesse (University of Waterloo)

AAAI Conferences

In this paper we consider a (possibly continuous) space of Bernoulli experiments. We assume that the Bernoulli distributions of the points are correlated. All evidence data comes in the form of successful or failed experiments at different points. Current state-of-the-art methods for expressing a distribution over a continuum of Bernoulli distributions use logistic Gaussian processes or Gaussian copula processes. However, both of these require computationally expensive matrix operations (cubic in the general case). We introduce a more intuitive approach, directly correlating beta distributions by sharing evidence between them according to a kernel function, an approach which has linear time complexity. The approach can easily be extended to multiple outcomes, giving a continuous correlated Dirichlet process.This approach can be used for classification (both binary and multi-class) and learning the actual probabilities of the Bernoulli distributions. We show results for a number of data sets, as well as a case-study where a mixture of continuous beta processes is used as part of an automated stroke rehabilitation system.