Biological synaptic pruning removes weak neural connections to improve efficiency, while standard dropout in artificial networks randomly deactivates neurons without considering connection importance. We propose a magnitude-based synaptic pruning method that better emulates biological processes by gradually removing connections according to their contribution to model performance. Integrated directly into the training loop as a dropout replacement, our method computes weight importance from absolute magnitudes across layers and applies a cubic schedule to progressively increase global sparsity. At regular intervals, pruning masks are updated by thresholding weights, permanently removing low-importance connections while preserving gradient flow for active ones. This continuous, data-driven pruning removes the need for separate pruning and fine-tuning phases. We evaluated the method across multiple time series forecasting architectures, including Recurrent Neural Networks, Long Short-Term Memory, and Patch Time Series Transformer models, using four datasets. Our synaptic pruning approach achieved the best overall performance ranking across all architectures, with statistically significant improvements confirmed by Friedman tests ( p < 0. 01). In financial forecasting tasks, it reduced Mean Absolute Error by up to 20% compared to models using no dropout or standard dropout, with reductions reaching 52% in select transformer models. The proposed mechanism advances regularization by coupling dynamic weight elimination with progressive sparsification during training.

Large molecular representation models pre-trained on massive unlabeled data have shown great success in predicting molecular properties. However, these models may tend to overfit the fine-tuning data, resulting in over-confident predictions on test data that fall outside of the training distribution. To address this issue, uncertainty quantification (UQ) methods can be used to improve the models' calibration of predictions. Although many UQ approaches exist, not all of them lead to improved performance. While some studies have included UQ to improve molecular pre-trained models, the process of selecting suitable backbone and UQ methods for reliable molecular uncertainty estimation remains underexplored. To address this gap, we present MUBen, which evaluates different UQ methods for state-of-the-art backbone molecular representation models to investigate their capabilities. By fine-tuning various backbones using different molecular descriptors as inputs with UQ methods from different categories, we critically assess the influence of architectural decisions and training strategies. Our study offers insights for selecting UQ for backbone models, which can facilitate research on uncertainty-critical applications in fields such as materials science and drug discovery.

Within the last few years, there has been a move towards using statistical models in conjunction with neural networks with the end goal of being able to better answer the question, "what do our models know?". From this trend, classical metrics such as Prediction Interval Coverage Probability (PICP) and new metrics such as calibration error have entered the general repertoire of model evaluation in order to gain better insight into how the uncertainty of our model compares to reality. One important component of uncertainty modeling is model uncertainty (epistemic uncertainty), a measurement of what the model does and does not know. However, current evaluation techniques tends to conflate model uncertainty with aleatoric uncertainty (irreducible error), leading to incorrect conclusions. In this paper, using posterior predictive checks, we show how calibration error and its variants are almost always incorrect to use given model uncertainty, and further show how this mistake can lead to trust in bad models and mistrust in good models. Though posterior predictive checks has often been used for in-sample evaluation of Bayesian models, we show it still has an important place in the modern deep learning world.

While deep learning methods continue to improve This paper describes initial work on URSABench, an open in predictive accuracy on a wide range source suite of benchmarking tools for assessment of approximate of application domains, significant issues remain Bayesian inference methods applied to deep with other aspects of their performance including neural network classification tasks. URSABench includes their ability to quantify uncertainty and their benchmark models, data sets, tasks and evaluation metrics robustness. Recent advances in approximate focused on simultaneously assessing the uncertainty Bayesian inference hold significant promise for quantification performance, robustness, computational scalability addressing these concerns, but the computational and accuracy of learning and inference methods.

Deep neural networks (NNs) are powerful black box predictors that have recently achieved impressive performance on a wide spectrum of tasks. Quantifying predictive uncertainty in NNs is a challenging and yet unsolved problem. Bayesian NNs, which learn a distribution over weights, are currently the state-of-the-art for estimating predictive uncertainty; however these require significant modifications to the training procedure and are computationally expensive compared to standard (non-Bayesian) NNs. We propose an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates. Through a series of experiments on classification and regression benchmarks, we demonstrate that our method produces well-calibrated uncertainty estimates which are as good or better than approximate Bayesian NNs. To assess robustness to dataset shift, we evaluate the predictive uncertainty on test examples from known and unknown distributions, and show that our method is able to express higher uncertainty on out-of-distribution examples. We demonstrate the scalability of our method by evaluating predictive uncertainty estimates on ImageNet.