Technology
Provably Strict Generalisation Benefit for Invariance in Kernel Methods
It is a commonly held belief that enforcing invariance improves generalisation. Although this approach enjoys widespread popularity, it is only very recently that a rigorous theoretical demonstration of this benefit has been established. In this work we build on the function space perspective of Elesedy and Zaidi [8] to derive a strictly non-zero generalisation benefit of incorporating invariance in kernel ridge regression when the target is invariant to the action of a compact group. We study invariance enforced by feature averaging and find that generalisation is governed by a notion of effective dimension that arises from the interplay between the kernel and the group. In building towards this result, we find that the action of the group induces an orthogonal decomposition of both the reproducing kernel Hilbert space and its kernel, which may be of interest in its own right.
Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction
Self-Supervised Learning (SSL) has been shown to learn useful and informationpreserving representations. Neural Networks (NNs) are widely applied, yet their weight space is still not fully understood. Therefore, we propose to use SSL to learn hyper-representations of the weights of populations of NNs. To that end, we introduce domain specific data augmentations and an adapted attention architecture. Our empirical evaluation demonstrates that self-supervised representation learning in this domain is able to recover diverse NN model characteristics. Further, we show that the proposed learned representations outperform prior work for predicting hyper-parameters, test accuracy, and generalization gap as well as transfer to out-of-distribution settings. Code and datasets are publicly available1.
World's longest tiramisu record broken in London
World's longest tiramisu record broken in London The record for the world's longest tiramisu has been broken in London. One-hundred Italian chefs gathered at Chelsea Town Hall on Saturday and Sunday to whip up a tiramisu long enough to topple the previous record set by Milanese Galbani in Milan, which stood at 273.5m (897ft). As per Guinness World Record rules, the record-breaking tiramisu was made and assembled live on site, and used 50,000 ladyfinger biscuits and more than 3,000 eggs. The record was broken with the dessert measuring 440.6 m (1,445ft). Mirko Ricci, the man behind the London record attempt, originally held the record in 2017 in Italy, but another Italian team broke that in 2019.
Smoke and Mirrors in Causal Downstream Tasks
Machine Learning and AI have the potential to transform data-driven scientific discovery, enabling accurate predictions for several scientific phenomena. As many scientific questions are inherently causal, this paper looks at the causal inference task of treatment effect estimation, where the outcome of interest is recorded in high-dimensional observations in a Randomized Controlled Trial (RCT). Despite being the simplest possible causal setting and a perfect fit for deep learning, we theoretically find that many common choices in the literature may lead to biased estimates. To test the practical impact of these considerations, we recorded ISTAnt, the first real-world benchmark for causal inference downstream tasks on high-dimensional observations as an RCT studying how garden ants (Lasius neglectus) respond to microparticles applied onto their colony members by hygienic grooming. Comparing 6 480 models fine-tuned from state-of-the-art visual backbones, we find that the sampling and modeling choices significantly affect the accuracy of the causal estimate, and that classification accuracy is not a proxy thereof. We further validated the analysis, repeating it on a synthetically generated visual data set controlling the causal model. Our results suggest that future benchmarks should carefully consider real downstream scientific questions, especially causal ones. Further, we highlight guidelines for representation learning methods to help answer causal questions in the sciences.
Metal detectorists discover rare, Anglo-Saxon coins likely hidden from Vikings
The hoard was likely buried sometime between 871 and 874 in present-day Worcestershire. More information Adding us as a Preferred Source in Google by using this link indicates that you would like to see more of our content in Google News results. One of the silver coins in the hands of their finder, moments after being found. Breakthroughs, discoveries, and DIY tips sent six days a week. In a classic example of lucky metal detectorists triggering an archaeological investigation, a group of metal detecting enthusiasts in England discovered a rare hoard of early medieval Anglo-Saxon coins in the parish of Bickmarsh, Worcestershire.