Goto

Collaborating Authors

 Denmark


Fed Free: Breaking Knowledge-sharing Barriers through Layer-wise Alignment in Heterogeneous Federated Learning

Neural Information Processing Systems

Heterogeneous Federated Learning (HtFL) enables collaborative learning across clients with diverse model architectures and non-IID data distributions, which are prevalent in real-world edge computing applications. Existing HtFL approaches typically employ proxy datasets to facilitate knowledge sharing or implement coarse-grained model-level knowledge transfer. However, such approaches not only elevate risks of user privacy leakage but also lead to the loss of fine-grained model-specific knowledge, ultimately creating barriers to effective knowledge sharing. To address these challenges, we propose FedFree, a novel proxy-datafree and model-free HtFL framework featuring two key innovations. First, FedFree introduces a reverse layer-wise knowledge transfer mechanism that aggregates heterogeneous client models into a global model solely using Gaussianbased pseudo-data, eliminating reliance on proxy datasets. Second, it leverages Knowledge Gain Entropy (KGE) to guide targeted layer-wise knowledge alignment, ensuring that each client receives the most relevant global updates tailored to its specific architecture. We provide rigorous theoretical convergence guarantees for FedFree and conduct extensive experiments on CIFAR-10 and CIFAR100. Results demonstrate that FedFree achieves substantial performance gains, with relative accuracy improving up to 46.3% over state-of-the-art baselines.


Autism and ADHD are on the rise due to widening diagnostic criteria

New Scientist

A study of 140,000 people suggests that a broadening of the diagnostic criteria for autism and ADHD explains the sharp rise in diagnoses, but that doesn't mean too many people are being told they are autistic or have ADHD We may be beginning to understand what is behind the recent explosion in diagnoses of ADHD and autism . A study of 140,000 people in Denmark reveals that those recently diagnosed with ADHD or autism have fewer genetic variations associated with them than people diagnosed a decade earlier. This suggests that a broadening of the diagnostic criteria is behind the rise, but it doesn't support claims that ADHD and autism are being overdiagnosed. Diagnoses for autism and ADHD have risen up to tenfold around the world over the past two decades, particularly among girls and adults. Several possibilities have been put forward to explain this, including better awareness and understanding, a broadening of the diagnostic criteria, and even the commercial interests of pharmaceutical companies and private diagnostic clinics.


Gaussian Processes with Sample Paths in Reproducing Kernel Banach Spaces

arXiv.org Machine Learning

We investigate the connection between Gaussian processes and Gaussian random elements in reproducing kernel Banach spaces. We show that the covariance operator of a weak second-order Radon probability measure on such a space is uniquely determined by a positive definite function. In the Gaussian case, we characterize those positive definite functions that arise from covariance operators in terms of $γ$-radonifying operators. Building on these results, we extend the classical Driscoll theorem to the Banach space setting.


Mapping the Schedule x Bit-Width Boundary in Sub-100M Quantisation-Aware Training

arXiv.org Machine Learning

We test whether the optimal learning-rate schedule depends on bit-width during from-initialisation quantisation-aware training (QAT) for sub-100M decoder language models. A 720-run factorial grid (Phase 2) over bit-width x warmdown fraction x LR magnitude x model size x seed (FP16/INT8/INT6, 15M-100M, 5 seeds) finds the optimal warmdown is 33% at every (bit-width, size) cell. The primary hypothesis -- that INT6 QAT requires a different schedule than higher-precision training -- is falsified at FP16/INT8/INT6. A 625-run follow-up (Phase 5) probes the null along five axes: optimiser (AdamW), schedule shape (cosine), training length (up to 9x more iterations), an extended size sweep (5M-350M), and an INT4 sweep from 3M to 100M. The null is robust under all three setup changes. The INT6 penalty follows a log-linear scaling law whose fit on Phase 2 predicts the five held-out Phase 5 sizes (5M, 8M, 175M, 250M, 350M) within their 95% prediction intervals (5/5). For INT4 the picture is sharper than the higher precisions: at 50M and 100M, wd33 is decisively optimal (paired z ~ 12-15, 10/10 seeds); below 50M, across the six tested sizes from 3M to 30M, no individual size shows a statistically significant schedule preference and the per-size mean penalty oscillates within seed-level noise. The boundary is therefore a transition between a noise-dominated regime below 50M and a decisive wd33 regime at and above 50M, not a clean wd10 region. A weight-to-grid-distance probe falsifies the simplest mechanism for the FP16/INT8/INT6 null result (rapid grid-snapping): pre-warmdown, INT6-QAT weights sit at essentially the same distance from the INT6 grid as FP16 weights (ratio ~ 1.04). Practical recommendation: at sub-100M scale, tune the LR schedule once at FP16 and apply unchanged to INT8/INT6 QAT; for INT4 at 50M+ use wd33; for INT4 below 50M the schedule choice is in the noise.


A Danish Couple's Maverick African Research Finds Its Moment in RFK Jr.'s Vaccine Policy

WIRED

The work of Peter Aaby and Christine Stabell Benn has long been controversial. Until Robert F. Kennedy Jr. became US health policy chief, most vaccine scientists tended to ignore it. In 1996, Guinea-Bissau seemed like an ideal research post for budding pediatrician Lone Graff Stensballe. Her supervisor, a fellow Dane named Peter Aaby, had spent nearly two decades collecting data on 100,000 people living in the mud brick homes of the West African country's capital. Aaby and his partner, Christine Stabell Benn, believed that the years of research in the impoverished country had yielded a major discovery about vaccines--and what they described as "non-specific effects": The measles and tuberculosis vaccines, which were derived from live, weakened viruses and bacteria, they said, boosted child survival beyond protecting against those particular pathogens. But, the scientists said, shots made from deactivated whole germs, or pieces of them, such as the diphtheria-tetanus-pertussis (DTP) shot, caused more deaths--especially in little girls--than getting no vaccine at all.





China's DeepSeek unveils latest models a year after upending global tech

Al Jazeera

China's DeepSeek unveils latest models a year after upending global tech China's DeepSeek has unveiled the latest versions of its signature artificial intelligence-powered chatbot, a year after its flagship model sent shockwaves through the global tech scene. The Chinese start-up launched preview versions of DeepSeek-V4-Pro and DeepSeek-V4-Flash on Friday as it touted its ability to go toe-to-toe with US rivals such as OpenAI and Google. The "flash" model has similar reasoning abilities to the "pro" version, while offering faster response times and more cost-effective pricing, the Hangzhou-based startup said. Like DeepSeek's previous chatbots, V4-Pro and V4-Flash follow an open-source model, meaning developers are free to use and modify them at will. The release comes after DeepSeek-R1 stunned the tech sector upon its launch in January last year with capabilities broadly comparable with those of ChatGPT and Gemini.


Post-hoc Self-explanation of CNNs

arXiv.org Machine Learning

Although standard Convolutional Neural Networks (CNNs) can be mathematically reinterpreted as Self-Explainable Models (SEMs), their built-in prototypes do not on their own accurately represent the data. Replacing the final linear layer with a $k$-means-based classifier addresses this limitation without compromising performance. This work introduces a common formalization of $k$-means-based post-hoc explanations for the classifier, the encoder's final output (B4), and combinations of intermediate feature activations. The latter approach leverages the spatial consistency of convolutional receptive fields to generate concept-based explanation maps, which are supported by gradient-free feature attribution maps. Empirical evaluation with a ResNet34 shows that using shallower, less compressed feature activations, such as those from the last three blocks (B234), results in a trade-off between semantic fidelity and a slight reduction in predictive performance.