To reduce the computational complexity and memory requirements of models, various quantization techniques are widely used today. Floating-po int numbers are typically reduced to 16-, 8-, or 4-bit representations. This te chnique allows preserving the traditional gradient-based optimization of a differe ntiable loss function through backpropagation. More recent research is moving toward more extreme forms of qua ntiza-tion. For instance, [1] introduces BitNet b1.58, a novel 1.58-bit Lar ge Language Model (LLM) where each parameter is represented using ternary values { -1, 0, 1 } .

While there are many advantages to deploying machine learning models on edge devices, the resource constraints of mobile platforms, the dynamic nature of the environment, and differences between the distribution of training versus in-the-wild data make such deployments challenging. Current test-time adaptation methods are often memory-intensive and not designed to be quantization-compatible or deployed on low-resource devices. To address these challenges, we present LeanTTA, a novel backpropagation-free and stateless framework for quantized test-time adaptation tailored to edge devices. Our approach minimizes computational costs by dynamically updating normalization statistics without backpropagation, which frees LeanTTA from the common pitfall of relying on large batches and historical data, making our method robust to realistic deployment scenarios. Our approach is the first to enable further computational gains by combining partial adaptation with quantized module fusion. We validate our framework across sensor modalities, demonstrating significant improvements over state-of-the-art TTA methods, including a 15.7% error reduction, peak memory usage of only 11.2MB for ResNet18, and fast adaptation within an order-of-magnitude of normal inference speeds on-device. LeanTTA provides a robust solution for achieving the right trade offs between accuracy and system efficiency in edge deployments, addressing the unique challenges posed by limited data and varied operational conditions.

Diffusion models have shown remarkable performance in image synthesis, but they demand extensive computational and memory resources for training, fine-tuning and inference. Although advanced quantization techniques have successfully minimized memory usage for inference, training and fine-tuning these quantized models still require large memory possibly due to dequantization for accurate computation of gradients and/or backpropagation for gradient-based algorithms. However, memory-efficient fine-tuning is particularly desirable for applications such as personalization that often must be run on edge devices like mobile phones with private data. In this work, we address this challenge by quantizing a diffusion model with personalization via Textual Inversion and by leveraging a zeroth-order optimization on personalization tokens without dequantization so that it does not require gradient and activation storage for backpropagation that consumes considerable memory. Since a gradient estimation using zeroth-order optimization is quite noisy for a single or a few images in personalization, we propose to denoise the estimated gradient by projecting it onto a subspace that is constructed with the past history of the tokens, dubbed Subspace Gradient. In addition, we investigated the influence of text embedding in image generation, leading to our proposed time steps sampling, dubbed Partial Uniform Timestep Sampling for sampling with effective diffusion timesteps. Our method achieves comparable performance to prior methods in image and text alignment scores for personalizing Stable Diffusion with only forward passes while reducing training memory demand up to $8.2\times$.

Out-of-Distribution (OOD) detection is critical for ensuring the reliability of machine learning models in safety-critical applications such as autonomous driving and medical diagnosis. While deploying personalized OOD detection directly on edge devices is desirable, it remains challenging due to large model sizes and the computational infeasibility of on-device training. Federated learning partially addresses this but still requires gradient computation and backpropagation, exceeding the capabilities of many edge devices. To overcome these challenges, we propose SecDOOD, a secure cloud-device collaboration framework for efficient on-device OOD detection without requiring device-side backpropagation. SecDOOD utilizes cloud resources for model training while ensuring user data privacy by retaining sensitive information on-device. Central to SecDOOD is a HyperNetwork-based personalized parameter generation module, which adapts cloud-trained models to device-specific distributions by dynamically generating local weight adjustments, effectively combining central and local information without local fine-tuning. Additionally, our dynamic feature sampling and encryption strategy selectively encrypts only the most informative feature channels, largely reducing encryption overhead without compromising detection performance. Extensive experiments across multiple datasets and OOD scenarios demonstrate that SecDOOD achieves performance comparable to fully fine-tuned models, enabling secure, efficient, and personalized OOD detection on resource-limited edge devices. To enhance accessibility and reproducibility, our code is publicly available at https://github.com/Dystopians/SecDOOD.

There has been difficulty utilising conditional statements as part of the neural network graph (e.g. if input $> x$, pass input to network $N$). This is due to the inability to backpropagate through branching conditions. The Linear Array of Conditions, TOpologies with Separated Error-backpropagation (LACTOSE) Algorithm addresses this issue and allows the conditional use of available machine learning layers for supervised learning models. In this paper, the LACTOSE algorithm is applied to a simple use of DDSP, however, the main point is the development of the "if" conditional for DDSP use. The LACTOSE algorithm stores trained parameters for each user-specified numerical range and loads the parameters dynamically during prediction.

Spiking Neural Networks (SNNs) offer a biologically inspired computational paradigm that emulates neuronal activity through discrete spike-based processing. Despite their advantages, training SNNs with traditional backpropagation (BP) remains challenging due to computational inefficiencies and a lack of biological plausibility. This study explores the Forward-Forward (FF) algorithm as an alternative learning framework for SNNs. Unlike backpropagation, which relies on forward and backward passes, the FF algorithm employs two forward passes, enabling localized learning, enhanced computational efficiency, and improved compatibility with neuromorphic hardware. We introduce an FF-based SNN training framework and evaluate its performance across both non-spiking (MNIST, Fashion-MNIST, CIFAR-10) and spiking (Neuro-MNIST, SHD) datasets. Experimental results demonstrate that our model surpasses existing FF-based SNNs by over 5% on MNIST and Fashion-MNIST while achieving accuracy comparable to state-of-the-art backpropagation-trained SNNs. On more complex tasks such as CIFAR-10 and SHD, our approach outperforms other SNN models by up to 6% and remains competitive with leading backpropagation-trained SNNs. These findings highlight the FF algorithm's potential to advance SNN training methodologies and neuromorphic computing by addressing key limitations of backpropagation.

Weaknesses: I have some critical remarks: 1.) Weight transport problem. This problem is not solved in the model. In fact the model needs symmetric weights. Feedback alignment will probably not work here, as I assume that the existence of an equilibrium state necessitates symmetric weights. The authors claim that the update rules are local.

Following the author response, we had a long discussion. On the positive side, this is the first algorithm with local update rules that exactly simulates BP (at least asymptotically, given complete convergence at the initialization). On the negative side, all reviewers agreed this algorithm has some reduced plausibility. Specifically, in IL (original PCN) we have to present both input and output, and wait sufficient time until convergence. In contrast, in Z-IL and Fa-Z-IL, we have to first present (only) the input, also wait sufficient time until convergence, and then present the output; In addition, the learning rule becomes more complicated (through the introduction of the Phi function) and we must detect when "the change in error node is caused by feedback input" (which seems to require some global signals). This seems more complicated and less plausible then the original IL.

Multilayer Neural Networks (MNNs) are commonly trained using gradient descent-based methods, such as BackPropagation (BP). Inference in probabilistic graphical models is often done using variational Bayes methods, such as Expectation Propagation (EP). We show how an EP based approach can also be used to train deterministic MNNs. Specifically, we approximate the posterior of the weights given the data using a "mean-field" factorized distribution, in an online setting. Using online EP and the central limit theorem we find an analytical approximation to the Bayes update of this posterior, as well as the resulting Bayes estimates of the weights and outputs. Despite a different origin, the resulting algorithm, Expectation BackPropagation (EBP), is very similar to BP in form and efficiency. However, it has several additional advantages: (1) Training is parameter-free, given initial conditions (prior) and the MNN architecture. This is useful for large-scale problems, where parameter tuning is a major challenge.

Mein Concerns: The main motivation of the paper, to solve Backprop in spiking neurons, is not an open problem in computational neuroscience. In fact, learning in spiking neural networks using standard methods is not a problem at all as recent work shows. It has been demonstrated multiple times that Backprop can be applied without much changes by applying pseudo-derivatives to circumvent the non-differentiable spikes. This works very well in practice and scales up to midscale benchmark problems (and possibly beyond) without performance loss compared to classical (analog) neural networks. In this context it hard to pinpoint the main innovation of the manuscript.