Figure1: Different mask criteria can be thought of as segmenting the 2D (wi = initial weight value,wf = final weight value) space into regionscorresponding tomaskvaluesof1vs0.

Weight-sharing quantization has emerged as a technique to reduce energy expenditure during inference in large neural networks by constraining their weights to a limited set of values.

Neural Information Processing Systems http://nips.cc/

Pruning the weights of randomly initialized neural networks plays an important role in the context of lottery ticket hypothesis. Ramanujan et al. (2020) empirically showed that only pruning the weights can achieve remarkable performance instead of optimizing the weight values. However, to achieve the same level of performance as the weight optimization, the pruning approach requires more parameters in the networks before pruning and thus more memory space. To overcome this parameter inefficiency, we introduce a novel framework to prune randomly initialized neural networks with iteratively randomizing weight values (IteRand). Theoretically, we prove an approximation theorem in our framework, which indicates that the randomizing operations are provably effective to reduce the required number of the parameters. We also empirically demonstrate the parameter efficiency in multiple experiments on CIFAR-10 and ImageNet.

The Drosophila mushroom body (MB) is known to be involved in olfactory learning and memory; the synaptic plasticity of the Kenyon cell (KC) to mushroom body output neuron (MBON) synapses plays a key role in the learning process. Previous research has focused on projection neuron (PN) to Kenyon cell (KC) connectivity within the MB; we examine how perturbations to the mushroom body circuit structure and changes in connectivity, specifically within the KC to mushroom body output neuron (MBON) neural circuit, affect the MBONs' ability to distinguish between odor classes. We constructed a neural network that incorporates the connectivity between PNs, KCs, and MBONs. To train our model, we generated ten artificial input classes, which represent the projection neuron activity in response to different odors. We collected data on the number of KC-to-MBON connections, MBON error rates, and KC-to-MBON synaptic weights, among other metrics. We observed that MBONs with very few presynaptic KCs consistently performed worse than others in the odor classification task. The developmental types of KCs also played a significant role in each MBON's output. We performed random and targeted KC ablation and observed that ablating developmentally mature KCs had a greater negative impact on MBONs' learning capacity than ablating immature KCs. Random and targeted pruning of KC-MBON synaptic connections yielded results largely consistent with the ablation experiments. To further explore the various types of KCs, we also performed rewiring experiments in the PN to KC circuit. Our study furthers our understanding of olfactory neuroplasticity and provides important clues to understanding learning and memory in general. Understanding how the olfactory circuits process and learn can also have potential applications in artificial intelligence and treatments for neurodegenerative diseases.

Neural Information Processing Systems http://nips.cc/

--This paper addresses two critical challenges in analog In-Memory Computing (IMC) systems that limit their scalability and deployability: the computational unreliability caused by stuck-at faults (SAFs) and the high compilation overhead of existing fault-mitigation algorithms, namely Fault-Free (FF). T o overcome these limitations, we first propose a novel multi-bit weight representation technique, termed row-column hybrid grouping, which generalizes conventional column grouping by introducing redundancy across both rows and columns. This structural redundancy enhances fault tolerance and can be effectively combined with existing fault-mitigation solutions. Further acceleration is achieved through theoretical insights that identify fault patterns amenable to trivial solutions, significantly reducing computation. Experimental results on convolutional networks and small language models demonstrate the effectiveness of our approach, achieving up to 8%p improvement in accuracy, 150 faster compilation, and 2 energy efficiency gain compared to existing baselines. The In-Memory Computing (IMC) paradigm marks a trans-formative shift toward non-von Neumann architectures by allowing data processing to occur directly within the memory array [1]-[4], thereby minimizing the overhead associated with off-chip data movement [5]. Among various implementations, analog IMC systems based on Resistive Random Access Memory (ReRAM) crossbar arrays have emerged as a particularly promising solution. These systems perform energy-efficient matrix-vector multiplication (MVM) [3], [4], a core operation that forms the computational backbone of modern deep learning systems. As such, analog IMC has become a focal point in DNN acceleration and efficient AI research, spearheading cutting-edge investigations in approximate computing, heterogeneous computing, and alternative learning paradigms. To perform MVM in the analog domain, the weights are stored as conductance values in ReRAM cells; input features are applied as voltages to the word lines, and the resulting bit-line currents naturally multiply-and-accumulate following Ohm's and Kirchhoff's laws [6], [7].

This is simple to see as the ranks in the uneven depthwise are computed per input and the merging is done by output. The proposed RED method is summarized in algorithm 1. Note that we didn't describe the Strategy % removed parameters linear descending 77.90 constant 78.69 linear ascending 80.35 block 84.52 The constant strategy provides the best results. Following the study from Section 5.2, we want to empirically validate that hashing a DNN RED appears to be robust to dropout.