Goto

Collaborating Authors

 npd




A Study of Neural Polar Decoders for Communication

Hirsch, Rom, Aharoni, Ziv, Pfister, Henry D., Permuter, Haim H.

arXiv.org Artificial Intelligence

Abstract--In this paper, we adapt and analyze Neural Polar Decoders (NPDs) for end-to-end communication systems. While prior work demonstrated the effectiveness of NPDs on synthetic channels, this study extends the NPD to real-world communication systems. The NPD was adapted to complete OFDM and single-carrier communication systems. T o satisfy practical system requirements, the NPD is extended to support any code length via rate matching, higher-order modulations, and robustness across diverse channel conditions. The NPD operates directly on channels with memory, exploiting their structure to achieve higher data rates without requiring pilots and a cyclic prefix. Although NPD entails higher computational complexity than the standard 5G polar decoder, its neural network architecture enables an efficient representation of channel statistics, resulting in manageable complexity suitable for practical systems. Experimental results over 5G channels demonstrate that the NPD consistently outperforms the 5G polar decoder in terms of BER, BLER, and throughput. These improvements are particularly significant for low-rate and short-block configurations, which are prevalent in 5G control channels. Furthermore, NPDs applied to single-carrier systems offer performance comparable to OFDM with lower PAPR, enabling effective single-carrier transmission over 5G channels. Polar codes, introduced by Arıkan in 2009 [1], are the first class of codes proven to achieve the capacity of symmetric binary-input discrete memoryless channels (B-DMCs) under low-complexity successive cancellation (SC) decoding. In 5G, polar codes are primarily used for control channels, where high performance is required with a low rate and short code length. Their inclusion in the 5G New Radio (NR) standard for uplink and downlink control information, use cases such as enhanced mobile broadband (eMBB) and broadcast channel (BCH) highlight their practical relevance in modern wireless communication systems.


Neural Polar Decoders for Deletion Channels

Aharoni, Ziv, Pfister, Henry D.

arXiv.org Artificial Intelligence

This paper introduces a neural polar decoder (NPD) for deletion channels with a constant deletion rate. Existing polar decoders for deletion channels exhibit high computational complexity of $O(N^4)$, where $N$ is the block length. This limits the application of polar codes for deletion channels to short-to-moderate block lengths. In this work, we demonstrate that employing NPDs for deletion channels can reduce the computational complexity. First, we extend the architecture of the NPD to support deletion channels. Specifically, the NPD architecture consists of four neural networks (NNs), each replicating fundamental successive cancellation (SC) decoder operations. To support deletion channels, we change the architecture of only one. The computational complexity of the NPD is $O(AN\log N)$, where the parameter $A$ represents a computational budget determined by the user and is independent of the channel. We evaluate the new extended NPD for deletion channels with deletion rates $δ\in\{0.01, 0.1\}$ and we verify the NPD with the ground truth given by the trellis decoder by Tal et al. We further show that due to the reduced complexity of the NPD, we are able to incorporate list decoding and further improve performance. We believe that the extended NPD presented here could have applications in future technologies like DNA storage.


Code Rate Optimization via Neural Polar Decoders

Aharoni, Ziv, Huleihel, Bashar, Pfister, Henry D, Permuter, Haim H

arXiv.org Artificial Intelligence

This paper proposes a method to optimize communication code rates via the application of neural polar decoders (NPDs). Employing this approach enables simultaneous optimization of code rates over input distributions while providing a practical coding scheme within the framework of polar codes. The proposed approach is designed for scenarios where the channel model is unknown, treating the channel as a black box that produces output samples from input samples. We employ polar codes to achieve our objectives, using NPDs to estimate mutual information (MI) between the channel inputs and outputs, and optimize a parametric model of the input distribution. The methodology involves a two-phase process: a training phase and an inference phase. In the training phase, two steps are repeated interchangeably. First, the estimation step estimates the MI of the channel inputs and outputs via NPDs. Second, the improvement step optimizes the input distribution parameters to maximize the MI estimate obtained by the NPDs. In the inference phase, the optimized model is used to construct polar codes. This involves incorporating the Honda-Yamamoto (HY) scheme to accommodate the optimized input distributions and list decoding to enhance decoding performance. Experimental results on memoryless and finite-state channels (FSCs) demonstrate the effectiveness of our approach, particularly in cases where the channel's capacity-achieving input distribution is non-uniform. For these cases, we show significant improvements in MI and bit error rates (BERs) over those achieved by uniform and independent and identically distributed (i.i.d.) input distributions, validating our method for block lengths up to 1024. This scalable approach has potential applications in real-world communication systems, bridging theoretical capacity estimation and practical coding performance.


Neural Polar Decoders for DNA Data Storage

Aharoni, Ziv, Pfister, Henry D.

arXiv.org Artificial Intelligence

Synchronization errors, such as insertions and deletions, present a fundamental challenge in DNA-based data storage systems, arising from both synthesis and sequencing noise. These channels are often modeled as insertion-deletion-substitution (IDS) channels, for which designing maximum-likelihood decoders is computationally expensive. In this work, we propose a data-driven approach based on neural polar decoders (NPDs) to design low-complexity decoders for channels with synchronization errors. The proposed architecture enables decoding over IDS channels with reduced complexity $O(AN log N )$, where $A$ is a tunable parameter independent of the channel. NPDs require only sample access to the channel and can be trained without an explicit channel model. Additionally, NPDs provide mutual information (MI) estimates that can be used to optimize input distributions and code design. We demonstrate the effectiveness of NPDs on both synthetic deletion and IDS channels. For deletion channels, we show that NPDs achieve near-optimal decoding performance and accurate MI estimation, with significantly lower complexity than trellis-based decoders. We also provide numerical estimates of the channel capacity for the deletion channel. We extend our evaluation to realistic DNA storage settings, including channels with multiple noisy reads and real-world Nanopore sequencing data. Our results show that NPDs match or surpass the performance of existing methods while using significantly fewer parameters than the state-of-the-art. These findings highlight the promise of NPDs for robust and efficient decoding in DNA data storage systems.


Neural Polarizer: A Lightweight and Effective Backdoor Defense via Purifying Poisoned Features

Zhu, Mingli, Wei, Shaokui, Zha, Hongyuan, Wu, Baoyuan

arXiv.org Artificial Intelligence

Recent studies have demonstrated the susceptibility of deep neural networks to backdoor attacks. Given a backdoored model, its prediction of a poisoned sample with trigger will be dominated by the trigger information, though trigger information and benign information coexist. Inspired by the mechanism of the optical polarizer that a polarizer could pass light waves with particular polarizations while filtering light waves with other polarizations, we propose a novel backdoor defense method by inserting a learnable neural polarizer into the backdoored model as an intermediate layer, in order to purify the poisoned sample via filtering trigger information while maintaining benign information. The neural polarizer is instantiated as one lightweight linear transformation layer, which is learned through solving a well designed bi-level optimization problem, based on a limited clean dataset. Compared to other fine-tuning-based defense methods which often adjust all parameters of the backdoored model, the proposed method only needs to learn one additional layer, such that it is more efficient and requires less clean data. Extensive experiments demonstrate the effectiveness and efficiency of our method in removing backdoors across various neural network architectures and datasets, especially in the case of very limited clean data.


Pruning Before Training May Improve Generalization, Provably

Yang, Hongru, Liang, Yingbin, Guo, Xiaojie, Wu, Lingfei, Wang, Zhangyang

arXiv.org Artificial Intelligence

It has been observed in practice that applying pruning-at-initialization methods to neural networks and training the sparsified networks can not only retain the testing performance of the original dense models, but also sometimes even slightly boost the generalization performance. Theoretical understanding for such experimental observations are yet to be developed. This work makes the first attempt to study how different pruning fractions affect the model's gradient descent dynamics and generalization. Specifically, this work considers a classification task for overparameterized two-layer neural networks, where the network is randomly pruned according to different rates at the initialization. It is shown that as long as the pruning fraction is below a certain threshold, gradient descent can drive the training loss toward zero and the network exhibits good generalization performance. More surprisingly, the generalization bound gets better as the pruning fraction gets larger. To complement this positive result, this work further shows a negative result: there exists a large pruning fraction such that while gradient descent is still able to drive the training loss toward zero (by memorizing noise), the generalization performance is no better than random guessing. This further suggests that pruning can change the feature learning process, which leads to the performance drop of the pruned neural network.


Here's why you won't see a new Nintendo Switch anytime soon

USATODAY - Tech Top Stories

With the latest and greatest new PlayStation and Xbox consoles in high demand, the video game makers at Nintendo surely must be ready to dump its nearly five-year-old Switch game system, right? Sure, Sony and Microsoft's new systems, released in November 2020, are lusted after and still nearly impossible to find on store shelves. But the Nintendo Switch is not always assured to be in stock, either. The Nintendo Switch has been the top-selling video game console in the U.S. for the past 30 months – yeah, you read that right, the leading unit seller for 2½ years – according to research firm The NPD Group. That surpasses the previous longest streak (21 months) held by the Xbox 360 having from August 2011 through April 2013, according to NPD. 'We need to continue to innovate':How charities are using NFTs as a way to fundraise The Nintendo Switch may not be as powerful as the newer Xbox Series X and PS5 consoles, which play games in 4K and eventually could display 8K games.


Video games breakout to record-setting levels as a perfect stay-at-home pastime amid coronavirus pandemic

USATODAY - Tech Top Stories

Video games are playing a big part in helping people cope during the coronavirus pandemic. Since earlier this spring with the onset of stay-at-home orders meant to stem the spread of COVID-19, more Americans have pressed play on video games. For some, games are an entertaining way to pass the time not spent on other pursuits. Others use them to stay connected with friends they used to see in person – and to bond with family members. Jennifer Fidler, 47, and her husband of Portland, Oregon, have been playing "Animal Crossing: New Horizons" with her two middle school-aged daughters since the pandemic led to school closings.