In this paper, we study norm-based regularization methods for neural networks. We compare existing penalization approaches and introduce two regularization strategies that extend classical ridge- and lasso-type penalties to neural network models. The first strategy modifies weight decay by incorporating the covariance structure of the input features into a ridge-type $\ell_2$ penalty, allowing regularization to account for feature dependence. The second combines an $\ell_1$ sparsity penalty with covariance-aware $\ell_2$ regularization, producing neural network weights that are both sparse and structurally informed. Monte Carlo simulations are used to evaluate these methods under different data-generating settings, followed by two real-data applications on building cooling-load prediction and leukemia cell-type classification from high-dimensional gene expression data. Across simulated and real-data examples, the proposed regularizers improve predictive performance on unseen data and provide more effective complexity control than standard norm-based penalties, particularly when features are correlated or high-dimensional.

This stands in sharp contrast to fully connected networks, which are not universal approximators if their width is the input dimensiond[21,11].

Thus, one important question one can ask here is how welldFnn(ˆµn,ˆνm)approximatesdFnn(µ,ν).

So far the community has considered`2 as the underlying distance.

Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this contribution, we provide a rigorous justification of these approaches for a two-layers neural network model called the committee machine. We also introduce a version of the approximate message passing (AMP) algorithm for the committee machine that allows to perform optimal learning in polynomial time for a large set of parameters.

Recurrent Neural Networks (RNNs) yield great results across many natural language processing applications, including speech recognition, machine translation, language modeling, and question answering [1,2,3,4,5].

We show the universality of depth-2 group convolutional neural networks (GCNNs) inaunified and constructivemanner based ontheridgelet theory.

The modernize result, our Second, the connects Ontheother77], theoutput BiLSTM. Weadopt AdamWoptimizer [wingthepreviousstudy [weadopt ratebatchsizesfor Sequencer2D-S, Sequencer2D-M, are 2048, 1536, and 1024, respectively.

This kernel is central to describe the generalization features ofANNs.

ModernCNNscanreachhundreds of millions of parameters and billions of operations, which makes it difficult to deploy. To alleviate aforementioned problem, various methods have been proposed to increase the efficiency of CNNs.