In this paper, we derive norm-based generalization bounds for sparse ReLU neural networks, including convolutional neural networks. These bounds differ from previous ones because they consider the sparse structure of the neural network architecture and the norms of the convolutional filters, rather than the norms of the (Toeplitz) matrices associated with the convolutional layers. Theoretically, we demonstrate that these bounds are significantly tighter than standard norm-based generalization bounds. Empirically, they offer relatively tight estimations of generalization for various simple classification problems. Collectively, these findings suggest that the sparsity of the underlying target function and the model's architecture plays a crucial role in the success of deep learning.

However, CTC tends to produce highly peaky and overconfident distributions, which is a symptom of overfitting. To remedy this, we propose a regularization method based on maximum conditional entropy which penalizes peaky distributions and encourages exploration.

To accelerate therapeutic antibody discovery, computational methods, especially machine learning, have attracted considerable interest for predicting specific interactions between antibody candidates and target antigens such as viruses and bacteria.

Neural Information Processing Systems http://nips.cc/

By analyzing the unique properties and behaviors of backdoor attacks, we formulate trigger restoration as an optimization problem and design a novel metric to detect back-doored policies.

Microsoft and Nvidia plan to invest in Anthropic under a new tie-up that includes a $30bn commitment by the Claude maker to use Microsoft's cloud services, the latest high-profile deal binding together major players in the AI industry. Nvidia will commit up to $10bn to Anthropic and Microsoft up to $5bn, the companies said on Tuesday, without sharing more details. The announcement underscores the AI industry's insatiable appetite for computing power as companies race to build systems that can rival or surpass human intelligence. It also ties major OpenAI-backer Microsoft, as well as key AI chip supplier Nvidia, closer to one of the ChatGPT maker's biggest rivals. "We're increasingly going to be customers of each other. We will use Anthropic models, they will use our infrastructure and we'll go to market together," Microsoft CEO Satya Nadella said in a video.

Optimization of deep neural networks is a challenging non-convex problem.

Unsupervised pre-training methods utilizing large and diverse datasets have achieved tremendous success across a range of domains.