Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance. However, the differences between the learned solutions of networks which generalize and those which do not remain unclear. Additionally, the tuning properties of single directions (defined as the activation of a single unit or some linear combination of units in response to some input) have been highlighted, but their importance has not been evaluated. Here, we connect these lines of inquiry to demonstrate that a network's reliance on single directions is a good predictor of its generalization performance, across networks trained on datasets with different fractions of corrupted labels, across ensembles of networks trained on datasets with unmodified labels, across different hyperparameters, and over the course of training. While dropout only regularizes this quantity up to a point, batch normalization implicitly discourages single direction reliance, in part by decreasing the class selectivity of individual units. Finally, we find that class selectivity is a poor predictor of task importance, suggesting not only that networks which generalize well minimize their dependence on individual units by reducing their selectivity, but also that individually selective units may not be necessary for strong network performance.

DNN is presenting human-level performance for many complex intelligent tasks in real-world applications. However, it also introduces ever-increasing security concerns. For example, the emerging adversarial attacks indicate that even very small and often imperceptible adversarial input perturbations can easily mislead the cognitive function of deep learning systems (DLS). Existing DNN adversarial studies are narrowly performed on the ideal software-level DNN models with a focus on single uncertainty factor, i.e. input perturbations, however, the impact of DNN model reshaping on adversarial attacks, which is introduced by various hardware-favorable techniques such as hash-based weight compression during modern DNN hardware implementation, has never been discussed. In this work, we for the first time investigate the multi-factor adversarial attack problem in practical model optimized deep learning systems by jointly considering the DNN model-reshaping (e.g. HashNet based deep compression) and the input perturbations. We first augment adversarial example generating method dedicated to the compressed DNN models by incorporating the software-based approaches and mathematical modeled DNN reshaping. We then conduct a comprehensive robustness and vulnerability analysis of deep compressed DNN models under derived adversarial attacks. A defense technique named "gradient inhibition" is further developed to ease the generating of adversarial examples thus to effectively mitigate adversarial attacks towards both software and hardware-oriented DNNs. Simulation results show that "gradient inhibition" can decrease the average success rate of adversarial attacks from 87.99% to 4.77% (from 86.74% to 4.64%) on MNIST (CIFAR-10) benchmark with marginal accuracy degradation across various DNNs.

International Centre for Theoretical Physics, Trieste, Italy Stochasticity and limited precision of synaptic weights in neural network models are key aspects of both biological and hardware modeling of learning processes. Here we show that a neural network model with stochastic binary weights naturally gives prominence to exponentially rare dense regions of solutions with a number of desirable properties such as robustness and good generalization performance, while typical solutions are isolated and hard to find. Binary solutions of the standard perceptron problem are obtained from a simple gradient descent procedure on a set of real values parametrizing a probability distribution over the binary synapses. Both analytical and numerical results are presented. An algorithmic extension aimed at training discrete deep neural networks is also investigated. Learning can be regarded as an optimization process over the connection weights of a neural network. In nature, synaptic weights are known to be plastic, low precision and unreliable, and it is an interesting issue to understand if this stochasticity can help learning or if it is an obstacle.

Artificial intelligence originally aspired to replace doctors. Researchers imagined robots that could ask you questions, run the answers through an algorithm that would learn with experience and tell whether you had the flu or a cold. However, those promises largely failed, as artificial intelligent algorithms were too rudimentary to perform those functions. Particularly tricky was the variability between people, which caused basic machine learning algorithms to miss the patterns. Eventually though, a subset of AI called deep learning became sensitive enough to recognize speech from voice data.

Deep learning has the potential to revolutionise the insurance sector – but the challenge is how to make the artificial intelligence (AI) models auditable. Check out the latest findings on how the hype around artificial intelligence could be sowing damaging confusion. Also, read a number of case studies on how enterprises are using AI to help reach business goals around the world. You forgot to provide an Email Address. This email address doesn't appear to be valid.

To borrow a punch line from Duke professor Dan Ariely, artificial intelligence is like teenage sex: "Everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it." Even though AI systems can now learn a game and beat champions within hours, they are hard to apply to business applications. M.I.T. Sloan Management Review and Boston Consulting Group surveyed 3,000 business executives and found that while 85 percent of them believed AI would provide their companies with a competitive advantage, only one in 20 had "extensively" incorporated it into their offerings or processes. The challenge is that implementing AI isn't as easy as installing software. It requires expertise, vision, and information that isn't easily accessible.

Deep learning is having a large impact on the field of natural language processing. But, as a beginner, where do you start? Both deep learning and natural language processing are huge fields. What are the salient aspects of each field to focus on and which areas of NLP is deep learning having the most impact? In this post, you will discover a primer on deep learning for natural language processing.

Electronic Arts (EA) yesterday revealed its latest title: a custom game environment for deep learning networks to learn how to play video games. In the future the "computer" player in games won't rely on basic scripts; it'll react to you, and play against you, using the same information and controls a human player does. If you're a gamer, you've probably played one of EA's games. The list of hits in the company's catalog contains some of the greatest selling franchises of all time. Many of us have grown up with games like Battlefield, Madden, and FIFA, each titles with robust computer (CPU) opponents built-in.

A new artificial intelligence that is learning to understand the'thoughts' of others has been built by Google-owned research firm DeepMind. The software is capable of predicting what other AIs will do, and can even understand whether they hold'false beliefs' about the world around them. DeepMind reports its bot can now pass a key psychological test that most children only develop the skills for at around age four. Its proficiency in this'theory of mind' test may lead to robots that can think more like humans. DeepMind reports its bot can now pass a key psychological test that most children only develop the skills for around age four.

Today, data is the'oil' that is driving every aspect of the business. And, in a scientific parlance, this data is reusable, replenishable and insightful. Every insight gleaned with data becomes valuable each passing day. With the coming of age of the Internet of Things (IoT), super-connectivity, data management, and analytics, data for business is a gold rush for all modern organizations. We provide you key insights on how to start on the path to data science success and make a dash into the'Great Rush'.