Advances in designing and training deep neural networks have led to the principle that the large and deeper a network is, the better it can perform. As a result, computational resources have become a key limiting factor in achieving better performance. One strategy to improve network capabilities while decreasing computation required is to replace dense fully-connected and convolutional layers with sparse layers. In this paper we experiment with training on sparse neural network topologies. First, we test pruning-based sparse topologies, which use a network topology obtained by initially training a dense network and then pruning low-weight connections. Second, we test RadiX-Nets, a class of sparse network structures with proven connectivity and sparsity properties. Results show that compared to dense topologies, sparse structures show promise in training potential but also can exhibit highly nonlinear convergence, which merits further study.

Although stochastic gradient descent (SGD) is a driving force behind the recent success of deep learning, our understanding of its dynamics in a high-dimensional parameter space is limited. In recent years, some researchers have used the stochasticity of minibatch gradients, or the signal-to-noise ratio, to better characterize the learning dynamics of SGD. Inspired from these work, we here analyze SGD from a geometrical perspective by inspecting the stochasticity of the norms and directions of minibatch gradients. We propose a model of the directional concentration for minibatch gradients through von Mises-Fisher (VMF) distribution, and show that the directional uniformity of minibatch gradients increases over the course of SGD. We empirically verify our result using deep convolutional networks and observe a higher correlation between the gradient stochasticity and the proposed directional uniformity than that against the gradient norm stochasticity, suggesting that the directional statistics of minibatch gradients is a major factor behind SGD. Stochastic gradient descent (SGD) has been a driving force behind the recent success of deep learning.

Neural networks are among the most accurate supervised learning methods in use today. However, their opacity makes them difficult to trust in critical applications, especially when conditions in training may differ from those in practice. Recent efforts to develop explanations for neural networks and machine learning models more generally have produced tools to shed light on the implicit rules behind predictions. These tools can help us identify when models are right for the wrong reasons. However, they do not always scale to explaining predictions for entire datasets, are not always at the right level of abstraction, and most importantly cannot correct the problems they reveal. In this thesis, we explore the possibility of training machine learning models (with a particular focus on neural networks) using explanations themselves. We consider approaches where models are penalized not only for making incorrect predictions but also for providing explanations that are either inconsistent with domain knowledge or overly complex. These methods let us train models which can not only provide more interpretable rationales for their predictions but also generalize better when training data is confounded or meaningfully different from test data (even adversarially so).

Deep neural networks (DNNs) are vulnerable to adversarial examples, perturbations carefully crafted to fool the targeted DNN, in both the non-targeted and targeted case. In the non-targeted case, the attacker simply aims to induce misclassification. In the targeted case, the attacker aims to induce classification to a specified target class. In addition, it has been observed that strong adversarial examples can transfer to unknown models, yielding a serious security concern. The NIPS 2017 competition was organized to accelerate research in adversarial attacks and defenses, taking place in the realistic setting where submitted adversarial attacks attempt to transfer to submitted defenses. The CAAD 2018 competition took place with nearly identical rules to the NIPS 2017 one. Given the requirement that the NIPS 2017 submissions were to be open-sourced, participants in the CAAD 2018 competition were able to directly build upon previous solutions, and thus improve the state-of-the-art in this setting. Our team participated in the CAAD 2018 competition, and won 1st place in both attack subtracks, non-targeted and targeted adversarial attacks, and 3rd place in defense. We outline our solutions and development results in this article. We hope our results can inform researchers in both generating and defending against adversarial examples.

Currently primarily used in pattern recognition and classification applications supported by very large data sets. Sample vendors: Deep Instinct, Ersatz Labs, Fluid AI, MathWorks, Peltarion, Saffron Technology, Sentient Technologies Biometrics: Enable more natural interactions between humans and machines, including but not limited to image and touch recognition, speech, and body language. Currently used primarily in market research.

Take the next step in implementing various common and not-so-common neural networks with Tensorflow 1.x In this book, you will learn how to efficiently use TensorFlow, Google's open source framework for deep learning. You will implement different deep learning networks such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Deep Q-learning Networks (DQNs), and Generative Adversarial Networks (GANs) with easy to follow independent recipes. You will learn how to make Keras as backend with TensorFlow. With a problem-solution approach, you will understand how to implement different deep neural architectures to carry out complex tasks at work.

Video: How to set up your Raspberry Pi 3 Model B . Besides putting a Raspberry Pi to work on a mini Mars rover, it's now going to be a lot easier to use Google's TensorFlow artificial-intelligence framework with the low-powered computer. Developers with Raspberry Pi have already been able to use TensorFlow in a variety of ways to add deep-learning models so that cheap or expensive hardware can do things like image classification. While TensorFlow can be used on Linux, Windows, Android, macOS, and iOS, it's hard to find cheaper hardware than the $35 Raspberry Pi. But as noted by Pete Warden, a software engineer and lead of the TensorFlow mobile and embedded team, running TensorFlow on Raspberry Pi "has involved a lot of work".

Established in 2018, the Machine Intelligence Technology Laboratory comprises of a group of outstanding scientists and engineers, with research centers located in Hangzhou, Beijing, Seattle, Silicon Valley, and Singapore. Machine Intelligence Technology Laboratory is Alibaba's core team responsible for the research and development of artificial intelligence technologies. Relying on Alibaba's valuable massive data and machine learning/deep learning technologies, the lab has developed image recognition, speech interaction, natural language understanding, intelligent decision-making, and other core artificial intelligence technologies. It fully empowers Alibaba Group's important businesses such as e-commerce, finance, logistics, social interaction, and entertainment, and also provides outputs to ecosystem partners to jointly build a smart future. Image Search is an intelligent image search product that enables search by image using image recognition and search functions, based on deep learning and large-scale machine learning technologies.

But hey, when your ad's copywriter is an artificial intelligence, you have to give it some credit for trying to connect with us weirdo organ-sack humans. Burger King's newest TV ad campaign claims each spot was "created by artificial intelligence," an explanation that precedes some truly bizarre voice-overs, such as, "Bed of lettuce for you to sleep on, bed of mayonnaise for extra sleep," and, "Burger King logo's chicken is the new potato." In a statement announcing the campaign--which will air during prime time on cable networks including MTV, History, TBS, Adult Swim and E!--the brand refers to as the creation of "a new deep learning algorithm that could give a glimpse into what the future of marketing and communications could look like." "AI, bots, machine learning, deep learning algorithms, blockchain, among others--these are all topical as we explore our future in marketing," says Marcelo Pascoa, Burger King's global head of brand marketing. "But we need to avoid getting lost in the sea of technology innovation and buzzwords and forget what really matters. Artificial intelligence is not a substitute for a great creative idea coming from a real person."

Today we are announcing our collaboration with DeepMind, a world leader in artificial intelligence research. The partnership will enable DeepMind to develop virtual environments and tasks in support of their fundamental AI research program. DeepMind researchers are addressing huge AI problems, and they have selected Unity as a primary research platform for creating complex virtual environments that will enable the development of algorithms capable of learning to solve complex tasks. We believe the future of AI is being shaped by increasingly sophisticated human-machine interactions, and Unity is proud to be the engine that is enabling these interactions. Unity is no stranger to forging thought-leadership in the AI field.