Deep Neural Networks (DNNs) have begun to thrive in the field of automation systems, owing to the recent advancements in standardising various aspects such as architecture, optimization techniques, and regularization. In this paper, we take a step towards a better understanding of Spectral Normalization (SN) and its potential for standardizing regularization of a wider range of Deep Learning models, following an empirical approach. We conduct several experiments to study their training dynamics, in comparison with the ubiquitous Batch Normalization (BN) and show that SN increases the gradient sparsity and controls the gradient variance. Furthermore, we show that SN suffers from a phenomenon, we call the mean-drift effect, which mitigates its performance. We, then, propose a weight reparameterization called as the Mean Spectral Normalization (MSN) to resolve the mean drift, thereby significantly improving the network's performance. Our model performs ~16% faster as compared to BN in practice, and has fewer trainable parameters. We also show the performance of our MSN for small, medium, and large CNNs - 3-layer CNN, VGG7 and DenseNet-BC, respectively - and unsupervised image generation tasks using Generative Adversarial Networks (GANs) to evaluate its applicability for a broad range of embedded automation tasks.

Deep Learning Accelerators are prone to faults which manifest in the form of errors in Neural Networks. Fault Tolerance in Neural Networks is crucial in real-time safety critical applications requiring computation for long durations. Neural Networks with high regularisation exhibit superior fault tolerance, however, at the cost of classification accuracy. In the view of difference in functionality, a Neural Network is modelled as two separate networks, i.e, the Feature Extractor with unsupervised learning objective and the Classifier with a supervised learning objective. Traditional approaches of training the entire network using a single supervised learning objective is insufficient to achieve the objectives of the individual components optimally. In this work, a novel multi-criteria objective function, combining unsupervised training of the Feature Extractor followed by supervised tuning with Classifier Network is proposed. The unsupervised training solves two games simultaneously in the presence of adversary neural networks with conflicting objectives to the Feature Extractor. The first game minimises the loss in reconstructing the input image for indistinguishability given the features from the Extractor, in the presence of a generative decoder. The second game solves a minimax constraint optimisation for distributional smoothening of feature space to match a prior distribution, in the presence of a Discriminator network. The resultant strongly regularised Feature Extractor is combined with the Classifier Network for supervised fine-tuning. The proposed Adversarial Fault Tolerant Neural Network Training is scalable to large networks and is independent of the architecture. The evaluation on benchmarking datasets: FashionMNIST and CIFAR10, indicates that the resultant networks have high accuracy with superior tolerance to stuck at "0" faults compared to widely used regularisers.

Deep generative models for symbolic music are typically designed to model temporal dependencies in music so as to predict the next musical event given previous events. In many cases, such models are expected to learn abstract concepts such as harmony, meter, and rhythm from raw musical data without any additional information. In this study, we investigate the effects of explicitly conditioning deep generative models with musically relevant information. Specifically, we study the effects of four different conditioning inputs on the performance of a recurrent monophonic melody generation model. Several combinations of these conditioning inputs are used to train different model variants which are then evaluated using three objective evaluation paradigms across two genres of music. The results indicate musically relevant conditioning significantly improves learning and performance, and reveal how this information affects learning of musical features related to pitch and rhythm. An informal subjective evaluation suggests a corresponding improvement in the aesthetic quality of generations.

Ever fantasied about having your own personal assistant to answer any questions you can ask, or have conversations with? Well, thanks to Machine Learning and Deep Neural Networks, this is not so far from happening. Think of the amazing capabilities exhibited by Apple's Siri or Amazon's Alexa. Don't get too excited, in this next series of posts we are not going to create an omnipotent Artificial Intelligence, rather we will create a simple chatbot that given some input information and a question about such information, responds to yes/no questions regarding what it has been told. It is nowhere near to Siri's or Alexa's capabilities, but it illustrates very well how even using very simple deep neural network structures, amazing results can be obtained.

As a human you instinctively know that a leopard is closer to a cat than a motorbike, but the way we train most AI makes them oblivious to these kinds of relations. Building the concept of similarity into our algorithms could make them far more capable, writes the author of a new paper in Science Robotics. Convolutional neural networks have revolutionized the field of computer vision to the point that machines are now outperforming humans on some of the most challenging visual tasks. But the way we train them to analyze images is very different from the way humans learn, says Atsuto Maki, an associate professor at KTH Royal Institute of Technology. "Imagine that you are two years old and being quizzed on what you see in a photo of a leopard," he writes.

According to McKinsey, AI will create an estimated $13 trillion of GDP growth between now and 2030. As a comparison, the GDP of the entire United States of America was around 19 trillion in 2017. Leading AI scientists, like Andrew Ng, describe AI as the fourth industrial revolution or „the new electricity". AI is undoubtedly a centerpiece of digital transformation and its application throughout the industry will dramatically change our world and how we do business. The problem is that many people want to participate in this AI-revolution but they are overwhelmed by its technological sophistication. They don't know what AI is capable of, let alone how they could use it for their company.

I want to take a minute to highlight one of the apps of Deep Learning Toolbox: Deep Network Designer. This app can be useful for more than just building a network from scratch, plus in 19a the app generates MATLAB code to programatically create networks! I want to walk through a few common uses for this app (and perhaps some not-so-common uses as well!) 1. Building a network from scratch Clearly this is one of the top benefits of Deep Network Designer, and there is a great introductory video on this topic. Click on the image to watch the short video. Link to full video 2. Importing a pretrained network and modifying it A second, popular use for the DND app is to import a pretrained network and modify it.

Are you sure you want to view these Tweets? Check out these new MATLAB features for deep learning! Check out this MATLAB visualization of fireworks happening across the United States today! See how to use MATLAB to simulate a robot model! We want to know...which topic are you most interested in?

Ever fantasied about having your own personal assistant to answer any questions you can ask, or have conversations with? Well, thanks to Machine Learning and Deep Neural Networks, this is not so far from happening. Think of the amazing capabilities exhibited by Apple's Siri or Amazon's Alexa. Don't get too excited, in this next series of posts we are not going to create an omnipotent Artificial Intelligence, rather we will create a simple chatbot that given some input information and a question about such information, responds to yes/no questions regarding what it has been told. It is nowhere near to Siri's or Alexa's capabilities, but it illustrates very well how even using very simple deep neural network structures, amazing results can be obtained.

Earlier this month, an unexceptional thread appeared on Reddit announcing that there is a new way "to cook egg white[s] without a frying pan". As so often happens on this website, which calls itself "the front page of the internet", this seemingly banal comment inspired a slew of responses. "I've never heard of people frying eggs without a frying pan," one incredulous Redditor replied. "I'm gonna try this," added another. One particularly enthusiastic commenter even offered to look up the scientific literature on the history of cooking egg whites without a frying pan. Every day, millions of these unremarkable conversations unfold on Reddit, spanning from cooking techniques to geopolitics in the Western Sahara to birds with arms.