Due to the success of deep learning to solving a variety of challenging machine learning tasks, there is a rising interest in understanding loss functions for training neural networks from a theoretical aspect. Particularly, the properties of critical points and the landscape around them are of importance to determine the convergence performance of optimization algorithms. In this paper, we provide full (necessary and sufficient) characterization of the analytical forms for the critical points (as well as global minimizers) of the square loss functions for various neural networks. We show that the analytical forms of the critical points characterize the values of the corresponding loss functions as well as the necessary and sufficient conditions to achieve global minimum. Furthermore, we exploit the analytical forms of the critical points to characterize the landscape properties for the loss functions of these neural networks. One particular conclusion is that: The loss function of linear networks has no spurious local minimum, while the loss function of one-hidden-layer nonlinear networks with ReLU activation function does have local minimum that is not global minimum.

ABSTRACT We consider the problem of transcribing polyphonic piano music with an emphasis on generalizing to unseen instruments. We use deep neural networks and propose a novel approach that predicts onsets and frames using both CNNs and LSTMs. This model predicts pitch onset events and then uses those predictions to condition framewise pitch predictions. During inference, we restrict the predictions from the framewise detector by not allowing a new note to start unless the onset detector also agrees that an onset for that pitch is present in the frame. We focus on improving onsets and offsets together instead of either in isolation as we believe it correlates better with human musical perception. This technique results in over a 100% relative improvement in note with offset score on the MAPS dataset.

Generative Adversarial Networks (GANs) have become a popular method to learn a probability model from data. Many GAN architectures with different optimization metrics have been introduced recently. Instead of proposing yet another architecture, this paper aims to provide an understanding of some of the basic issues surrounding GANs. First, we propose a natural way of specifying the loss function for GANs by drawing a connection with supervised learning. Second, we shed light on the generalization peformance of GANs through the analysis of a simple LQG setting: the generator is Linear, the loss function is Quadratic and the data is drawn from a Gaussian distribution. We show that in this setting: 1) the optimal GAN solution converges to population Principal Component Analysis (PCA) as the number of training samples increases; 2) the number of samples required scales exponentially with the dimension of the data; 3) the number of samples scales almost linearly if the discriminator is constrained to be quadratic. Thus, linear generators and quadratic discriminators provide a good balance for fast learning.

Recurrent neural networks, and in particular long short-term memory (LSTM) networks, are a remarkably effective tool for sequence modeling that learn a dense black-box hidden representation of their sequential input. Researchers interested in better understanding these models have studied the changes in hidden state representations over time and noticed some interpretable patterns but also significant noise. In this work, we present LSTMVIS, a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics. The tool allows users to select a hypothesis input range to focus on local state changes, to match these states changes to similar patterns in a large data set, and to align these results with structural annotations from their domain. We show several use cases of the tool for analyzing specific hidden state properties on dataset containing nesting, phrase structure, and chord progressions, and demonstrate how the tool can be used to isolate patterns for further statistical analysis. We characterize the domain, the different stakeholders, and their goals and tasks.

There are a number of machine learning (ML) architectures that utilize deep neural networks (DNNs), including AlexNet, VGGNet, GoogLeNet, Inception, ResNet, FCN, and U-Net. These in turn run on frameworks like Berkeley's Caffe, Google's TensorFlow, Torch, Microsoft's Cognitive Toolkit (CNTK), and Apache's mxnet. Of course, support for these frameworks on specific hardware is required to actually run the ML applications. Each framework has advantages and disadvantages. For example, Caffe is an easy platform to start with, especially since ones of its popular uses is image recognition.

The consternation of radiologists about the impact of artificial intelligence is real--but so are the benefits of machine learning. Recent research showed that deep learning with a convolutional neural network (CNN) was successful in differentiating liver masses in CT. The retrospective study, published online Oct. 23 in Radiology, examined the diagnostic abilities of a deep learning method with a CNN. Researchers tested the CNN with 100 liver mass image sets from 2016, including 74 men and 26 women with the average age of 66 years old. "This preliminary study, which used 55, 536 image sets (1068 image sets augmented by a factor of 52) to obtain models, indicated that classifying liver masses into five categories can be accomplished with a high degree of accuracy by using a deep learning method with a CNN on dynamic contrast-enhanced CT images," wrote Koichiro Yasaka, MD, PhD, with the department of radiology at the University of Tokyo Hospital in Japan, and colleagues.

The consternation of radiologists about the impact of artificial intelligence is real--but so are the benefits of machine learning. Recent research showed that deep learning with a convolutional neural network (CNN) was successful in differentiating liver masses in CT. The retrospective study, published online Oct. 23 in Radiology, examined the diagnostic abilities of a deep learning method with a CNN. Researchers tested the CNN with 100 liver mass image sets from 2016, including 74 men and 26 women with the average age of 66 years old. "This preliminary study, which used 55, 536 image sets (1068 image sets augmented by a factor of 52) to obtain models, indicated that classifying liver masses into five categories can be accomplished with a high degree of accuracy by using a deep learning method with a CNN on dynamic contrast-enhanced CT images," wrote Koichiro Yasaka, MD, PhD, with the department of radiology at the University of Tokyo Hospital in Japan, and colleagues.

We have lot of amazing frameworks for deep learning which allow us easy and fast prototyping and learning complex architectures even not thinking about what happening inside of them. But sometimes you need to deploy your model somewhere… let's say where you can't use your favorite I recently faced this problem, when I had to deploy recurrent neural network for action recognition trained in Keras in Java. My client doesn't want to use some microservices architecture, he wants everything in Java and basta cosi:) So, let's see how we can do it. Embedding is vector length of 11, hidden units 15. First, let's load our weights from .hdf5

What to expect • Quick intro • 3 demo applications • Polly • Rekognition • MXnet • Wrap up. 4. 2017, Amazon Web Services, Inc. or its Affiliates. Event driven A B CEvent on B by A triggers C Invocation Lambda functions Action 6. 2017, Amazon Web Services, Inc. or its Affiliates. How Lambda works S3 event notifications DynamoDB Streams Kinesis events Cognito events SNS events Custom events CloudTrail events LambdaDynamoDB Kinesis S3 Any custom Invoked in response to events - Changes in data - Changes in state Redshift SNS Access any service, including your own Such as… Lambda functions CloudWatch events 7. 2017, Amazon Web Services, Inc. or its Affiliates. No servers to provision or manage Scales with usage Never pay for idle Availability and fault tolerance built in Serverless means… 9. 2017, Amazon Web Services, Inc. or its Affiliates. EVENT DRIVEN CONTINUOUS SCALING PAY BY USAGE Serverless means… 10. 2017, Amazon Web Services, Inc. or its Affiliates.

Deep learning currently carries the torch for artificial intelligence, providing us with a glimpse of how powerfully intelligent machines may do our bidding in the future. But there's a big problem with deep learning: Nobody really knows how it works. That's not to say that it's a complete mystery. The machine learning algorithms at the heart of today's neural networks are decades old and are well-defined and extensively documented in academia. These algorithms have been employed in regulated industries like banking and insurance for years without causing a major stir.