Deep Learning
AWS Announces Five New Machine Learning Services and the World's First Deep Learning-Enabled Video Camera for Developers
Amazon SageMaker is a fully managed service for developers and data scientists to quickly build, train, deploy, and manage their own machine learning models. AWS also introduced AWS DeepLens, a deep learning-enabled wireless video camera that can run real-time computer vision models to give developers hands-on experience with machine learning. And, AWS announced four new application services that allow developers to build applications that emulate human-like cognition: Amazon Transcribe for converting speech to text; Amazon Translate for translating text between languages; Amazon Comprehend for understanding natural language; and, Amazon Rekognition Video, a new computer vision service for analyzing videos in batches and in real-time. Today, implementing machine learning is complex, involves a great deal of trial and error, and requires specialized skills. Developers and data scientists must first visualize, transform, and pre-process data to get it into a format that an algorithm can use to train a model.
How to stop fearing black box AI and love the robot-ruled future
A winding thread throughout the entire "AI is going to rise up and destroy all humans" narrative is the terrifying concept of deep learning occurring in a black box. How do you provide oversight for a system you can't understand? Just like any other tool, however, it's not whether black box AI represents a danger or not, it's how we choose to use it. The headlines declare this kind of AI untrustworthy, and tell us that experts seek to end its use in government. But what is black box AI?
What's the Difference Between Artificial Intelligence (AI), Machine Learning, and Deep Learning?
It seems as if these three terms are appearing everywhere these days, from news stories to press releases and even TV shows and advertising. For many of us, when we hear artificial intelligence (or AI), we think of either Commander Data from Star Trek (the utopian version) or Cylons from Battlestar Galactica (the dystopian version). But what does AI really mean? And how is it different from machine learning (ML) and deep learning (DL)? Here at Prowess, we've been taking on more and more projects that touch on these technologies. In this post, I'll pass on some of what we've learned about what the terms mean, how the technologies are changing our lives, and what hardware and software developments are enabling AI technologies to take off.
Deep Echo State Network (DeepESN): A Brief Survey
Gallicchio, Claudio, Micheli, Alessio
The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced deep Echo State Network (deepESN) model opened the way to an extremely efficient approach for designing deep neural networks for temporal data. At the same time, the study of deepESNs allowed to shed light on the intrinsic properties of state dynamics developed by hierarchical compositions of recurrent layers, i.e. on the bias of depth in RNNs architectural design. In this paper, we summarize the advancements in the development, analysis and applications of deepESNs.
Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning
He, Zhen, Gao, Shaobing, Xiao, Liang, Liu, Daxue, He, Hangen, Barber, David
Long Short-Term Memory (LSTM) is a popular approach to boosting the ability of Recurrent Neural Networks to store longer term temporal information. The capacity of an LSTM network can be increased by widening and adding layers. However, usually the former introduces additional parameters, while the latter increases the runtime. As an alternative we propose the Tensorized LSTM in which the hidden states are represented by tensors and updated via a cross-layer convolution. By increasing the tensor size, the network can be widened efficiently without additional parameters since the parameters are shared across different locations in the tensor; by delaying the output, the network can be deepened implicitly with little additional runtime since deep computations for each timestep are merged into temporal computations of the sequence. Experiments conducted on five challenging sequence learning tasks show the potential of the proposed model.
Multi-Speaker Localization Using Convolutional Neural Network Trained with Noise
Chakrabarty, Soumitro, Habets, Emanuël A. P.
The problem of multi-speaker localization is formulated as a multi-class multi-label classification problem, which is solved using a convolutional neural network (CNN) based source localization method. Utilizing the common assumption of disjoint speaker activities, we propose a novel method to train the CNN using synthesized noise signals. The proposed localization method is evaluated for two speakers and compared to a well-known steered response power method.
Concept Formation and Dynamics of Repeated Inference in Deep Generative Models
Nagano, Yoshihiro, Karakida, Ryo, Okada, Masato
Deep generative models are reported to be useful in broad applications including image generation. Repeated inference between data space and latent space in these models can denoise cluttered images and improve the quality of inferred results. However, previous studies only qualitatively evaluated image outputs in data space, and the mechanism behind the inference has not been investigated. The purpose of the current study is to numerically analyze changes in activity patterns of neurons in the latent space of a deep generative model called a "variational auto-encoder" (VAE). What kinds of inference dynamics the VAE demonstrates when noise is added to the input data are identified. The VAE embeds a dataset with clear cluster structures in the latent space and the center of each cluster of multiple correlated data points (memories) is referred as the concept. Our study demonstrated that transient dynamics of inference first approaches a concept, and then moves close to a memory. Moreover, the VAE revealed that the inference dynamics approaches a more abstract concept to the extent that the uncertainty of input data increases due to noise. It was demonstrated that by increasing the number of the latent variables, the trend of the inference dynamics to approach a concept can be enhanced, and the generalization ability of the VAE can be improved.
Temporal Stability in Predictive Process Monitoring
Teinemaa, Irene, Dumas, Marlon, Leontjeva, Anna, Maggi, Fabrizio Maria
Noname manuscript No. (will be inserted by the editor) Abstract Predictive business process monitoring is concerned with the analysis of events produced during the execution of a business process in order to predict as early as possible the final outcome of an ongoing case. Traditionally, predictive process monitoring methods are optimized with respect to accuracy. However, in environments where users make decisions and take actions in response to the predictions they receive, it is equally important to optimize the stability of the successive predictions made for each case. To this end, this paper defines a notion of temporal stability for predictive process monitoring and evaluates existing methods with respect to both temporal stability and accuracy. We find that methods based on XGBoost and LSTM neural networks exhibit the highest temporal stability. We then show that temporal stability can be enhanced by hyperparameter-optimizing random forests and XGBoost classifiers with respect to inter-run stability. Finally, we show that time series smoothing techniques can further enhance temporal stability at the expense of slightly lower accuracy. Keywords Predictive Monitoring · Early Sequence Classification · Stability 1 Introduction Modern organizations generally execute their business processes on top of processaware information systems, such as Enterprise Resource Planning (ERP) systems, Customer Relationship Management (CRM) systems, and Business Process Management Systems (BPMS), among others [8]. These systems record a range of events that occur during the execution of the processes they support, including events signaling the creation and completion of business process instances (herein called cases) and the start and completion of activities within each case. Event records produced by process-aware information systems can be extracted and pre-processed to produce business process event logs [1]. A business process event log consists of a set of traces, each trace consisting of the sequence of event records produced by one case. Each event record has a number of attributes. Three of these attributes are present in every event record, namely the event class (a.k.a. In other words, every event represents the occurrence of an activity at a particular point in time and in the context of a given case.
Transportation analysis of denoising autoencoders: a novel method for analyzing deep neural networks
The feature map obtained from the denoising autoencoder (DAE) is investigated by determining transportation dynamics of the DAE, which is a cornerstone for deep learning. Despite the rapid development in its application, deep neural networks remain analytically unexplained, because the feature maps are nested and parameters are not faithful. In this paper, we address the problem of the formulation of nested complex of parameters by regarding the feature map as a transport map. Even when a feature map has different dimensions between input and output, we can regard it as a transportation map by considering that both the input and output spaces are embedded in a common high-dimensional space. In addition, the trajectory is a geometric object and thus, is independent of parameterization. In this manner, transportation can be regarded as a universal character of deep neural networks. By determining and analyzing the transportation dynamics, we can understand the behavior of a deep neural network. In this paper, we investigate a fundamental case of deep neural networks: the DAE. We derive the transport map of the DAE, and reveal that the infinitely deep DAE transports mass to decrease a certain quantity, such as entropy, of the data distribution. These results though analytically simple, shed light on the correspondence between deep neural networks and the Wasserstein gradient flows.