Deep Learning
Using Deep Learning to Automate Feature Modeling in Learning by Observation
Floyd, Michael W. (Knexus Research) | Turner, J. T. (Knexus Research) | Aha, David W. (Naval Research Laboratory)
Learning by observation allows non-technical experts to transfer their skills to an agent by shifting the knowledge-transfer task to the agent. However, for the agent to learn regardless of expert, domain, or observed behavior, it must learn in a general-purpose manner. Existing learning by observation agents allow for domain-independent learning and reasoning but require human intervention to model the agentโs inputs and outputs. We describe Domain-Independent Deep Feature Learning by Observation (DIDFLO), an agent that uses convolutional neural networks to learn without explicitly defining input features. DIDFLO uses the raw visual inputs at two levels of granularity to automatically learn input features using limited training data. We evaluate DIDFLO in scenarios drawn from a simulated soccer domain and provide a comparison to other learning by observation agents in this domain.
Predictive Business Process Monitoring with LSTM Neural Networks
Tax, Niek, Verenich, Ilya, La Rosa, Marcello, Dumas, Marlon
Predictive business process monitoring methods exploit logs of completed cases of a process in order to make predictions about running cases thereof. Existing methods in this space are tailor-made for specific prediction tasks. Moreover, their relative accuracy is highly sensitive to the dataset at hand, thus requiring users to engage in trial-and-error and tuning when applying them in a specific setting. This paper investigates Long Short-Term Memory (LSTM) neural networks as an approach to build consistently accurate models for a wide range of predictive process monitoring tasks. First, we show that LSTMs outperform existing techniques to predict the next event of a running case and its timestamp. Next, we show how to use models for predicting the next task in order to predict the full continuation of a running case. Finally, we apply the same approach to predict the remaining time, and show that this approach outperforms existing tailor-made methods.
Deep Learning and the Artificial Intelligence Revolution: Part 3
If you want to get started right now, download the complete Deep Learning and Artificial Intelligence white paper. Deep learning is a subset of machine learning that has attracted worldwide attention for its recent success solving particularly hard and large-scale problems in areas such as speech recognition, natural language processing, and image classification. Deep learning is a refinement of ANNs, which, as discussed earlier, "loosely" emulate how the human brain learns and solves problems. Before diving into how deep learning works, it's important to first understand how ANNs work. ANNs are made up of an interconnected group of neurons, similar to the network of neurons in the brain.
Deep Learning in Production for Predicting Consumer Behavior โ Zalando Tech Blog
At Zalando adtech lab in Hamburg, machine learning drives many of our production systems to build great user experiences. Our most recent product requires precise estimates of future interests of Zalando consumers based on their history of interacting with the fashion platform. For example, we want to predict a consumer's interest in ordering selected fashion articles. We set ourselves the goal to build a powerful and versatile prediction tool that not only fits the task at hand, but is also ready for future product developments. Deep learning approaches have many advantages over traditional techniques, making them a great fit for our requirements.
AI-equipped Apple Watch Can Detect Irregular Heartbeat and Signs of a Stroke
Apple has been pitching the Watch with ResearchKit to doctors and scientists as a serious health tool. Cardiogram initiated the research last year to figure out whether it could detect the signs of a stroke, a quarter of which are caused by irregular heartbeats. The study drew 6,158 Apple Watch users via the Cardiogram app -- most had normal EKG readings, but 200 had an existing AF condition that made their hearts beat erratically. Engineers used those subjects to train a deep learning system to discern patients with arrhythmia vs. those with normal heartbeats. They then tested the system on 51 patients scheduled for a procedure to restore normal heart rhythms.
NVIDIA and SAP Partner to Create a New Wave of AI Business Applications - The Official NVIDIA Blog
Businesses collect mountains of data daily. Now it's time to make those mountains move. NVIDIA CEO and founder Jensen Huang announced today at our GPU Technology Conference that SAP and NVIDIA are working together to help businesses use AI in ways that will change the world's view of business applications. "With strong partners like NVIDIA at our side, the possibilities are limitless," wrote SAP Chief Innovation Officer Juergen Mueller in a blog post published today. "New applications, unprecedented value in existing applications, and easy access to machine learning services will allow you to make your own enterprise intelligent."
10 things in tech you need to know today
Here is the tech news you need to know this Monday. Europol's executive director said there were at least 200,000 victims across 150 countries so far, and that number will go up on when people go back to work. The deal closed a few weeks ago and roughly 20 engineers from Lattice have joined Apple, according to the report, which cited an anonymous source. Waymo, which spun out of Google's self-driving car division, has nearly a decade of experience of working in the space. The Information Commissioner's Office (ICO) has been looking into why the Royal Free gave DeepMind access to so much patient data for its kidney monitoring app.
ShortFuse: Biomedical Time Series Representations in the Presence of Structured Information
Fiterau, Madalina, Bhooshan, Suvrat, Fries, Jason, Bournhonesque, Charles, Hicks, Jennifer, Halilaj, Eni, Rรฉ, Christopher, Delp, Scott
In healthcare applications, temporal variables that encode movement, health status and longitudinal patient evolution are often accompanied by rich structured information such as demographics, diagnostics and medical exam data. However, current methods do not jointly optimize over structured covariates and time series in the feature extraction process. We present ShortFuse, a method that boosts the accuracy of deep learning models for time series by explicitly modeling temporal interactions and dependencies with structured covariates. ShortFuse introduces hybrid convolutional and LSTM cells that incorporate the covariates via weights that are shared across the temporal domain. ShortFuse outperforms competing models by 3% on two biomedical applications, forecasting osteoarthritis-related cartilage degeneration and predicting surgical outcomes for cerebral palsy patients, matching or exceeding the accuracy of models that use features engineered by domain experts.
Curiosity-driven Exploration by Self-supervised Prediction
Pathak, Deepak, Agrawal, Pulkit, Efros, Alexei A., Darrell, Trevor
In many real-world scenarios, rewards extrinsic to the agent are extremely sparse, or absent altogether. In such cases, curiosity can serve as an intrinsic reward signal to enable the agent to explore its environment and learn skills that might be useful later in its life. We formulate curiosity as the error in an agent's ability to predict the consequence of its own actions in a visual feature space learned by a self-supervised inverse dynamics model. Our formulation scales to high-dimensional continuous state spaces like images, bypasses the difficulties of directly predicting pixels, and, critically, ignores the aspects of the environment that cannot affect the agent. The proposed approach is evaluated in two environments: VizDoom and Super Mario Bros. Three broad settings are investigated: 1) sparse extrinsic reward, where curiosity allows for far fewer interactions with the environment to reach the goal; 2) exploration with no extrinsic reward, where curiosity pushes the agent to explore more efficiently; and 3) generalization to unseen scenarios (e.g.
Extending Defensive Distillation
Papernot, Nicolas, McDaniel, Patrick
Deployed machine learning (ML) models are vulnerable to inputs maliciously perturbed to force them to mispredict [1, 2]. A class of such inputs, named adversarial examples, are systematically constructed through slight perturbations of otherwise correctly classified inputs [3, 4]. These perturbations are chosen to maximize the model's prediction error while leaving the semantics of the input unchanged. Although this often poses a non-tractable optimization problem for popular architectures like deep neural networks, heuristics allow the adversary to find effective perturbations--typically through the evaluation of gradients of the model's output with respect to its inputs [3, 5]. To defend against adversarial examples, two classes of approaches exist.