Deep Learning
Applied Artificial Intelligence Conference 2019
Artificial intelligence, machine learning, deep learning and neural networks are the logical continuation of automation provided by higher computing power. Artificial intelligence has already made its entry into everyday life through enormous increases in efficiency, especially for businesses. In order to further raise the awareness of this important trend in the Austrian business landscape, ADVANTAGE AUSTRIA is organizing the "Applied Artificial Intelligence Conference" in Vienna - the largest event on Artificial Intelligence for businesses in the German language area. The aim of the event is to familiarize Austrian companies with the practical applications of AI in businesses across all sectors. In order to do so, developers and users of solutions are connected with potential users, creating a realistic professional exchange on how businesses can make AI work in their business.
NLP Overview: Modern Deep Learning Techniques Applied to Natural Language Processing
Over the recent number of years, neural networks have come to play an increasingly central role in natural language processing. Owing in large part to milestones such as word embeddings, and the explosion of chatbots powered by language models built, at least in part, by neural networks, the achievements of neural networks in the domain are come increasingly quickly. Trying to keep up with these advancements can be troublesome. That's where the today's spotlighted resource comes in. NLP Overview: Modern Deep Learning Techniques Applied to Natural Language Processing is a living resource maintained by Elvis Saravia and Soujanya Poria -- with a major part of the project having been directly borrowed from the work of Young et al. (2017), as per the resource maintainers.
A Simple Guide to Semantic Segmentation
Semantic Segmentation is the process of assigning a label to every pixel in the image. This is in stark contrast to classification, where a single label is assigned to the entire picture. Semantic segmentation treats multiple objects of the same class as a single entity. On the other hand, instance segmentation treats multiple objects of the same class as distinct individual objects (or instances). Typically, instance segmentation is harder than semantic segmentation.
Keras Tutorial: The Ultimate Beginner's Guide to Deep Learning in Python
In this step-by-step Keras tutorial, you'll learn how to build a convolutional neural network in Python! In fact, we'll be training a classifier for handwritten digits that boasts over 99% accuracy on the famous MNIST dataset. Before we begin, we should note that this guide is geared toward beginners who are interested in applied deep learning. Our goal is to introduce you to one of the most popular and powerful libraries for building neural networks in Python. That means we'll brush over much of the theory and math, but we'll also point you to great resources for learning those.
Distributed Artificial Intelligence: A primer on Multi-Agent Systems, Agent-Based Modeling, and Swarm Intelligence
Almost two years ago, I paused thinking about the future of AI and drew down some "predictions" about where I thought the field was going. One of those forecasts concerned reaching a general intelligence in several years, not through a super powerful 100-layers deep learning algorithm, but rather through something called collective intelligence. However, except for very obvious applications (e.g., drones), I have not read or seen any big development in the field and I thus thought to dig a bit into that to check what is currently going on. As part of the AI Knowledge Map then, I will have a look here not only at Swarm Intelligence (SI) but more generally at Distributed AI, which also includes Agent-Based Modeling (ABM) and Multi-Agent Systems (MAS). Let's start from the broader classification.
Physics-as-Inverse-Graphics: Joint Unsupervised Learning of Objects and Physics from Video
Jaques, Miguel, Burke, Michael, Hospedales, Timothy
We aim to perform unsupervised discovery of objects and their states such as location and velocity, as well as physical system parameters such as mass and gravity from video -- given only the differential equations governing the scene dynamics. Existing physical scene understanding methods require either object state supervision, or do not integrate with differentiable physics to learn interpretable system parameters and states. We address this problem through a $\textit{physics-as-inverse-graphics}$ approach that brings together vision-as-inverse-graphics and differentiable physics engines. This framework allows us to perform long term extrapolative video prediction, as well as vision-based model-predictive control. Our approach significantly outperforms related unsupervised methods in long-term future frame prediction of systems with interacting objects (such as ball-spring or 3-body gravitational systems). We further show the value of this tight vision-physics integration by demonstrating data-efficient learning of vision-actuated model-based control for a pendulum system. The controller's interpretability also provides unique capabilities in goal-driven control and physical reasoning for zero-data adaptation.
Machine Learning for Fluid Mechanics
Brunton, Steven, Noack, Bernd, Koumoutsakos, Petros
The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from experiments, field measurements, and large-scale simulations at multiple spatiotemporal scales. Machine learning presents us with a wealth of techniques to extract information from data that can be translated into knowledge about the underlying fluid mechanics. Moreover, machine learning algorithms can augment domain knowledge and automate tasks related to flow control and optimization. This article presents an overview of past history, current developments, and emerging opportunities of machine learning for fluid mechanics. We outline fundamental machine learning methodologies and discuss their uses for understanding, modeling, optimizing, and controlling fluid flows. The strengths and limitations of these methods are addressed from the perspective of scientific inquiry that links data with modeling, experiments, and simulations. Machine learning provides a powerful information processing framework that can augment, and possibly even transform, current lines of fluid mechanics research and industrial applications.
Combating Label Noise in Deep Learning Using Abstention
Thulasidasan, Sunil, Bhattacharya, Tanmoy, Bilmes, Jeff, Chennupati, Gopinath, Mohd-Yusof, Jamal
We introduce a novel method to combat label noise when training deep neural networks for classification. We propose a loss function that permits abstention during training thereby allowing the DNN to abstain on confusing samples while continuing to learn and improve classification performance on the non-abstained samples. We show how such a deep abstaining classifier (DAC) can be used for robust learning in the presence of different types of label noise. In the case of structured or systematic label noise -- where noisy training labels or confusing examples are correlated with underlying features of the data-- training with abstention enables representation learning for features that are associated with unreliable labels. In the case of unstructured (arbitrary) label noise, abstention during training enables the DAC to be used as an effective data cleaner by identifying samples that are likely to have label noise. We provide analytical results on the loss function behavior that enable dynamic adaption of abstention rates based on learning progress during training. We demonstrate the utility of the deep abstaining classifier for various image classification tasks under different types of label noise; in the case of arbitrary label noise, we show significant improvements over previously published results on multiple image benchmarks.
ODE$^2$VAE: Deep generative second order ODEs with Bayesian neural networks
Yıldız, Çağatay, Heinonen, Markus, Lähdesmäki, Harri
We present Ordinary Differential Equation Variational Auto-Encoder (ODE$^2$VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE$^2$VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex continuous-time latent dynamics. Our model explicitly decomposes the latent space into momentum and position components and solves a second order ODE system, which is in contrast to recurrent neural network (RNN) based time series models and recently proposed non-parametric ODE techniques. In order to account for uncertainty, we propose probabilistic latent ODE dynamics parameterized by deep Bayesian neural networks. We demonstrate our approach on motion capture, image rotation and bouncing balls datasets. We achieve state-of-the-art performance in long term motion prediction and imputation tasks.