My last "AI 101" post covered the difference between artificial intelligence, machine learning, and deep learning. In this post, I'll cover deep learning training and inference -- two key processes associated with developing and using AI. In the last post, I explained that deep learning (DL) is a special type of machine learning that involves a deep neural network (DNN) composed of many layers of interconnected artificial neurons. Training is the process of "teaching" a DNN to perform a desired AI task (such as image classification or converting speech into text) by feeding it data, resulting in a trained deep learning model. During the training process, known data is fed to the DNN, and the DNN makes a prediction about what the data represents.

Free Coupon Discount - Theano / Tensorflow: Autoencoders, Restricted Boltzmann Machines, Deep Neural Networks, t-SNE and PCA Created by Lazy Programmer Inc. Students also bought Artificial Intelligence: Reinforcement Learning in Python Advanced AI: Deep Reinforcement Learning in Python Machine Learning A-Z: Hands-On Python & R In Data Science Learn Python Programming Masterclass Complete Python Developer in 2020: Zero to Mastery Preview this Udemy Course GET COUPON CODE Description This course is the next logical step in my deep learning, data science, and machine learning series. I've done a lot of courses about deep learning, and I just released a course about unsupervised learning, where I talked about clustering and density estimation. So what do you get when you put these 2 together? In these course we'll start with some very basic stuff - principal components analysis (PCA), and a popular nonlinear dimensionality reduction technique known as t-SNE (t-distributed stochastic neighbor embedding). Next, we'll look at a special type of unsupervised neural network called the autoencoder.

In our deep learning model, depending on whether we are doing classification or regression tasks, we have to define a loss function that compares the predicted result vs the expected result (labeled data). In classification problems, the loss function calculates the loss for misclassifications. Cross-Entropy Loss measures the performance of classification. It is a combination of the Negative Log-Likelihood Loss (NLL) and Softmax. Some of the most used loss functions for regression tasks are L1 loss, MSE (L2 loss), and Smooth L1 Loss.

Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context, with a training objective that makes them approximately sample in proportion to a given reward function. In this paper, we show a number of additional theoretical properties of GFlowNets. They can be used to estimate joint probability distributions and the corresponding marginal distributions where some variables are unspecified and, of particular interest, can represent distributions over composite objects like sets and graphs. GFlowNets amortize the work typically done by computationally expensive MCMC methods in a single but trained generative pass. They could also be used to estimate partition functions and free energies, conditional probabilities of supersets (supergraphs) given a subset (subgraph), as well as marginal distributions over all supersets (supergraphs) of a given set (graph). We introduce variations enabling the estimation of entropy and mutual information, sampling from a Pareto frontier, connections to reward-maximizing policies, and extensions to stochastic environments, continuous actions and modular energy functions.

This work investigates the use of a Deep Neural Network (DNN) to perform an estimation of the Weapon Engagement Zone (WEZ) maximum launch range. The WEZ allows the pilot to identify an airspace in which the available missile has a more significant probability of successfully engaging a particular target, i.e., a hypothetical area surrounding an aircraft in which an adversary is vulnerable to a shot. We propose an approach to determine the WEZ of a given missile using 50,000 simulated launches in variate conditions. These simulations are used to train a DNN that can predict the WEZ when the aircraft finds itself on different firing conditions, with a coefficient of determination of 0.99. It provides another procedure concerning preceding research since it employs a non-discretized model, i.e., it considers all directions of the WEZ at once, which has not been done previously. Additionally, the proposed method uses an experimental design that allows for fewer simulation runs, providing faster model training.

Taxi-out time is an indicator of departure efficiency and is often the early signal of large holding and diversion events for airports that have constrained surface space. This is one of the real-time performance metrics that is of great interest to air traffic managers and flight dispatchers. For a busy airport that has limited tarmac space like LaGuardia Airport (LGA), an increasing average taxi-out time under a deteriorating visibility condition could soon lead to surface gridlock that would cause significant delays to both arrivals and departures. Thus, research is needed to develop an early alert or prediction of long taxi-out times to enable early delay mitigation actions. The problem of predicting taxi-out times has received considerable treatment in the aviation literature. Most research exploring the domain of taxi time prediction has focused on predicting taxi-out times for individual aircraft. Inaccurate taxi-out times can lead to a variety of National Airspace System (NAS) inefficiencies, such as a reduction in predictability for downstream Traffic Flow Management (TFM) applications and excess fuel consumption after push back from the gate. By better predicting aircraft-specific taxi-out times, informed updates can be made to the flight schedule to improve predictability and more efficiently use available NAS resources (e.g., capacity). Although our focus is on predicting average taxi-out time, it's worth reviewing the literature on aircraft-specific taxi-out time predictions for historical context.

Federated learning allows clients to collaboratively learn statistical models while keeping their data local. Federated learning was originally used to train a unique global model to be served to all clients, but this approach might be sub-optimal when clients' local data distributions are heterogeneous. In order to tackle this limitation, recent personalized federated learning methods train a separate model for each client while still leveraging the knowledge available at other clients. In this work, we exploit the ability of deep neural networks to extract high quality vectorial representations (embeddings) from non-tabular data, e.g., images and text, to propose a personalization mechanism based on local memorization. Personalization is obtained interpolating a pre-trained global model with a $k$-nearest neighbors (kNN) model based on the shared representation provided by the global model. We provide generalization bounds for the proposed approach and we show on a suite of federated datasets that this approach achieves significantly higher accuracy and fairness than state-of-the-art methods.

We present MEDCOD, a Medically-Accurate, Emotive, Diverse, and Controllable Dialog system with a unique approach to the natural language generator module. MEDCOD has been developed and evaluated specifically for the history taking task. It integrates the advantage of a traditional modular approach to incorporate (medical) domain knowledge with modern deep learning techniques to generate flexible, human-like natural language expressions. Two key aspects of MEDCOD's natural language output are described in detail. First, the generated sentences are emotive and empathetic, similar to how a doctor would communicate to the patient. Second, the generated sentence structures and phrasings are varied and diverse while maintaining medical consistency with the desired medical concept (provided by the dialogue manager module of MEDCOD). Experimental results demonstrate the effectiveness of our approach in creating a human-like medical dialogue system. Relevant code is available at https://github.com/curai/curai-research/tree/main/MEDCOD

The visual world around us can be described as a structured set of objects and their associated relations. An image of a room may be conjured given only the description of the underlying objects and their associated relations. While there has been significant work on designing deep neural networks which may compose individual objects together, less work has been done on composing the individual relations between objects. A principal difficulty is that while the placement of objects is mutually independent, their relations are entangled and dependent on each other. To circumvent this issue, existing works primarily compose relations by utilizing a holistic encoder, in the form of text or graphs. In this work, we instead propose to represent each relation as an unnormalized density (an energy-based model), enabling us to compose separate relations in a factorized manner. We show that such a factorized decomposition allows the model to both generate and edit scenes that have multiple sets of relations more faithfully. We further show that decomposition enables our model to effectively understand the underlying relational scene structure.

Randomized smoothing is currently a state-of-the-art method to construct a certifiably robust classifier from neural networks against $\ell_2$-adversarial perturbations. Under the paradigm, the robustness of a classifier is aligned with the prediction confidence, i.e., the higher confidence from a smoothed classifier implies the better robustness. This motivates us to rethink the fundamental trade-off between accuracy and robustness in terms of calibrating confidences of a smoothed classifier. In this paper, we propose a simple training scheme, coined SmoothMix, to control the robustness of smoothed classifiers via self-mixup: it trains on convex combinations of samples along the direction of adversarial perturbation for each input. The proposed procedure effectively identifies over-confident, near off-class samples as a cause of limited robustness in case of smoothed classifiers, and offers an intuitive way to adaptively set a new decision boundary between these samples for better robustness. Our experimental results demonstrate that the proposed method can significantly improve the certified $\ell_2$-robustness of smoothed classifiers compared to existing state-of-the-art robust training methods.