The Women in AI Podcast - Women at the forefront of AI discuss their work and diversity issues faced in STEM - Listen here. The DeepMind Podcast - A new series that we hope will answer the difficult questions in AI - Listen here. Lex Fridman's AI Podcast - a series of conversations about technology, science, and the human condition hosted by MIT's Lex Fridman - Listen here. The Eye on AI - Justin Gottschlich explains his group's efforts to automate software development - Listen along here. The NVIDIA AI Podcast - NVIDIA release new episodes every other week with guest speakers at the forefront of AI - Listen along here. Artificially Intelligent - Weekly discussions on the impacts of AI - Listen here. Underrated ML - Regular RE•WORK speaker, Sara Hooker & her brother, Sean Hooker have started their new podcast based on underrated ML papers - Listen here. Concerning AI - A series on AI hosted by Ted Sarvata & Brandon Sanders - Listen here.

Everything we know about memory requirements in machine learning may be wrong. Today, when data scientists process deep learning models using a "throughput computing" device like a GPU, TPU, or similar hardware accelerator, they're likely faced with a decision to shrink their model or input size to fit within the device's memory limitations. Training a large, deep neural network (or even a wide, shallow one) on a single GPU, in many cases, may be impossible. Ever wonder why on the original Resnet 152, the winner of the ILSVRC-2015 image detection competition had 152 layers and not 153? Is it a coincidence that the parameters in 152 layers have a memory footprint of slightly less than 12G, while 153 layers go beyond 12G (the standard size of GPU memory at the time)?

It took me a long time to realise that search is the biggest problem in NLP. Just look at Google, Amazon and Bing. These are multi-billion dollar businesses possible only due to their powerful search engines. My initial thoughts on search were centered around unsupervised ML, but I participated in Microsoft Hackathon 2018 for Bing and came to know the various ways a search engine can be made with deep learning. Do you find this in-depth technical education about NLP applications to be useful?

Speech recognition startup Deepgram has secured $12 million in Series A funding led by Wing VC, writes TechCrunch. Deepgram leverages deep learning and has already raised a few million in capital in its five-year existence. Other investors that joined the funding round include Nvidia, Y Combinator and SAP. The startup wants to use the investment to create new job opportunities in go-to-market and engineering, and expand its team from its current 40 members to an undisclosed number. It also wants to purchase some new hardware, as it runs its own service for better margins, which also makes it a natural partner for Nvidia.

The book AI Crash Course by Hadelin de Ponteves contains a toolkit of four different AI models: Thompson Sampling, Q-Learning, Deep Q-Learning and Deep Convolutional Q-learning. It teaches the theory of these AI models and provides coding examples for solving industry cases based on these models. InfoQ readers can find an excerpt of AI Crash Course on the publisher's website. InfoQ interviewed Hadelin de Ponteves about using different AI models and how to develop AI skills. InfoQ: Why did you write this book?

Fortinet, a global integrated and automated cybersecurity solutions, today announced FortiAI, an appliance that leverages self-learning Deep Neural Networks (DNN) to speed threat remediation and handle time-consuming, manual security analyst tasks. FortiAI's Virtual Security AnalystÔ embeds a mature cybersecurity artificial intelligence, developed by Fortinet's FortiGuard Labs, directly into an organization's network to deliver sub-second detection of advanced threats. To address the challenges faced by security professionals today, Fortinet is unveiling FortiAI Virtual Security AnalystÔ to accelerate threat remediation. FortiAI handles many of the time consuming, manual tasks currently expected of security professionals, preserving their time for higher-value security functions. FortiAI's self-learning capabilities continue to get smarter once deployed in an organization's network.

Leveraging the advances in deep generative models, ODE2VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex continuous-time latent dynamics. Our model explicitly decomposes the latent space into momentum and position components and solves a second order ODE system, which is in contrast to recurrent neural network (RNN) based time series models and recently proposed black-box ODE techniques. In order to account for uncertainty, we propose probabilistic latent ODE dynamics parameterized by deep Bayesian neural networks. We demonstrate our approach on motion capture, image rotation, and bouncing balls datasets. We achieve state-of-the-art performance in long term motion prediction and imputation tasks.

Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive uncertainty. Quantifying uncertainty is especially critical in real-world settings, which often involve input distributions that are shifted from the training distribution due to a variety of factors including sample bias and non-stationarity. In such settings, well calibrated uncertainty estimates convey information about when a model's output should (or should not) be trusted. Many probabilistic deep learning methods, including Bayesian-and non-Bayesian methods, have been proposed in the literature for quantifying predictive uncertainty, but to our knowledge there has not previously been a rigorous large-scale empirical comparison of these methods under dataset shift. We present a large-scale benchmark of existing state-of-the-art methods on classification problems and investigate the effect of dataset shift on accuracy and calibration.

Deep networks are commonly used to model dynamical systems, predicting how the state of a system will evolve over time (either autonomously or in response to control inputs). Despite the predictive power of these systems, it has been difficult to make formal claims about the basic properties of the learned systems. In this paper, we propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space. The approach works by jointly learning a dynamics model and Lyapunov function that guarantees non-expansiveness of the dynamics under the learned Lyapunov function. We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics, such as video textures, in a fully end-to-end fashion.

Deep neural networks (DNNs) excel at learning representations when trained on large-scale datasets. Pre-trained DNNs also show strong transferability when fine-tuned to other labeled datasets. However, such transferability becomes weak when the target dataset is fully unlabeled as in Unsupervised Domain Adaptation (UDA). We envision that the loss of transferability may stem from the intrinsic limitation of the architecture design of DNNs. In this paper, we delve into the components of DNN architectures and propose Transferable Normalization (TransNorm) in place of existing normalization techniques.