We introduce Gluon Time Series (GluonTS, available at https://gluon-ts.mxnet.io), a library for deep-learning-based time series modeling. GluonTS simplifies the development of and experimentation with time series models for common tasks such as forecasting or anomaly detection. It provides all necessary components and tools that scientists need for quickly building new models, for efficiently running and analyzing experiments and for evaluating model accuracy.

In human perception and cognition, the fundamental operation that brains perform is interpretation: constructing coherent neural states from noisy, incomplete, and intrinsically ambiguous evidence. The problem of interpretation is well matched to an early and often overlooked architecture, the attractor network---a recurrent neural network that performs constraint satisfaction, imputation of missing features, and clean up of noisy data via energy minimization dynamics. We revisit attractor nets in light of modern deep learning methods, and propose a convolutional bipartite architecture with a novel training loss, activation function, and connectivity constraints. We tackle problems much larger than have been previously explored with attractor nets and demonstrate their potential for image denoising, completion, and super-resolution. We argue that this architecture is better motivated than ever-deeper feedforward models and is a viable alternative to more costly sampling-based methods on a range of supervised and unsupervised tasks.

Variational autoencoders (VAEs) have become one of the most popular deep learning approaches to unsupervised learning and data generation. However, traditional VAEs suffer from the constraint that the latent space must distributionally match a simple prior (e.g. normal, uniform), independent of the initial data distribution. This leads to a number of issues around modeling manifold data, as there is no function with a bounded Jacobian that maps a normal distribution to certain manifolds (e.g. a hypersphere). Similarly, there are not many theoretical guarantees on the encoder and decoder created by the VAE. In this work, we propose a variational autoencoder that maps manifold valued data to its diffusion map coordinates in the latent space, resamples in a neighborhood around a given point in the latent space, and learns a decoder that maps the newly resampled points back to the manifold. The framework is built off of SpectralNet [Shaham et al., 2018a] and is capable of learning this data dependent latent space without computing the eigenfunction of the Laplacian explicitly. We prove that our method is capable of learning a locally bi-Lipschitz map between the manifold and the latent space, and that our resampling method around a point in the latent space $\psi(x)$ maps points back to the manifold around the point $x$, specifically into a neighborbood on the tangent space at the point $x$ on the manifold. We also provide empirical evidence of the benefits of using a diffusion map latent space on manifold data.

According to the Software and Information Industry Association (SIIA), the CODiE Awards have honored thousands of software, education, information and media products for achieving excellence and innovation in technology for over 30 years. "They are the only peer-recognized program in the business and education technology industries, so each CODiE Award win serves as incredible market validation for a product's innovation, vision, and overall industry impact. The CODiE Awards highlights the very best products, innovators and leaders in today's tech market." Separately, the Adobe XD Team was also honored in the "Product Team of the Year" category, as well as Adobe Captivate Prime for "Best Corporate / Enterprise Learning Solution." Adobe Sensei was recognized in the "Best Artificial Intelligence Enabled Solution" category as the "solution best able to augment human intelligence and better automate decision support tasks" by incorporating "machine and deep learning algorithms into its everyday functionality."

According to foreign-policy experts and the defense establishment, the United States is caught in an artificial intelligence arms race with China--one with serious implications for national security. The conventional version of this story suggests that the United States is at a disadvantage because of self-imposed restraints on the collection of data and the privacy of its citizens, while China, an unrestrained surveillance state, is at an advantage. In this vision, the data that China collects will be fed into its systems, leading to more powerful AI with capabilities we can only imagine today. Since Western countries can't or won't reap such a comprehensive harvest of data from their citizens, China will win the AI arms race and dominate the next century. This idea makes for a compelling narrative, especially for those trying to justify surveillance--whether government- or corporate-run.

Deep learning AI is ready for adoption in our industry, but we need to first understand how artificial intelligence works, address privacy concerns, and how to implement it to solve specific safety, security or business risks,

If you consider Google is the leader globally in artificial intelligence, DeepMind is their crown jewel. When they moved the DeepMind Health unit, the healthcare subsidiary, into their main company -- that broke a pledge that'data will not be connected to Google accounts' -- you knew Google was cutting corners. Bigger than the Department of Justice going after Google for antitrust is the harm DeepMind could do to the future of artificial intelligence. They are arguably the leader in deep learning. The choices they make will decide many things about the fate of humanity in an AI-centric world.

In this post, I walk through how to build and train an neural translation model with attention. This model will be used to translate French to English. This post will focus on the conceptual explanation, while a detailed walk through of the project code can be found in the associated Jupyter notebook. This notebook can be viewed here or cloned from the project Github repository, here. This project closely follows the PyTorch Sequence to Sequence tutorial, while attempting to go more in depth with both the model implementation and the explanation. We are trying to build a translation model. One model that has been successful in this task is an Encoder-Decoder network.

Backpropagation has been widely used in deep learning approaches, but it is inefficient and sometimes unstable because of backward locking and vanishing/exploding gradient problems, especially when the gradient flow is long. Additionally, updating all edge weights based on a single objective seems biologically implausible. In this paper, we introduce a novel biologically motivated learning structure called Associated Learning, which modularizes the network into smaller components, each of which has a local objective. Because the objectives are mutually independent, Associated Learning can learn the parameters independently and simultaneously when these parameters belong to different components. Surprisingly, training deep models by Associated Learning yields comparable accuracies to models trained using typical backpropagation methods, which aims at fitting the target variable directly. Moreover, probably because the gradient flow of each component is short, deep networks can still be trained with Associated Learning even when some of the activation functions are sigmoid-a situation that usually results in the vanishing gradient problem when using typical backpropagation. We also found that the Associated Learning generates better metafeatures, which we demonstrated both quantitatively (via inter-class and intra-class distance comparisons in the hidden layers) and qualitatively (by visualizing the hidden layers using t-SNE).

In recent years, deep learning has proven to be a viable methodology for surrogate modeling and uncertainty quantification for a vast number of physical systems. However, in their traditional form, such models require a large amount of training data. This is of particular importance for various engineering and scientific applications where data may be extremely expensive to obtain. To overcome this shortcoming, physics-constrained deep learning provides a promising methodology as it only utilizes the governing equations. In this work, we propose a novel auto-regressive dense encoder-decoder convolutional neural network to solve and model transient systems with non-linear dynamics at a computational cost that is potentially magnitudes lower than standard numerical solvers. This model includes a Bayesian framework that allows for uncertainty quantification of the predicted quantities of interest at each time-step. We rigorously test this model on several non-linear transient partial differential equation systems including the turbulence of the Kuramoto-Sivashinsky equation, multi-shock formation and interaction with 1D Burgers' equation and 2D wave dynamics with coupled Burgers' equations. For each system, the predictive results and uncertainty are presented and discussed together with comparisons to the results obtained from traditional numerical analysis methods.