We augment recurrent neural networks with an external memory mechanism that builds upon recent progress in metalearning. We conceptualize this memory as a rapidly adaptable function that we parameterize as a deep neural network. Reading from the neural memory function amounts to pushing an input (the key vector) through the function to produce an output (the value vector). Writing to memory means changing the function; specifically, updating the parameters of the neural network to encode desired information. We leverage training and algorithmic techniques from metalearning to update the neural memory function in one shot. The proposed memory-augmented model achieves strong performance on a variety of learning problems, from supervised question answering to reinforcement learning.

Graph structured data provide two-fold information: graph structures and node attributes. Numerous graph-based algorithms rely on both information to achieve success in supervised tasks, such as node classification and link prediction. However, node attributes could be missing or incomplete, which significantly deteriorates the performance. The task of node attribute generation aims to generate attributes for those nodes whose attributes are completely unobserved. This task benefits many real-world problems like profiling, node classification and graph data augmentation. To tackle this task, we propose a deep adversarial learning based method to generate node attributes; called node attribute neural generator (NANG). NANG learns a unifying latent representation which is shared by both node attributes and graph structures and can be translated to different modalities. We thus use this latent representation as a bridge to convert information from one modality to another. We further introduce practical applications to quantify the performance of node attribute generation. Extensive experiments are conducted on four real-world datasets and the empirical results show that node attributes generated by the proposed method are high-qualitative and beneficial to other applications. The datasets and codes are available online.

Variational autoencoders (VAE) have quickly become a central tool in machine learning, applicable to a broad range of data types and latent variable models. By far the most common first step, taken by seminal papers and by core software libraries alike, is to model MNIST data using a deep network parameterizing a Bernoulli likelihood. This practice contains what appears to be and what is often set aside as a minor inconvenience: the pixel data is [0, 1] valued, not {0, 1} as supported by the Bernoulli likelihood. Here we show that, far from being a triviality or nuisance that is convenient to ignore, this error has profound importance to VAE, both qualitative and quantitative. We introduce and fully characterize a new [0, 1]-supported, single parameter distribution: the continuous Bernoulli, which patches this pervasive bug in VAE. This distribution is not nitpicking; it produces meaningful performance improvements across a range of metrics and datasets, including sharper image samples, and suggests a broader class of performant VAE.

Graph Neural Networks (GNNs) have become a topic of intense research recently due to their powerful capability in high-dimensional classification and regression tasks for graph-structured data. However, as GNNs typically define the graph convolution by the orthonormal basis for the graph Laplacian, they suffer from high computational cost when the graph size is large. This paper introduces the Haar basis, a sparse and localized orthonormal system for graph, constructed from a coarse-grained chain on the graph. The graph convolution under Haar basis --- the Haar convolution can be defined accordingly for GNNs. The sparsity and locality of the Haar basis allow Fast Haar Transforms (FHTs) on graph, by which a fast evaluation of Haar convolution between the graph signals and the filters can be achieved. We conduct preliminary experiments on GNNs equipped with Haar convolution, which can obtain state-of-the-art results for a variety of geometric deep learning tasks.

LOS ANGELES - Microsoft on Monday announced a $1 billion investment in an OpenAI ethical artificial intelligence project backed by Tesla's Elon Musk and Amazon. The partnership will be devoted to developing advanced AI models on Microsoft's Azure cloud computing platform while adhering to "shared principles on ethics and trust," the companies said in a joint release. OpenAI and Microsoft expressed a vision of "artificial general intelligence" (AGI) working with people to help solve daunting problems such as climate change. OpenAI chief executive Sam Altman said the goal of the effort is to allow artificial intelligence to be "deployed safely and securely and that its economic benefits are widely distributed." Microsoft will become the preferred partner for commercializing new "supercomputing" artificial intelligence technologies developed as part of the initiative.

Elon Musk announced that his company Neuralink plans to link human brains directly to computers, saying the first prototype could be implanted in a person by the end of 2020. Microsoft has agreed to invest $1 billion in and partner with research company OpenAI, co-founded by Elon Musk, to develop artificial general intelligence, a technology that could have human-level intellectual capacity. The companies said Monday that they will build a hardware and software platform of "unprecedented scale" within Microsoft's cloud service provider Azure that will train and run increasingly advanced AI models. Microsoft will also become OpenAI's preferred partner for selling its technologies and the two will jointly develop Azure's supercomputing technology. "By bringing together OpenAI's breakthrough technology with new Azure AI supercomputing technologies, our ambition is to democratize AI -- while always keeping AI safety front and center -- so everyone can benefit," said Microsoft CEO Satya Nadella in the statement.

This article was originally published on the National Center for Supercomputing Applications website. In 2007, the Sloan Digital Sky Survey (SDSS) launched a citizen science campaign called Galaxy Zoo to enlist the public's help in classifying the hundreds of thousands of galaxy images captured by an optical telescope. Through this highly successful crowdsourcing effort, volunteers reviewed the images online to help determine whether each galaxy had a spiral or elliptical structure. Leveraging data generated by the Galaxy Zoo project, a team of scientists is now applying the power of artificial intelligence (AI) and high-performance supercomputers to accelerate efforts to analyze the increasingly massive datasets produced by ongoing and future cosmological surveys. In a new study, researchers from the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign and the Argonne Leadership Computing Facility (ALCF) at the U.S. Department of Energy's (DOE) Argonne National Laboratory have developed a novel combination of deep learning methods to provide a highly accurate approach to classifying hundreds of millions of unlabeled galaxies.

Microsoft is investing $1 billion in OpenAI, a San Francisco-based research lab founded by Silicon Valley luminaries, including Elon Musk and Sam Altman, that's dedicated to creating artificial general intelligence (AGI). The investment will make Microsoft the "exclusive" provider of cloud computing services to OpenAI, and the two companies will work together to develop new technologies. OpenAI will also license some of its tech to Microsoft to commercialize, though when this may happen and what tech will be involved has yet to be announced. OpenAI began as a nonprofit research lab in 2015 and was intended to match the high-tech R&D of companies like Google and Amazon while focusing on developing AI in a safe and democratic fashion. But earlier this year, OpenAI said it needed more money to continue this work, and it set up a new for-profit firm to seek outside investment. To attract backers, OpenAI has made outrageous promises about the potential of its technology.

Machine Learning Algorithms – DataScientist may be the sexiest job of today but the understanding, implementation, applied ML experience is missing. Having the top algorithms on your fingertips in real business is missing big time. The real job for any data scientist is the ability to clarify, demonstrate, extract real values out of data and reap rewards. "Machine Learning" as a basic skill sounds like teleportation tool to many businesses especially for the companies which are actually data factories i.e social media platforms. Describing and picturising the top few machine learning algorithms is the main idea of this post.

A language model can predict the probability of the next word in the sequence, based on the words already observed in the sequence. Neural network models are a preferred method for developing statistical language models because they can use a distributed representation where different words with similar meanings have similar representation and because they can use a large context of recently observed words when making predictions. In this tutorial, you will discover how to develop a statistical language model using deep learning in Python. How to Develop a Word-Level Neural Language Model and Use it to Generate Text Photo by Carlo Raso, some rights reserved. The Republic is the classical Greek philosopher Plato's most famous work. It is structured as a dialog (e.g. The entire text is available for free in the public domain. It is available on the Project Gutenberg website in a number of formats. Download the book text and place it in your current working directly with the filename'republic.txt'