The recently proposed Temporal Ensembling has achieved state-of-the-art results in several semi-supervised learning benchmarks. It maintains an exponential moving average of label predictions on each training example, and penalizes predictions that are inconsistent with this target. However, because the targets change only once per epoch, Temporal Ensembling becomes unwieldy when learning large datasets. To overcome this problem, we propose Mean Teacher, a method that averages model weights instead of label predictions. As an additional benefit, Mean Teacher improves test accuracy and enables training with fewer labels than Temporal Ensembling. Without changing the network architecture, Mean Teacher achieves an error rate of 4.35% on SVHN with 250 labels, outperforming Temporal Ensembling trained with 1000 labels. We also show that a good network architecture is crucial to performance. Combining Mean Teacher and Residual Networks, we improve the state of the art on CIFAR-10 with 4000 labels from 10.55% to 6.28%, and on ImageNet 2012 with 10% of the labels from 35.24% to 9.11%.

The present paper aims to demonstrate the usage of Convolutional Neural Networks as a generative model for stochastic processes, enabling researchers from a wide range of fields - such as quantitative finance and physics - to develop a general tool for forecasts and simulations without the need to identify/assume a specific system structure or estimate its parameters.

Pileup involves the contamination of the energy distribution arising from the primary collision of interest (leading vertex) by radiation from soft collisions (pileup). We develop a new technique for removing this contamination using machine learning and convolutional neural networks. The network takes as input the energy distribution of charged leading vertex particles, charged pileup particles, and all neutral particles and outputs the energy distribution of particles coming from leading vertex alone. The PUMML algorithm performs remarkably well at eliminating pileup distortion on a wide range of simple and complex jet observables. We test the robustness of the algorithm in a number of ways and discuss how the network can be trained directly on data.

To perform tasks specified by natural language instructions, autonomous agents need to extract semantically meaningful representations of language and map it to visual elements and actions in the environment. This problem is called task-oriented language grounding. We propose an end-to-end trainable neural architecture for task-oriented language grounding in 3D environments which assumes no prior linguistic or perceptual knowledge and requires only raw pixels from the environment and the natural language instruction as input. The proposed model combines the image and text representations using a Gated-Attention mechanism and learns a policy to execute the natural language instruction using standard reinforcement and imitation learning methods. We show the effectiveness of the proposed model on unseen instructions as well as unseen maps, both quantitatively and qualitatively. We also introduce a novel environment based on a 3D game engine to simulate the challenges of task-oriented language grounding over a rich set of instructions and environment states.

The combination of Deep Learning with Apache Spark has the potential for tremendous impact in many sectors of the industry. Apache Spark has quickly become a critical technology for data scientists to explore, understand and transform massive datasets and to build and train advanced machine learning models. Machine learning and AI has begun to unlock new possibilities that are creating a competitive advantage for companies. However, companies continue to struggle to increase the productivity of data scientists. The biggest hurdle to accelerate innovation has been the time to train, validate and deploy models.

I am both personally interested in Machine Learning, as well as for my PhD which is loosely related to ML, and as someone working on a startup in his spare time. I often come into situations where I need an expert in Machine Learning to push me into the right direction. But I often cannot disclose my problem publicly (e.g. because my PhD has a project sponsor). What are options for me to get a Machine Learning mentor (an actual expert ideally at the cross section of computer vision / image manipulation and Deep Learning)? How could I set up a basic agreement that I could ask some questions once in a while in exchange for some compensation (monetary or otherwise)?

CES covers the full technology ecosystem, from fitness and health to self-driving technology. Artificial Intelligence (AI) was the tech buzz word for 2017, but behind all of the headlines were real developments. Narrow applications of AI will be everywhere at CES. These include speech recognition, computer vision and machine learning. Some of the biggest trends in the tech industry have AI behind them and we'll see them all at CES 2018.

The revelation of the security bugs couldn't come at a worse time for Intel, making this even more of a PR nightmare than it might first seem. It was just in November that Intel announced that it hired AMD's former graphics head and plans to enter the discrete GPU market. This move wasn't all that surprising, as NVIDIA's data center platform has been profiting mightily from rapid adoption of the company's GPU-based approach to deep learning, a category of artificial intelligence (AI) that essentially trains a machine to think -- or make inferences from data -- like we humans do. All the major internet companies, cloud computing service providers, and computer server makers are using NVIDIA's GPUs to accelerate their deep-learning training and high-performance computing.

Machine Learning and Deep Learning are increasingly used to analyze scientific data, in fields as diverse as neuroscience, climate science and particle physics. In this page you will find links to examples of scientific use cases using deep learning at NERSC, information about what deep learning packages are available at NERSC, and details of how to scale up your deep learning code on Cori to take advantage of the compute power available from Cori's KNL nodes. We have assembled some examples of machine learning projects being carried out at NERSC, in most cases including links to the codebase. NERSC supports several software frameworks for machine learning and deep learning. If there is a framework you would like to see evaluated and supported at NERSC, please let us know.

In order to put more computing power in the hands of customers who need powerful supercomputers and Artificial Intelligence (AI) platforms, Lenovo has expanded its support for multiple graphics processing units, (GPUs), from Nvidia. Select Lenovo systems will now be supporting the NVIDIA Tesla M10, P40, and P100 GPU accelerators and NVIDIA deep learning platform for traditional HPC, as well as emerging AI deep learning applications. Tesla P100 for PCIe based servers are ideal for both HPC and deep learning training deployments. The P100 is powered by the powerful NVIDIA Pascal architecture, with 16GB of onboard memory, for demanding HPC applications. With up to 4.7 TeraFLOPS of double-precision performance, a single P100 node can replace up to 32 traditional CPU nodes.