Goto

Collaborating Authors

 Deep Learning


Low-bit Quantization of Neural Networks for Efficient Inference

arXiv.org Machine Learning

Recent breakthrough methods in machine learning make use of increasingly large deep neural networks. The gains in performance have come at the cost of a substantial increase in computation and storage, making real-time implementation on limited hardware a very challenging task. One popular approach to address this challenge is to perform low-bit precision computations via neural network quantization. However, aggressive quantization generally entails a severe penalty in terms of accuracy and usually requires the retraining of the network or resorts to higher bit precision quantization. In this paper, we formalize the linear quantization task as a Minimum Mean Squared Error (MMSE) problem for both weights and activations. This allows low-bit precision inference without the need for full network retraining. The main contributions of our approach is the optimization of the constrained MSE problem at each layer of the network, the hardware aware partitioning of the neural network parameters, and the use of multiple low precision quantized tensors for poorly approximated layers. The proposed approach allows for the first time a linear 4 bits integer precision (INT4) quantization for deployment of pretrained models on limited hardware resources.


Deep Learning for Video Game Playing

arXiv.org Artificial Intelligence

In this article, we review recent Deep Learning advances in the context of how they have been applied to play different types of video games such as first-person shooters, arcade games, and real-time strategy games. We analyze the unique requirements that different game genres pose to a deep learning system and highlight important open challenges in the context of applying these machine learning methods to video games, such as general game playing, dealing with extremely large decision spaces and sparse rewards.


batra-mlp-lab/visdial-challenge-starter-pytorch

#artificialintelligence

This starter code is implemented using PyTorch v1.0, and provides out of the box support with CUDA 9 and CuDNN 7. There are two recommended ways to set up this codebase: Anaconda or Miniconda, and Docker. Note: Docker setup is necessary if you wish to extract image features using Detectron. We provide a Dockerfile which creates a light-weight image with all the dependencies installed. We recommend this development workflow, attaching the codebase as a volume would immediately reflect source code changes inside the container environment.


AI's Deep Problem

#artificialintelligence

Artificial intelligence is modeled to some extent on the human brain; and there's a deep problem with this approach. Machine learning is a subset of artificial intelligence (AI) where computer programs automatically learn from data without explicit programming. Inspired in part by the human biology, deep learning is a machine learning method that deploys layers of artificial neurons, called nodes, in an artificial brain called a neural network. Neuroscientists and psychologists have yet to fully understand how the human brain works. Similarly, there's a big problem with deep learning; scientists do not really know exactly how deep learning reaches its decisions.


AI news-writing system deemed too dangerous to release

#artificialintelligence

OpenAI, a company backed by Elon Musk, has decided not to release an AI system that can generate news stories and fiction on the grounds that it could be dangerous in the wrong hands. OpenAI is a non-profit company that aims to finding a way to safely bring about artificial general intelligence. Normally it releases its research to the public, but its latest AI model, known as GPT-2, is reportedly so convincing that it has too much potential for misuse, generating huge volumes of misleading news stories. GPT-2 takes a sample of text (a few words of several paragraphs) and predicts the following sentences in a similar style, with surprisingly plausible results. The system was trained using a dataset of roughly 10 million news articles sourced by trawling Reddit – several times the size of those used by previous state-of-the-art systems. The sheer volume of data gave the system a much better understanding of written language, and means it's more general purpose than other systems.


This Article Is Fake News. But It's Also The Work of AI

#artificialintelligence

The use of fake news stories for political disinformation has become a major concern for governments around the world in the wake of the 2016 U.S. presidential election. The Federal Bureau of Investigation concluded Russia used false news reports, spread through social media, to try to sway voters. Writing these stories still needed someone to sit behind a keyboard. Now OpenAI, a non-profit artificial intelligence research group in San Francisco, has unveiled a machine learning algorithm that can generate coherent text, including fake news articles, after being given just a small sample to build on. The algorithm can be tuned to imitate the writing style of the sample text.


Researchers, scared by their own work, hold back "deepfakes for text" AI

#artificialintelligence

OpenAI, a non-profit research company investigating "the path to safe artificial intelligence," has developed a machine learning system called Generative Pre-trained Transformer-2 (GPT-2), capable of generating text based on brief writing prompts. The result comes so close to mimicking human writing that it could potentially be used for "deepfake" content. Built based on 40 gigabytes of text retrieved from sources on the Internet (including "all outbound links from Reddit, a social media platform, which received at least 3 karma"), GPT-2 generates plausible "news" stories and other text that match the style and content of a brief text prompt. The performance of the system was so disconcerting, now the researchers are only releasing a reduced version of GPT-2 based on a much smaller text corpus. Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT-2 along with sampling code.


An Elon Musk-backed AI firm is keeping a text generating tool under wraps amid fears it's too dangerous

#artificialintelligence

AI research nonprofit OpenAI has created a system that can generate fake text from a single line -- and it's not open-sourcing the code for fear of misuse. OpenAI was cofounded by tech mogul Elon Musk, and its sponsors include Silicon Valley heavy-hitters such as Peter Thiel and Amazon Web Services. Last year it gained the praise of Bill Gates after it built a team of five neural networks capable of beating human players in the computer game "Dota 2." Read more: Bill Gates hails "huge milestone" for AI as bots work in a team to destroy humans at video game "Dota 2" Now the company has created a system, named GPT2, capable of imitating and generating text based on only a sentence. The Guardian's Alex Hern got to play with the system, and tried typing in a single Guardian headline about Brexit. From that headline alone, GPT2 was able to generate quotes from UK Labour leader Jeremy Corbyn as well as a fictional spokesman for Prime Minister Theresa May.


Nvidia has created the first video game demo using AI-generated graphics

#artificialintelligence

The recent boom in artificial intelligence has produced impressive results in a somewhat surprising realm: the world of image and video generation. The latest example comes from chip designer Nvidia, which today published research showing how AI-generated visuals can be combined with a traditional video game engine. The result is a hybrid graphics system that could one day be used in video games, movies, and virtual reality. "It's a new way to render video content using deep learning," Nvidia's vice president of applied deep learning, Bryan Catanzaro, told The Verge. "Obviously Nvidia cares a lot about generating graphics [and] we're thinking about how AI is going to revolutionize the field."


How one scientist coped when AI beat him at his life's work

#artificialintelligence

It was with a strangely deflated feeling in his gut that Harvard biologist Mohammed AlQuraishi made his way to Cancun for a scientific conference in December. Strange because a major advance had just been made in his field, something that might normally make him happy. Deflated because the advance hadn't been made by him or by any of his fellow academic researchers. It had been made by a machine. DeepMind, an AI company that Google bought in 2014, had outperformed all the researchers who'd submitted entries to the Critical Assessment of Structure Prediction (CASP) conference, which is basically a fancy science contest for grown-ups. Every two years, researchers working on one of the biggest puzzles in biochemistry, known as the protein folding problem, try to prove how good their predictive powers are by submitting a prediction about the 3D shapes that certain proteins will take.