Online discussions often derail into toxic exchanges between participants. Recent efforts mostly focused on detecting antisocial behavior after the fact, by analyzing single comments in isolation. To provide more timely notice to human moderators, a system needs to preemptively detect that a conversation is heading towards derailment before it actually turns toxic. This means modeling derailment as an emerging property of a conversation rather than as an isolated utterance-level event. Forecasting emerging conversational properties, however, poses several inherent modeling challenges. First, since conversations are dynamic, a forecasting model needs to capture the flow of the discussion, rather than properties of individual comments. Second, real conversations have an unknown horizon: they can end or derail at any time; thus a practical forecasting model needs to assess the risk in an online fashion, as the conversation develops. In this work we introduce a conversational forecasting model that learns an unsupervised representation of conversational dynamics and exploits it to predict future derailment as the conversation develops. By applying this model to two new diverse datasets of online conversations with labels for antisocial events, we show that it outperforms state-of-the-art systems at forecasting derailment.

W e present a systematic study of biases in natural language generation (NLG) by analyzing text generated from prompts that contain mentions of different demographic groups. In this work, we introduce the notion of the regard towards a demographic, use the varying levels of regard towards different demographics as a defining metric for bias in NLG, and analyze the extent to which sentiment scores are a relevant proxy metric for regard. To this end, we collect strategically-generated text from language models and manually annotate the text with both sentiment and regard scores. Additionally, we build an automatic regard classifier through transfer learning, so that we can analyze biases in unseen text. Together, these methods reveal the extent of the biased nature of language model generations. Our analysis provides a study of biases in NLG, bias metrics and correlated human judgments, and empirical evidence on the usefulness of our annotated dataset.

Reproducibility is an essential requirement for many fields of research including those based on machine learning techniques. However, many machine learning publications are either not reproducible or are difficult to reproduce. With the continued growth in the number of research publications, including tens of thousands of papers now hosted on arXiv and submissions to conferences at an all time high, research reproducibility is more important than ever. While many of these publications are accompanied by code as well as trained models which is helpful but still leaves a number of steps for users to figure out for themselves. We are excited to announce the availability of PyTorch Hub, a simple API and workflow that provides the basic building blocks for improving machine learning research reproducibility.

For readers unfamiliar with the drug discovery and development process, it is worthwhile to point out that the pharmaceutical industry is one of the most inefficient and risky industries on the planet. The efficiency of the industry has been on the decline since the 1950s. It costs over $2.6 Billion to bring the New Molecular Entity (NME) to the market. And despite the many advances disrupting other industries including personal computing, the Internet, and genome sequencing, the cost to develop a drug is steadily increasing. This is one of the reasons why most industry experts are skeptical about the promises of deep learning.

This article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI. Last week, artificial intelligence research lab OpenAI decided to release a more expanded version of GPT-2, the controversial text-generating AI model it first introduced in February. At the time, the lab refrained from releasing the full AI model, fearing it would be used for malicious purposes. Instead, OpenAI opted for a staged release of the AI, starting with a limited model (124 million parameters), and gradually releasing more capable models. In May, the research lab released the 355-million-parameter version of GPT-2, and last week, it finally released the 774-million-model, at 50 percent capacity of the text generator.

The latest AI project from the MIT Media Lab is demonstrating just how terrifying the prospects of deep learning can go. Welcome to the Nightmare Machine: an algorithm that has been trained to generate horrifying images. It is attempting to find the scariest faces and locations possible, and gets humans to tell it which are the worst. The first aspect of the project, Haunted Faces, is truly terrifying. The team behind the project, led by Iyad Rahwan, associate professor at MIT Media Lab, used deep learning to generate new faces, before dropping "a hint of scariness" onto the generated faces in the spirit of Halloween.

In this talk, I want to begin a conversation about what is the essence of deep learning and how we can optimally support this essence in the form of a programming interface or language. I'll give you my own impressions, and I hope to provoke an ongoing conversation. Despite the phenomenal success of deep learning, it's my sense that most of the choices made in the theory and practice of deep learning are nonessential and even harmful (unnecessarily complex and limited). I'll suggest that a very small addition to a modern typed functional programming language such as Haskell yields an ideal basis for deep learning that is much simpler, more general, and more rigorous that currently popular approaches.

Over the last six months I've been looking at deep learning on the edge, and investigating the new generation of custom silicon designed to speed up machine learning inferencing on embedded devices. The original accelerator hardware was launched by Intel back in 2017, but since then we've seen more hardware from Intel, Google, NVIDIA, and others. Right now, we're seeing a deluge of new hardware based around the Intel Movidius, and the Gyrfalcon Lightspeeur chips. I'm also expecting to see hardware based around Google's Edge TPU later in the year when their System-on-Module (SoM) is finally available in volume. However, there are other less known players in the accelerator market, one of these is Rockchip with their Neural Processing Unit (NPU).

Insilico Medicine aims to bring deep learning to the drug discovery process. Hong Kong-based Insilico Medicine published research Monday showing that its deep learning system could identify potential treatments for fibrosis. That system, called generative tensorial reinforcement learning, or GENTRL for short, was able to find six promising treatments in just 21 days, one of which showed promising results in an experiment involving mice. The research has been published in Nature Biotechnology, and the code for the model has been made available on Github. "We've got AI strategy combined with AI imagination," says Insilico CEO Alex Zhavoronkov, who compares the operation of GENTRL to the AlphaGo machine learning system that Google's Deepmind developed to challenge champion Go players.

Reinforcement learning and deep learning arose as separate disciplines within AI, but researchers are increasingly finding that pairing the two can deliver promising applications. Deep learning has excelled at tasks like training classifiers for image and speech recognition. Reinforcement learning techniques have excelled at creating AI systems that improve through trial and error to produce game-playing bots and recommendation engines. At the Re•Work Deep Reinforcement Learning Summit in San Francisco, researchers explored how the two approaches are being combined to craft more automated and optimized reinforcement learning algorithms. "In the last six years, we've been really focusing on getting this combination of deep networks and reinforcement learning to be more stable, more reliable, more predictable," said Marc Bellemare, research scientist at Google Brain, in an interview.