We asked GPT-3, OpenAI’s powerful new language generator, to write an essay for us from scratch. The assignment? To convince us robots come in peace

The release of openly available, robust natural language generation algorithms (NLG) has spurred much public attention and debate. One reason lies in the algorithms' purported ability to generate human-like text across various domains. Empirical evidence using incentivized tasks to assess whether people (a) can distinguish and (b) prefer algorithm-generated versus human-written text is lacking. We conducted two experiments assessing behavioral reactions to the state-of-the-art Natural Language Generation algorithm GPT-2 (Ntotal = 830). Using the identical starting lines of human poems, GPT-2 produced samples of poems. From these samples, either a random poem was chosen (Human-out-of-the-loop) or the best one was selected (Human-in-the-loop) and in turn matched with a human-written poem. In a new incentivized version of the Turing Test, participants failed to reliably detect the algorithmically-generated poems in the Human-in-the-loop treatment, yet succeeded in the Human-out-of-the-loop treatment. Further, people reveal a slight aversion to algorithm-generated poetry, independent on whether participants were informed about the algorithmic origin of the poem (Transparency) or not (Opacity). We discuss what these results convey about the performance of NLG algorithms to produce human-like text and propose methodologies to study such learning algorithms in human-agent experimental settings.

Few in the natural language processing (NLP) community expected the world's most powerful large language model to come cheap, but some are worried the hefty price tag could put it out of reach of startups. OpenAI's 175 billion parameter language model GPT-3 (Generative Pre-trained Transformer 3) turned heads in the NLP community when it was released in June, and now it's back in the spotlight. A Reddit post this week by independent writer and researcher Gwern Branwen detailed the pricing plan OpenAI has provided to GPT-3 Beta API users. The scheme, which goes into effect on October 1, has already raised as many questions as it has answered. According to reports, OpenAI announced the pricing scheme for GPT-3's API usage from October. The plan has four tiers: Explore, Create, Build, Scale.

Google Maps is one of the company's most widely-used products, and its ability to predict upcoming traffic jams makes it indispensable for many drivers. Each day, says Google, more than 1 billion kilometers of road are driven with the app's help. But, as the search giant explains in a blog post today, its features have got more accurate thanks to machine learning tools from DeepMind, the London-based AI lab owned by Google's parent company Alphabet. In the blog post, Google and DeepMind researchers explain how they take data from various sources and feed it into machine learning models to predict traffic flows. This data includes live traffic information collected anonymously from Android devices, historical traffic data, information like speed limits and construction sites from local governments, and also factors like the quality, size, and direction of any given road.

Note that our human feedback models generate summaries that are significantly shorter than summaries from models trained on CNN/DM. At a given summary length, our 6.7B human feedback model trained on Reddit performs almost as well as a fine-tuned 11B T5 model, despite not being re-trained on CNN/DM. To test our models' generalization, we also applied them directly to the popular CNN/DM news dataset. These articles are more than twice as long as Reddit posts and are written in a very different style. Our models have seen news articles during pre-training, but all of our human data and RL fine-tuning was on the Reddit TL;DR dataset.

This article presents a hybrid approach based on a Grounded Text Generation (GTG) model to building robust task bots at scale. GTG is a hybrid model which uses a large-scale Transformer neural network as its backbone, combined with symbol-manipulation modules for knowledge base inference and prior knowledge encoding, to generate responses grounded in dialog belief state and real-world knowledge for task completion. GTG is pre-trained on large amounts of raw text and human conversational data, and can be fine-tuned to complete a wide range of tasks. The hybrid approach and its variants are being developed simultaneously by multiple research teams. The primary results reported on task-oriented dialog benchmarks are very promising, demonstrating the big potential of this approach. This article provides an overview of this progress and discusses related methods and technologies that can be incorporated for building robust conversational AI systems.

In Machine Learning, White Box Adversarial Attacks rely on knowing underlying knowledge about the model attributes. This works focuses on discovering to distrinct pieces of model information: the underlying architecture and primary training dataset. With the process in this paper, a structured set of input probes and the output of the model become the training data for a deep classifier. Two subdomains in Machine Learning are explored: image based classifiers and text transformers with GPT-2. With image classification, the focus is on exploring commonly deployed architectures and datasets available in popular public libraries. Using a single transformer architecture with multiple levels of parameters, text generation is explored by fine tuning off different datasets. Each dataset explored in image and text are distinguishable from one another. Diversity in text transformer outputs implies further research is needed to successfully classify architecture attribution in text domain.

OpenAI Gym is a toolkit that provides a wide variety of simulated environments (Atari games, board games, 2D and 3D physical simulations, and so on), so you can train agents, compare them, or develop new Machine Learning algorithms (Reinforcement Learning). OpenAI is an artificial intelligence research company, funded in part by Elon Musk. Its stated goal is to promote and develop friendly AIs that will benefit humanity (rather than exterminate it). In this article, I will be using the OpenAI gym, a great toolkit for developing and comparing Reinforcement Learning algorithms. It provides many environments for your learning agents to interact with.

Throughout time, people have dreamt of creating human-like intelligent machines. We've been hearing recently about GPT3 – a new AI speech system from San Francisco Its developers claim that it can answer general questions, correct and complete texts, and even write them itself, without any task-specific training. GPT3 is so good that the texts it generates can scarcely be distinguished from those written by a human. So what do we make out of this? GPT3 is an artificial neuronal network that is trained with a text data set of 500 billion character strings drawn from the entire Internet (filtered), Wikipedia and several digitised book collections.

GPT-3: Language Models are Few-Shot Learners May 29, 2020 Link Hacker News (431 points, 291 comments) Reddit (271 points, 113 comments) The original GPT-3 paper from OpenAI.