Goto

Collaborating Authors

Generation


AI's Latest Breakthrough Will Transform Learning--Here Are 5 Ways

#artificialintelligence

The Fourth Industrial Revolution just took a huge step forward, thanks to a breakthrough artificial intelligence (AI) model that can learn virtually anything about the world -- and produce the content to tell us about it. The AI program is GPT-3 by OpenAI, which started out as a language model to predict the next word in a sentence and has vastly exceeded that capability. Now, drawing from voluminous data -- essentially all of Wikipedia, links from Reddit, and other Internet content -- GPT-3 has shown it can also compose text that is virtually indistinguishable from human-generated content. Asger Alstrup Palm, Area9's chief technology officer, explained that GPT-3 was tasked with testing the "scaling hypothesis" -- to see if a bigger model with ever-increasing amounts of information would lead to better performance. Although it's too early to call the scaling hypothesis proven, there are some strong indications that this is, indeed, the case. Further validating the potential of GPT-3, Microsoft recently announced it will exclusively license the model from OpenAI, with the intention of developing and delivering AI solutions for customers and creating new solutions using natural language generation.


This extraordinary AI has stunned computer scientists with its writing ability

#artificialintelligence

However, our bot didn't "know" anything about "Chitra" or Tagore. It didn't generate fundamentally new ideas or sentences. It simply cobbled together parts of existing sentences from existing articles to make new ones. OpenAI, a for-profit company under a nonprofit parent company, has built a language generation program dubbed GPT-3, an acronym for "Generative Pre-trained Transformer 3." Its ability to learn, summarize, and compose text has stunned computer scientists like me. "I have created a voice for the unknown human who hides within the binary," GPT-3 wrote in response to one prompt. "I have created a writer, a sculptor, an artist.


AI's Latest Breakthrough Will Transform Learning--Here Are 5 Ways

#artificialintelligence

The Fourth Industrial Revolution just took a huge step forward, thanks to a breakthrough artificial intelligence (AI) model that can learn virtually anything about the world -- and produce the content to tell us about it. The AI program is GPT-3 by OpenAI, which started out as a language model to predict the next word in a sentence and has vastly exceeded that capability. Now, drawing from voluminous data -- essentially all of Wikipedia, links from Reddit, and other Internet content -- GPT-3 has shown it can also compose text that is virtually indistinguishable from human-generated content. Asger Alstrup Palm, Area9's chief technology officer, explained that GPT-3 was tasked with testing the "scaling hypothesis" -- to see if a bigger model with ever-increasing amounts of information would lead to better performance. Although it's too early to call the scaling hypothesis proven, there are some strong indications that this is, indeed, the case. Further validating the potential of GPT-3, Microsoft recently announced it will exclusively license the model from OpenAI, with the intention of developing and delivering AI solutions for customers and creating new solutions using natural language generation.


Microsoft gets exclusive license for OpenAI's GPT-3 language model

#artificialintelligence

Microsoft today announced that it will exclusively license GPT-3, one of the most powerful language understanding models in the world, from AI startup OpenAI. In a blog post, Microsoft EVP Kevin Scott said that the new deal will allow Microsoft to leverage OpenAI's technical innovations to develop and deliver AI solutions for customers, as well as create new solutions that harness the power of natural language generation. "We see this as an incredible opportunity to expand our Azure-powered AI platform in a way that democratizes AI technology, enables new products, services and experiences, and increases the positive impact of AI at scale," Scott wrote. "The scope of commercial and creative potential that can be unlocked through the GPT-3 model is profound, with genuinely novel capabilities -- most of which we haven't even imagined yet. Directly aiding human creativity and ingenuity in areas like writing and composition, describing and summarizing large blocks of long-form data (including code), converting natural language to another language -- the possibilities are limited only by the ideas and scenarios that we bring to the table."


A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation

arXiv.org Artificial Intelligence

This work studies the widely adopted ancestral sampling algorithms for auto-regressive language models, which is not widely studied in the literature. We use the quality-diversity (Q-D) trade-off to investigate three popular sampling algorithms (top-k, nucleus and tempered sampling). We focus on the task of open-ended language generation. We first show that the existing sampling algorithms have similar performance. After carefully inspecting the transformations defined by different sampling algorithms, we identify three key properties that are shared among them: entropy reduction, order preservation, and slope preservation. To validate the importance of the identified properties, we design two sets of new sampling algorithms: one set in which each algorithm satisfies all three properties, and one set in which each algorithm violates at least one of the properties. We compare their performance with existing sampling algorithms, and find that violating the identified properties could lead to drastic performance degradation, as measured by the Q-D trade-off. On the other hand, we find that the set of sampling algorithms that satisfies these properties performs on par with the existing sampling algorithms. Our data and code are available at https://github.com/moinnadeem/characterizing-sampling-algorithms


The Development of Augmented Analytics.

#artificialintelligence

If data is the gas in a car, then, analytics is the car itself. Currently, there are a few trends and topics in tech without which the talk around technology and innovation is incomplete -- analytics, artificial intelligence, blockchain to name a few. Augmented analytics is an extension of analytics that focuses on three main areas -- Machine Learning, Natural language generation (NLP) and, Insight automation. The basic premise of augmented analytics is the elimination of painstaking tasks in the process of data analysis and, replacing them by automation thus, refocusing human attention on modern analytics, business process, and business value generation. As per predictions made by Gartner, over 40% of tasks involved in data science will be automated thus, increasing productivity, quickening the process, and initiating broader usage of data and analytics.


Bringing AI Supercomputing To Customers - Liwaiwai

#artificialintelligence

The trend toward the use of massive AI models to power a large number of tasks is changing how AI is built. At Microsoft Build 2020, we shared our vision for AI at Scale utilizing state-of-the-art AI supercomputing in Azure and a new class of large-scale AI models enabling next-generation AI. The advantage of large scale models is that they only need to be trained once with massive amounts of data using AI supercomputing, enabling them to then be "fine-tuned" for different tasks and domains with much smaller datasets and resources. The more parameters that a model has, the better it can capture the difficult nuances of the data, as demonstrated by our 17-billion-parameter Turing Natural Language Generation (T-NLG) model and its ability to understand language to answer questions from or summarize documents seen for the first time. Natural language models like this, significantly larger than the state-of-the-art models a year ago, and many orders of magnitude the size of earlier image-centric models, are now powering a variety of tasks throughout Bing, Word, Outlook, and Dynamics.


Discovering Textual Structures: Generative Grammar Induction using Template Trees

arXiv.org Artificial Intelligence

Natural language generation provides designers with methods for automatically generating text, e.g. for creating summaries, chatbots and game content. In practise, text generators are often either learned and hard to interpret, or created by hand using techniques such as grammars and templates. In this paper, we introduce a novel grammar induction algorithm for learning interpretable grammars for generative purposes, called Gitta. We also introduce the novel notion of template trees to discover latent templates in corpora to derive these generative grammars. By using existing human-created grammars, we found that the algorithm can reasonably approximate these grammars using only a few examples. These results indicate that Gitta could be used to automatically learn interpretable and easily modifiable grammars, and thus provide a stepping stone for human-machine co-creation of generative models.


Artificial Intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry

arXiv.org Artificial Intelligence

The release of openly available, robust natural language generation algorithms (NLG) has spurred much public attention and debate. One reason lies in the algorithms' purported ability to generate human-like text across various domains. Empirical evidence using incentivized tasks to assess whether people (a) can distinguish and (b) prefer algorithm-generated versus human-written text is lacking. We conducted two experiments assessing behavioral reactions to the state-of-the-art Natural Language Generation algorithm GPT-2 (Ntotal = 830). Using the identical starting lines of human poems, GPT-2 produced samples of poems. From these samples, either a random poem was chosen (Human-out-of-the-loop) or the best one was selected (Human-in-the-loop) and in turn matched with a human-written poem. In a new incentivized version of the Turing Test, participants failed to reliably detect the algorithmically-generated poems in the Human-in-the-loop treatment, yet succeeded in the Human-out-of-the-loop treatment. Further, people reveal a slight aversion to algorithm-generated poetry, independent on whether participants were informed about the algorithmic origin of the poem (Transparency) or not (Opacity). We discuss what these results convey about the performance of NLG algorithms to produce human-like text and propose methodologies to study such learning algorithms in human-agent experimental settings.


The Impact of AI on Journalism

#artificialintelligence

Back in 2014, the Los Angeles Times published a report about an earthquake three minutes after it happened. This feat was possible because a staffer had developed a bot (a software robot) called Quakebot to write automated articles based on data generated by the US Geological Survey. Today, AIs write hundreds of thousands of the articles that are published by mainstream media outlets every week. At first, most of the Natural Language Generation (NLG) tools producing these articles were provided by software companies like Narrative Science. Today, many media organisations have developed in-house versions.