"As for why I tell a lot of stories, there's a joke about that. There was once a man who had a computer, and he asked it, 'Do you compute that you will ever be able to think like a human being?' And after assorted grindings and beepings, a slip of paper came out of the computer that said, 'That reminds me of a story . . . "
– from ANGELS FEAR: TOWARDS AN EPISTEMOLOGY OF THE SACRED. Gregory Bateson & Mary Catherine Bateson. (Part III 'Metalogue').
Generative adversarial networks (GANs) have great successes on synthesizing data. However, the existing GANs restrict the discriminator to be a binary classifier, and thus limit their learning capacity for tasks that need to synthesize output with rich structures such as natural language descriptions. In this paper, we propose a novel generative adversarial network, RankGAN, for generating high-quality language descriptions. Rather than training the discriminator to learn and assign absolute binary predicate for individual data sample, the proposed RankGAN is able to analyze and rank a collection of human-written and machine-written sentences by giving a reference group. By viewing a set of data samples collectively and evaluating their quality through relative ranking scores, the discriminator is able to make better assessment which in turn helps to learn a better generator.
The latest trend in AI is that larger natural language models provide better accuracy; however, larger models are difficult to train because of cost, time, and ease of code integration. Microsoft is releasing an open-source library called DeepSpeed, which vastly advances large model training by improving scale, speed, cost, and usability, unlocking the ability to train 100-billion-parameter models. One piece of that library, called ZeRO, is a new parallelized optimizer that greatly reduces the resources needed for model and data parallelism while massively increasing the number of parameters that can be trained. Researchers have used these breakthroughs to create Turing Natural Language Generation (Turing-NLG), the largest publicly known language model at 17 billion parameters, which you can learn more about in this accompanying blog post. The Zero Redundancy Optimizer (abbreviated ZeRO) is a novel memory optimization technology for large-scale distributed deep learning.
Massive deep learning language models (LM), such as BERT and GPT-2, with billions of parameters learned from essentially all the text published on the internet, have improved the state of the art on nearly every downstream natural language processing (NLP) task, including question answering, conversational agents, and document understanding among others. Better natural language generation can be transformational for a variety of applications, such as assisting authors with composing their content, saving one time by summarizing a long piece of text, or improving customer experience with digital assistants. Following the trend that larger natural language models lead to better results, Microsoft is introducing Turing Natural Language Generation (T-NLG), the largest model ever published at 17 billion parameters, which outperforms the state of the art on a variety of language modeling benchmarks and also excels when applied to numerous practical tasks, including summarization and question answering. This work would not be possible without breakthroughs produced by the DeepSpeed library (compatible with PyTorch) and ZeRO optimizer, which can be explored more in this accompanying blog post. We are releasing a private demo of T-NLG, including its freeform generation, question answering, and summarization capabilities, to a small set of users within the academic community for initial testing and feedback.
It has never been easier to measure and monitor business operations -- the amount of data available to organizations is staggering. Access to insight provides businesses with a clear competitive advantage, but many enterprises struggle to make sense of the seemingly endless reams of data at their disposal. To overcome hurdles with data literacy, smart businesses have embraced various business intelligence (BI) solutions to collect, aggregate, translate and present business information. An invaluable asset for enterprises worldwide, BI dashboards are data visualization tools that display the status of business analytics metrics, key performance indicators (KPIs) and other important data points on a single screen. To underscore the widespread adoption of BI, note that the global business intelligence market is projected to reach USD $147.19 billion by 2025.
Recent progress in natural language generation has raised dual-use concerns. While applications like summarization and translation are positive, the underlying technology also might enable adversaries to generate neural fake news: targeted propaganda that closely mimics the style of real news. Modern computer security relies on careful threat modeling: identifying potential threats and vulnerabilities from an adversary's point of view, and exploring potential mitigations to these threats. Likewise, developing robust defenses against neural fake news requires us first to carefully investigate and characterize the risks of these models. We thus present a model for controllable text generation called Grover.
The advances you hear about most when it comes to AI are the technologies once discussed in science fiction novels: nanobots that root disease out of the body; cars that navigate relentless traffic better than a human driver; scanners that detect potential skin cancer as well as a dermatologist. What you don't hear about nearly as much, however, are the myriad ways that technologies powered by AI are transforming routine processes and common frustrations, and helping businesses handle mundane tasks so employees can focus on more interesting work. There is perhaps no better example of this than content generation software, powered by AI and Natural Language Processing (NLP). Content generation software is incredibly intuitive and easy to use: most companies can start generating content like BSS reports, e-commerce product descriptions, news content like weather reports, and more within a manner of minutes by inputting basic information via an Excel spreadsheet. The natural language generation (NLG) market includes my company, AX Semantics, as well as other leaders like Arria, Narrative Science, Yseop and Automated Insights.
"Arria Is A Perfect Fit For You" "Arria is a Leader with robust writing automation and analytics capabilities. We found Arria's writing automation capabilities to be the most comprehensive among all participants. Its founders have written definitive books on NLG and hold more than a dozen patents on the technology… Arria is best for companies that need a "space shuttle" for writing automation. If your writing automation and NLG for analytics call for the top-of-the-line capabilities with a comprehensive IDE, and a myriad of rules and templates, Arria is a perfect fit for you.
Rome wasn't built in a day. It has taken years for computers to exhibit the level of intelligence they do today and be able to produce text that sounds and read human-like. It's time to appreciate this revolutionary journey. In 2007, The first step was taken by Robbie Allen, who was a veteran engineer at Cisco. He created an online college basketball website that automatically published game reviews, real-time updates, recaps, and incidents of injury.
AX Semantics an artificial intelligence (AI)-powered, natural language generation (NLG) company said it could create AI-produced content in more than 110 languages. The Stuttgart-based company which launched in the US today, 12 December 2019 already works with hundreds of customers, including several Fortune 500 companies such as Deloitte for BSS reporting, Porsche and Nestlé. The demand for digital content continues to rise. And technology like AI is considered to be instrumental in helping companies keep pace. The global NLG market size is expected to reach $1,150.9
Natural Language Generation (NLG) is a well studied subject among the NLP community. With the rise of deep learning methods, NLG has become better and better. Recently, OpenAI has pushed the limits, with the release of GPT-2 -- a Transformers based model that predicts the next token at each time space. Nowadays it's quite easy to use these models -- you don't need to implement the code yourself, or train the models using expensive resources. HuggingFace, for instance, has released an API that eases the access to the pretrained GPT-2 OpenAI has published.