Large Language Model
Opinion
Imagine a world where autonomous weapons roam the streets, decisions about your life are made by AI systems that perpetuate societal biases and hackers use AI to launch devastating cyberattacks. This dystopian future may sound like science fiction, but the truth is that without proper regulations for the development and deployment of Artificial Intelligence (AI), it could become a reality. The rapid advancements in AI technology have made it clear that the time to act is now to ensure that AI is used in ways that are safe, ethical and beneficial for society. Failure to do so could lead to a future where the risks of AI far outweigh its benefits. I didn't write the above paragraph.
OpenAI's ChatGPT is a morally corrupting influence โข The Register
OpenAI's conversational language model ChatGPT has a lot to say, but is likely to lead you astray if you ask it for moral guidance. Introduced in November, ChatGPT is the latest of several recently released AI models eliciting interest and concern about the commercial and social implications of mechanized content recombination and regurgitation. These include DALL-E, Stable Diffusion, Codex, and GPT-3. While DALL-E and Stable Diffusion have raised eyebrows, funding, and litigation by ingesting art without permission and reconstituting strangely familiar, sometimes evocative imagery on demand, ChatGPT has been answering query prompts with passable coherence. That being the standard for public discourse, pundits have been sufficiently wowed that they foresee some future iteration of an AI-informed chatbot challenging the supremacy of Google Search and do all sorts of other once primarily human labor, such as writing inaccurate financial news or increasing the supply of insecure code.
Can AI-written content help your business? Here's what you need to know
Artificial intelligence โ if you've peeked at LinkedIn over the end of year break, it's likely you've witnessed a flurry of interest in this next big cybernetic advance. As a copywriter, I got about five panicked messages wondering if I needed to change careers in a hurry. The buzz is all about the public research preview of the Microsoft-backed OpenAI ChatGPT program, a model that interacts with language queries in a natural and readable way. According to the developers, ChatGPT can "[ask] follow-up questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests." These assistants will retrieve secondary sources of information for you โ ChatGPT will give you detailed answers all by itself. People have been amazed and terrified at the speed and fluency of its responses to ended questions such as "Who was the best test cricketer of all time?" or "What was the cause of the American Civil War?" The AI was so adept at providing answers, New York City school officials started blocking ChatGPT, fearing it would lead to rampant cheating.
Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing
Bhat, Advait, Agashe, Saaket, Mohile, Niharika, Oberoi, Parth, Jangir, Ravi, Joshi, Anirudha
Writing with next-phrase suggestions powered by large language models is becoming more pervasive by the day. However, research to understand writers' interaction and decision-making processes while engaging with such systems is still emerging. We conducted a qualitative study to shed light on writers' cognitive processes while writing with next-phrase suggestion systems. To do so, we recruited 14 amateur writers to write two reviews each, one without suggestions and one with suggestions. Additionally, we also positively and negatively biased the suggestion system to get a diverse range of instances where writers' opinions and the bias in the language model align or misalign to varying degrees. We found that writers interact with next-phrase suggestions in various complex ways: Writers abstracted and extracted multiple parts of the suggestions and incorporated them within their writing, even when they disagreed with the suggestion as a whole; along with evaluating the suggestions on various criteria. The suggestion system also had various effects on the writing process, such as altering the writer's usual writing plans, leading to higher levels of distraction etc. Based on our qualitative analysis using the cognitive process model of writing by Hayes as a lens, we propose a theoretical model of 'writer-suggestion interaction' for writing with GPT-2 (and causal language models in general) for a movie review writing task, followed by directions for future research and design.
Large language models can segment narrative events similarly to humans
Michelmann, Sebastian, Kumar, Manoj, Norman, Kenneth A., Toneva, Mariya
Humans perceive discrete events such as "restaurant visits" and "train rides" in their continuous experience. One important prerequisite for studying human event perception is the ability of researchers to quantify when one event ends and another begins. Typically, this information is derived by aggregating behavioral annotations from several observers. Here we present an alternative computational approach where event boundaries are derived using a large language model, GPT-3, instead of using human annotations. We demonstrate that GPT-3 can segment continuous narrative text into events. GPT-3-annotated events are significantly correlated with human event annotations. Furthermore, these GPT-derived annotations achieve a good approximation of the "consensus" solution (obtained by averaging across human annotations); the boundaries identified by GPT-3 are closer to the consensus, on average, than boundaries identified by individual human annotators. This finding suggests that GPT-3 provides a feasible solution for automated event annotations, and it demonstrates a further parallel between human cognition and prediction in large language models. In the future, GPT-3 may thereby help to elucidate the principles underlying human event perception.
Multitask Instruction-based Prompting for Fallacy Recognition
Alhindi, Tariq, Chakrabarty, Tuhin, Musi, Elena, Muresan, Smaranda
Fallacies are used as seemingly valid arguments to support a position and persuade the audience about its validity. Recognizing fallacies is an intrinsically difficult task both for humans and machines. Moreover, a big challenge for computational models lies in the fact that fallacies are formulated differently across the datasets with differences in the input format (e.g., question-answer pair, sentence with fallacy fragment), genre (e.g., social media, dialogue, news), as well as types and number of fallacies (from 5 to 18 types per dataset). To move towards solving the fallacy recognition task, we approach these differences across datasets as multiple tasks and show how instruction-based prompting in a multitask setup based on the T5 model improves the results against approaches built for a specific dataset such as T5, BERT or GPT-3. We show the ability of this multitask prompting approach to recognize 28 unique fallacies across domains and genres and study the effect of model size and prompt choice by analyzing the per-class (i.e., fallacy type) results. Finally, we analyze the effect of annotation quality on model performance, and the feasibility of complementing this approach with external knowledge.
Mathematics, word problems, common sense, and artificial intelligence
The paper discusses the capacities and limitations of current artificial intelligence (AI) technology to solve word problems that combine elementary knowledge with commonsense reasoning. No existing AI systems can solve these reliably. We review three approaches that have been developed, using AI natural language technology: outputting the answer directly, outputting a computer program that solves the problem, and outputting a formalized representation that can be input to an automated theorem verifier. We review some benchmarks that have been developed to evaluate these systems and some experimental studies. We discuss the limitations of the existing technology at solving these kinds of problems. We argue that it is not clear whether these kinds of limitations will be important in developing AI technology for pure mathematical research, but that they will be important in applications of mathematics, and may well be important in developing programs capable of reading and understanding mathematical content written by humans.
SMART: Self-supervised Multi-task pretrAining with contRol Transformers
Sun, Yanchao, Ma, Shuang, Madaan, Ratnesh, Bonatti, Rogerio, Huang, Furong, Kapoor, Ashish
Self-supervised pretraining has been extensively studied in language and vision domains, where a unified model can be easily adapted to various downstream tasks by pretraining representations without explicit labels. When it comes to sequential decision-making tasks, however, it is difficult to properly design such a pretraining approach that can cope with both high-dimensional perceptual information and the complexity of sequential control over long interaction horizons. The challenge becomes combinatorially more complex if we want to pretrain representations amenable to a large variety of tasks. To tackle this problem, in this work, we formulate a general pretraining-finetuning pipeline for sequential decision making, under which we propose a generic pretraining framework \textit{Self-supervised Multi-task pretrAining with contRol Transformer (SMART)}. By systematically investigating pretraining regimes, we carefully design a Control Transformer (CT) coupled with a novel control-centric pretraining objective in a self-supervised manner. SMART encourages the representation to capture the common essential information relevant to short-term control and long-term control, which is transferrable across tasks. We show by extensive experiments in DeepMind Control Suite that SMART significantly improves the learning efficiency among seen and unseen downstream tasks and domains under different learning scenarios including Imitation Learning (IL) and Reinforcement Learning (RL). Benefiting from the proposed control-centric objective, SMART is resilient to distribution shift between pretraining and finetuning, and even works well with low-quality pretraining datasets that are randomly collected.
ChatGPT's killer enterprise use case will be managing knowledge, says EY CTO
Check out all the on-demand sessions from the Intelligent Security Summit here. Right now there is no "killer" use case for using ChatGPT in the enterprise -- that is, one that will have an enormous impact on the top and the bottom line -- according to EY's global chief technology officer, Nicola Morini Bianzino. But that could soon change: The next six to 12 months will bring an explosion of experimentation, he predicted, especially once companies are able to build on top of ChatGPT using OpenAI's API. And the killer use case that emerges could be around generative AI's impact on knowledge management -- that Bianzino describes as the "dialectic of AI." "Knowledge companies tend to store knowledge in a very flat, two-dimensional way that makes it difficult to access, interact and have a dialogue with," he told VentureBeat in an interview. "We tried 20, 30, 40 years ago to build expert systems. That didn't go really well because they were too rigid. I think this technology promises to overcome a lot of issues that expert systems have."
ChatGPT may charge $42/mo for a paid tier, as Microsoft invests again
Microsoft said Monday that it is entering a "third phase" of its relationship with ChatGPT developer OpenAI through a multibillion-dollar investment. But the deal may come with a price for users, too: a $42 monthly "professional" tier subscription that sources say OpenAI is testing among early adopters. In 2019, Microsoft said it would invest $1 billion into OpenAI, then a relatively unknown developer of AI technologies. On Monday, Microsoft said it would extend its partnership through a "multiyear, multibillion dollar investment to accelerate AI breakthroughs." "Microsoft will deploy OpenAI's models across our consumer and enterprise products and introduce new categories of digital experiences built on OpenAI's technology," the company said.