Since the invention of computers, communication through natural language (actual human language) has been a dream technology. However, natural language is extremely difficult to mathematically formulate, making it difficult to realize as an algorithm without considering programming. While there have been numerous technological developments, one cannot say that any results allowing free utilization have been achieved thus far. In the case of language learning in humans, for instance when learning one's mother tongue or foreign language, one must admit that this process is similar to the adage "practice makes perfect" in principle, even though the learning method is significant up to a point. Deep learning has played a central role in contemporary AI technology in recent years. When applied to natural language processing (NLP), this produced unprecedented results. Achievements exceeding the initial predictions have been reported from the results of learning vast amounts of textual data using deep learning. For instance, four arithmetic operations could be performed without explicit learning, thereby enabling the explanation of complex images and the generation of images from corresponding explanatory texts. It is an accurate example of the learner embodying the concept of "practice makes perfect" by using vast amounts of textual data. This report provides a technological explanation of how cutting-edge NLP has made it possible to realize the "practice makes perfect" principle. Additionally, examples of how this can be applied to business are provided. We reported in June 2022 in Japanese on the NLP movement from late 2021 to early 2022. We would like to summarize this as a memorandum since this is just the initial movement leading to the current large language models (LLMs).

Translation of natural language into syntactically correct commands for data visualization is an important application of natural language models and could be leveraged to many different tasks. A closely related effort is the task of translating natural languages into SQL queries, which in turn could be translated into visualization with additional information from the natural language query supplied[1]. Contributing to the progress in this area of research, we built natural language translation models to construct simplified versions of data and visualization queries in a language called Vega Zero first proposed by Luo, Yuyu, et al[2]. In this paper, we explore the design and performance of these sequence to sequence transformer based machine learning model architectures using large language models such as BERT as encoders to predict visualization commands from natural language queries, as well as apply available T5 sequence to sequence models to the problem for comparison.

Neural network language models can serve as computational hypotheses about how humans process language. We compared the model-human consistency of diverse language models using a novel experimental approach: controversial sentence pairs. For each controversial sentence pair, two language models disagree about which sentence is more likely to occur in natural text. Considering nine language models (including n-gram, recurrent neural networks, and transformer models), we created hundreds of such controversial sentence pairs by either selecting sentences from a corpus or synthetically optimizing sentence pairs to be highly controversial. Human subjects then provided judgments indicating for each pair which of the two sentences is more likely. Controversial sentence pairs proved highly effective at revealing model failures and identifying models that aligned most closely with human judgments. The most human-consistent model tested was GPT-2, although experiments also revealed significant shortcomings of its alignment with human perception.

Recently, large pretrained language models have achieved compelling performance on commonsense benchmarks. Nevertheless, it is unclear what commonsense knowledge the models learn and whether they solely exploit spurious patterns. Feature attributions are popular explainability techniques that identify important input concepts for model outputs. However, commonsense knowledge tends to be implicit and rarely explicitly presented in inputs. These methods cannot infer models' implicit reasoning over mentioned concepts. We present CommonsenseVIS, a visual explanatory system that utilizes external commonsense knowledge bases to contextualize model behavior for commonsense question-answering. Specifically, we extract relevant commonsense knowledge in inputs as references to align model behavior with human knowledge. Our system features multi-level visualization and interactive model probing and editing for different concepts and their underlying relations. Through a user study, we show that CommonsenseVIS helps NLP experts conduct a systematic and scalable visual analysis of models' relational reasoning over concepts in different situations.

For search engines and enterprise writing assistance, the top contender is OpenAI, which yesterday announced the latest model of its language model, GPT-4. GPT-4 is now available on ChatGPT Plus and as an API, for which developers can join a waitlist. It's throwing a new weapon into the AI war, in which organizations jostle to provide the best, most flexible writing AI. OpenAI demonstrated the new natural language model with a challenge: "Explain the plot of Cinderella in a sentence where each word has to begin with the next letter in the alphabet from A to Z, without repeating any letters." It's a neat riddle to show the AI can perform some reasoning along with producing straightforward text, but what does it do in the office?

Do you want to use a natural language model comparable to GPT-3 for free? Do you want to try a natural language model published by Google? In such cases, I recommend Flan-T5. This article describes Flan-T5, a great language model developed by Google.

Natural language processing (NLP) is a subfield of artificial intelligence that deals with the interaction between computers and human (natural) language. One important aspect of NLP is natural language understanding, which involves analyzing and interpreting text or speech input in a way that a computer can understand and respond to it appropriately. A natural language model is a type of machine learning model that is trained to process and analyze human language data. These models can be used for a variety of tasks, such as language translation, text summarization, and question answering. One common type of natural language model is the language model, which is used to predict the likelihood of a sequence of words occurring in a given language.

What would it take for a natural language model to understand a novel, such as The Lord of the Rings? Among other things, such a model must be able to: (a) identify and record new characters (entities) and their attributes as they are introduced in the text, and (b) identify subsequent references to the characters previously introduced and update their attributes. This problem of entity tracking is essential for language understanding, and thus, useful for a wide array of downstream applications in NLP such as question-answering, summarization. In this thesis, we focus on two key problems in relation to facilitating the use of entity tracking models: (i) scaling entity tracking models to long documents, such as a novel, and (ii) integrating entity tracking into language models. Applying language technologies to long documents has garnered interest recently, but computational constraints are a significant bottleneck in scaling up current methods. In this thesis, we argue that computationally efficient entity tracking models can be developed by representing entities with rich, fixed-dimensional vector representations derived from pretrained language models, and by exploiting the ephemeral nature of entities. We also argue for the integration of entity tracking into language models as it will allow for: (i) wider application given the current ubiquitous use of pretrained language models in NLP applications, and (ii) easier adoption since it is much easier to swap in a new pretrained language model than to integrate a separate standalone entity tracking model.

Google is announcing news at breakneck pace at its I/O developer conference today, and as usual it's flexing its machine-learning smarts. In addition to unveiling its new LaMDA 2 conversational AI model, the company also showed off a new app called AI Test Kitchen. The app offers three demos that showcase what LaMDA 2 can do. The first is a simple brainstorm tool that asks the app to help you imagine if you were in various scenarios. During the keynote demo, Google entered "I'm at the deepest part of the ocean" as a response to the app's prompt of "Imagine if."

This is an Op-ed about the future of the internet and, while speculative, it's an example and an attempt to demonstrate how Artificial Intelligence at scale in a human would or could have disastrous impacts without AI regulation and AI ethics to protect us. GPT-3 stands for Generative Pre-trained Transformer. As you likely already know GPT-3 is an autoregressive language model that uses deep learning to produce human-like text. It is the third-generation language prediction model in the GPT-n series (and the successor to GPT-2) created by Microsoft-funded OpenAI (that was supposed to be a not for profit firm). In 2021 we've had a NLP-explosion year in terms of Artificial Intelligence activity.