The adoption of Deep Neural Networks (DNNs) has greatly benefited Natural Language Processing (NLP) during the past decade. However, the demands of long document analysis are quite different from those of shorter texts, while the ever increasing size of documents uploaded on-line renders automated understanding of lengthy texts a critical issue. Relevant applications include automated Web mining, legal document review, medical records analysis, financial reports analysis, contract management, environmental impact assessment, news aggregation, etc. Despite the relatively recent development of efficient algorithms for analyzing long documents, practical tools in this field are currently flourishing. This article serves as an entry point into this dynamic domain and aims to achieve two objectives. Firstly, it provides an overview of the relevant neural building blocks, serving as a concise tutorial for the field. Secondly, it offers a brief examination of the current state-of-the-art in long document NLP, with a primary focus on two key tasks: document classification and document summarization. Sentiment analysis for long texts is also covered, since it is typically treated as a particular case of document classification. Consequently, this article presents an introductory exploration of document-level analysis, addressing the primary challenges, concerns, and existing solutions. Finally, the article presents publicly available annotated datasets that can facilitate further research in this area.

In this paper, we show that representations capturing syllabic units emerge when training a self-supervised speech model with a visually-grounded training objective. We demonstrate that a nearly identical model architecture (HuBERT) trained with a masked language modeling loss does not exhibit this same ability, suggesting that the visual grounding objective is responsible for the emergence of this phenomenon. We propose the use of a minimum cut algorithm to automatically predict syllable boundaries in speech, followed by a 2-stage clustering method to group identical syllables together. We show that our model not only outperforms a state-of-the-art syllabic segmentation method on the language it was trained on (English), but also generalizes in a zero-shot fashion to Estonian. Finally, we show that the same model is capable of zero-shot generalization for a word segmentation task on 4 other languages from the Zerospeech Challenge, in some cases beating the previous state-of-the-art.

Menu system design is a challenging task involving many design options and various human factors. For example, one crucial factor that designers need to consider is the semantic and systematic relation of menu commands. However, capturing these relations can be challenging due to limited available resources. With the advancement of neural language models, large language models can utilize their vast pre-existing knowledge in designing and refining menu systems. In this paper, we propose MenuCraft, an AI-assisted designer for menu design that enables collaboration between the designer and a dialogue system to design menus. MenuCraft offers an interactive language-based menu design tool that simplifies the menu design process and enables easy customization of design options. MenuCraft supports a variety of interactions through dialog that allows performing zero/few-shot learning.

The increasing fluency and widespread usage of large language models (LLMs) highlight the desirability of corresponding tools aiding detection of LLM-generated text. In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection. Specifically, we demonstrate that text sampled from an LLM tends to occupy negative curvature regions of the model's log probability function. Leveraging this observation, we then define a new curvature-based criterion for judging if a passage is generated from a given LLM. This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g., T5). We find DetectGPT is more discriminative than existing zero-shot methods for model sample detection, notably improving detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT. See https://ericmitchell.ai/detectgpt for code, data, and other project information.

Meta's global policy head, Sir Nick Clegg, has backed calls for an international agency to guide the regulation of artificial intelligence if it becomes autonomous, saying governments globally should avoid "fragmented" laws around the technology. But Clegg downplayed suggestions of payment for content creators like artists or news outlets whose work is scraped to teach chatbots and generative AI, suggesting such information would be available under fair use arrangements. "Creators who lean in to using this technology, rather than trying to block it or slow it down or prevent it from drawing on their own creative output, will in the long run be better placed than those who set their face against this technology," Clegg told Guardian Australia. "We believe we're using [data] entirely in line with existing law. A lot of this data is being transformed in the way it's being deployed by these generative AI models. In the long run, I can't see how you put the genie back in the bottle, given that these models do use publicly available information across the internet, and not unreasonably so."

In 2017, researchers at the British AI company DeepMind (now Google DeepMind) published an extraordinary paper describing how their new algorithm, AlphaZero, had taught itself to play a number of games to superhuman standards without any instruction. The machine could, they wrote, "achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case." Speaking afterwards at a big machine-learning conference, DeepMind's chief executive, Demis Hassabis (himself a world-class chess player), observed that the program often made moves that would seem unthinkable to a human chess player. "It doesn't play like a human," he said, "and it doesn't play like a program. It plays in a third, almost alien, way."

When OpenAI released a ChatGPT app for the iPhone in May, it promised that Android users will get theirs soon. Now, the company has announced that ChatGPT for Android is rolling out to users sometime next week. Moreover, its Google Play listing is already up, and users can pre-register to get it as soon as it becomes available. It's unclear if the app will initially only be available in the US like the iPhone app, but I was able to pre-order it from Asia. OpenAI expanded the iOS app's reach to more regions just a few days after it was released, so the Android app will most likely be accessible in other countries soon even if it does launch only in the US.

Washington – U.S. President Joe Biden evoked AI's "enormous" risk and promise Friday at a White House meeting with tech leaders who committed to guarding against everything from cyberattacks to fraud as the sector revolutionizes society. "It is astounding," Biden said, highlighting AI's "enormous, enormous promise of both risk to our society and our economy and our national security, but also incredible opportunities." Standing alongside top representatives from Amazon, Anthropic, Google, Inflection, Meta, Microsoft and OpenAI, Biden said the cutting-edge companies had made commitments to "guide responsible innovation" as AI rips ever deeper into personal and business life. This could be due to a conflict with your ad-blocking or security software. Please add japantimes.co.jp and piano.io to your list of allowed sites.

"Control it all the way you want to," reads the slogan for AI girlfriend app Eva AI. "Connect with a virtual AI partner who listens, responds, and appreciates you." A decade since Joaquin Phoenix fell in love with his AI companion Samantha, played by Scarlett Johansson in the Spike Jonze film Her, the proliferation of large language models has brought companion apps closer than ever. As chatbots like OpenAI's ChatGPT and Google's Bard get better at mimicking human conversation, it seems inevitable they would come to play a role in human relationships. And Eva AI is just one of several options on the market. Replika, the most popular app of the kind, has its own subreddit where users talk about how much they love their "rep", with some saying they had been converted after initially thinking they would never want to form a relationship with a bot.

Advancements in generative artificial intelligence (AI) and large language models (LLMs) have fueled the development of many educational technology innovations that aim to automate the often time-consuming and laborious tasks of generating and analysing textual content (e.g., generating open-ended questions and analysing student feedback survey) (Kasneci et al., 2023; Wollny et al., 2021; Leiker et al., 2023). LLMs are generative artificial intelligence models that have been trained on an extensive amount of text data, capable of generating human-like text content based on natural language inputs. Specifically, these LLMs, such as Bidirectional Encoder Representations from Transformers (BERT) (Devlin et al., 2018) and Generative Pre-trained Transformer (GPT) (Brown et al., 2020), utilise deep learning and self-attention mechanisms (Vaswani et al., 2017) to selectively attend to the different parts of input texts, depending on the focus of the current tasks, allowing the model to learn complex patterns and relationships among textual contents, such as their semantic, contextual, and syntactic relationships (Min et al., 2021; Liu et al., 2023). As several LLMs (e.g., GPT-3 and Codex) have been pre-trained on massive amounts of data across multiple disciplines, they are capable of completing natural language processing tasks with little (few-shot learning) or no additional training (zero-shot learning) (Brown et al., 2020; Wu et al., 2023). This could lower the technological barriers to LLMs-based innovations as researchers and practitioners can develop new educational technologies by fine-tuning LLMs on specific educational tasks without starting from scratch (Caines et al., 2023; Sridhar et al., 2023). The recent release of ChatGPT, an LLMs-based generative AI chatbot that requires only natural language prompts without additional model training or fine-tuning (OpenAI, 2023), has further lowered the barrier for individuals without technological background to leverage the generative powers of LLMs. Although educational research that leverages LLMs to develop technological innovations for automating educational tasks is yet to achieve its full potential (i.e., most works have focused on improving model performances (Kurdi et al., 2020; Ramesh and Sanampudi, 2022)), a growing body of literature hints at how different stakeholders could potentially benefit from such innovations.