Media
Evaluating Large Language Models as Generative User Simulators for Conversational Recommendation
Yoon, Se-eun, He, Zhankui, Echterhoff, Jessica Maria, McAuley, Julian
Synthetic users are cost-effective proxies for real users in the evaluation of conversational recommender systems. Large language models show promise in simulating human-like behavior, raising the question of their ability to represent a diverse population of users. We introduce a new protocol to measure the degree to which language models can accurately emulate human behavior in conversational recommendation. This protocol is comprised of five tasks, each designed to evaluate a key property that a synthetic user should exhibit: choosing which items to talk about, expressing binary preferences, expressing open-ended preferences, requesting recommendations, and giving feedback. Through evaluation of baseline simulators, we demonstrate these tasks effectively reveal deviations of language models from human behavior, and offer insights on how to reduce the deviations with model selection and prompting strategies.
The Solution of the Zodiac Killer's 340-Character Cipher
Oranchak, David, Blake, Sam, Van Eycke, Jarl
The case of the Zodiac Killer is one of the most widely known unsolved serial killer cases in history. The unidentified killer murdered five known victims and terrorized the state of California. He also communicated extensively with the press and law enforcement. Besides his murders, Zodiac was known for his use of ciphers. The first Zodiac cipher was solved within a week of its publication, while the second cipher was solved by the authors after 51 years, when it was discovered to be a transposition and homophonic substitution cipher with unusual qualities. In this paper, we detail the historical significance of this cipher and the numerous efforts which culminated in its solution.
Grammatical vs Spelling Error Correction: An Investigation into the Responsiveness of Transformer-based Language Models using BART and MarianMT
Raju, Rohit, Pati, Peeta Basa, Gandheesh, SA, Sannala, Gayatri Sanjana, KS, Suriya
ORC ID: 0000-0003-2376-4591 Abstract Text continues to remain a relevant form of representation for information. Text documents are created either in digital native platforms or through conversion of other media files such as images and speech. While the digital native text is invariably obtained through physical or virtual keyboards, technologies such as OCR & speech recognition are utilized to transform the images and speech signals to text content. All these variety of mechanisms of text generation also introduce error into the captured text. This project aims at analyzing different kinds of errors that occurs in text documents. The work employs two of the advanced deep neural network based language models, namely, BART and MarianMT, for rectifying the anomalies present in text. Transfer learning of these models with available dataset is performed to finetune their capacity for error correction. A comparative study is conducted to investigate the effectiveness of these models in handling each of the defined error categories. It is observed that while both the models are able to bring down the erroneous sentences by 20+%, BART is able to handle spelling errors far better (24.6%) than grammatical errors (8.8%). I. Introduction Text is a natural representation of all the existing languages in the world. Texts help one express and communicate with others. Handwritten texts have been part of the history for ages, while digital texts have evolved to keep up with the rapidly growing technology in day to day lives. It is due to texts that one can extend from their knowledge and memory beyond their body into the environment around [1]. Text is available in various forms, from handwritten manuscripts to This is a pre-print version of the paper. Texts can be utilized for personal reasons such as diary entry, blog, etc., as well as for professional purposes like advertising, surveying, etc. Right from the newspaper one reads in the morning to the social media scrolling before going to bed, people are surrounded by text. It is human nature to categorize any kind of data they receive. As there is so much text available around, it is obvious that humans tend to inspect and review the text they require. It is the process of scanning the textual data in order to derive some meaning and store information. Most businesses rely on text analysis to extract valuable insights from various raw sources. The feedback received from these sources such as emails, chat messages, social media posts, comments & statements and survey responses help them in their decision-making strategies.
Outcome-Constrained Large Language Models for Countering Hate Speech
Hong, Lingzi, Luo, Pengcheng, Blanco, Eduardo, Song, Xiaoying
Counterspeech that challenges or responds to hate speech has been seen as an alternative to mitigate the negative impact of hate speech and foster productive online communications. Research endeavors have been directed to using language models for the automatic generation of counterspeech to assist efforts in combating online hate. Existing research focuses on the generation of counterspeech with certain linguistic attributes, such as being polite, informative, and intent-driven. However, it remains unclear what impact the counterspeech might have in an online environment. We first explore methods that utilize large language models (LLM) to generate counterspeech constrained by potential conversation outcomes. We build two conversation outcome classifiers that predict the incivility level and the hater reentry behavior following replies to hate with Reddit data, then propose four methods to incorporate the desired outcomes, i.e., low conversation incivility and non-hateful hater reentry, into the text generation process, including Prompt with Instructions, Prompt and Select, LLM finetune, and LLM transformer reinforcement learning (TRL). Evaluation results show effective strategies to generate outcome-constrained counterspeech and the linguistic characteristics of texts generated by different methods.
NSINA: A News Corpus for Sinhala
Hettiarachchi, Hansi, Premasiri, Damith, Uyangodage, Lasitha, Ranasinghe, Tharindu
The introduction of large language models (LLMs) has advanced natural language processing (NLP), but their effectiveness is largely dependent on pre-training resources. This is especially evident in low-resource languages, such as Sinhala, which face two primary challenges: the lack of substantial training data and limited benchmarking datasets. In response, this study introduces NSina, a comprehensive news corpus of over 500,000 articles from popular Sinhala news websites, along with three NLP tasks: news media identification, news category prediction, and news headline generation. The release of NSina aims to provide a solution to challenges in adapting LLMs to Sinhala, offering valuable resources and benchmarks for improving NLP in the Sinhala language. NSina is the largest news corpus for Sinhala, available up to date.
Exploring ChatGPT and its Impact on Society
Artificial intelligence has been around for a while, but suddenly it has received more attention than ever before. Thanks to innovations from companies like Google, Microsoft, Meta, and other major brands in technology. OpenAI, though, has triggered the button with its ground-breaking invention ChatGPT. ChatGPT is a Large Language Model (LLM) based on Transformer architecture that has the ability to generate human-like responses in a conversational context. It uses deep learning algorithms to generate natural language responses to input text. Its large number of parameters, contextual generation, and open-domain training make it a versatile and effective tool for a wide range of applications, from chatbots to customer service to language translation. It has the potential to revolutionize various industries and transform the way we interact with technology. However, the use of ChatGPT has also raised several concerns, including ethical, social, and employment challenges, which must be carefully considered to ensure the responsible use of this technology. The article provides an overview of ChatGPT, delving into its architecture and training process. It highlights the potential impacts of ChatGPT on the society. In this paper, we suggest some approaches involving technology, regulation, education, and ethics in an effort to maximize ChatGPT's benefits while minimizing its negative impacts. This study is expected to contribute to a greater understanding of ChatGPT and aid in predicting the potential changes it may bring about.
Prompting the E-Brushes: Users as Authors in Generative AI
Since its introduction in 2022, Generative AI has significantly impacted the art world, from winning state art fairs to creating complex videos from simple prompts. Amid this renaissance, a pivotal issue emerges: should users of Generative AI be recognized as authors eligible for copyright protection? The Copyright Office, in its March 2023 Guidance, argues against this notion. By comparing the prompts to clients' instructions for commissioned art, the Office denies users authorship due to their limited role in the creative process. This Article challenges this viewpoint and advocates for the recognition of Generative AI users who incorporate these tools into their creative endeavors. It argues that the current policy fails to consider the intricate and dynamic interaction between Generative AI users and the models, where users actively influence the output through a process of adjustment, refinement, selection, and arrangement. Rather than dismissing the contributions generated by AI, this Article suggests a simplified and streamlined registration process that acknowledges the role of AI in creation. This approach not only aligns with the constitutional goal of promoting the progress of science and useful arts but also encourages public engagement in the creative process, which contributes to the pool of training data for AI. Moreover, it advocates for a flexible framework that evolves alongside technological advancements while ensuring safety and public interest. In conclusion, by examining text-to-image generators and addressing misconceptions about Generative AI and user interaction, this Article calls for a regulatory framework that adapts to technological developments and safeguards public interests
Modeling Analog Dynamic Range Compressors using Deep Learning and State-space Models
Yin, Hanzhi, Cheng, Gang, Steinmetz, Christian J., Yuan, Ruibin, Stern, Richard M., Dannenberg, Roger B.
We describe a novel approach for developing realistic digital models of dynamic range compressors for digital audio production by analyzing their analog prototypes. While realistic digital dynamic compressors are potentially useful for many applications, the design process is challenging because the compressors operate nonlinearly over long time scales. Our approach is based on the structured state space sequence model (S4), as implementing the state-space model (SSM) has proven to be efficient at learning long-range dependencies and is promising for modeling dynamic range compressors. We present in this paper a deep learning model with S4 layers to model the Teletronix LA-2A analog dynamic range compressor. The model is causal, executes efficiently in real time, and achieves roughly the same quality as previous deep-learning models but with fewer parameters.
A Transfer Attack to Image Watermarks
Hu, Yuepeng, Jiang, Zhengyuan, Guo, Moyang, Gong, Neil
Generative AI (GenAI) can synthesize extremely realistic-looking images, posing growing challenges to information authenticity on the Internet. Watermarking [1-7] was suggested as a key technology to distinguish AI-generated and non-AI-generated content in the Executive Order on AI security issued by the White House in October 2023. In watermarkbased detection, a watermark is embedded into an AI-generated image before releasing it; and an image is detected as AI-generated if the same watermark can be decoded from it. Watermarking AI-generated images has been widely deployed in industry. For instance, Google's SynthID watermarks images generated by Imagen [8]; OpenAI embeds a watermark into images generated by DALL-E [9]; and Stable Diffusion enables users to embed a watermark into the generated images [10]. An attacker can use evasion attacks [11] to remove the watermark in a watermarked image to evade detection. Specifically, an evasion attack strategically adds a perturbation into a watermarked image such that the target watermark-based detector falsely detects the perturbed image as non-AI-generated. The literature has well understood the robustness of watermark-based detector against evasion attacks in the white-box setting (i.e., the attacker has access to the target watermarking model) and black-box setting (i.e., the attacker has access to the detection API) [11]. Specifically, in the white-box setting, an attacker can find a small perturbation for a given watermarked image such that the perturbed image evades detection while maintaining the image's visual quality; and in the
A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence
Zhao, Penghai, Zhang, Xin, Cheng, Ming-Ming, Yang, Jian, Li, Xiang
By consolidating scattered knowledge, the literature review provides a comprehensive understanding of the investigated topic. However, reading, conducting, or peer-reviewing review papers generally demands a significant investment of time and effort from researchers. To improve efficiency, this paper aims to provide a thorough review of reviews in the PAMI field from diverse perspectives. First, this paper proposes several article-level, field-normalized, and large language model-empowered bibliometric indicators to evaluate reviews. To facilitate this, a meta-data database dubbed RiPAMI, and a topic dataset are constructed. Second, based on these indicators, the study presents comparative analyses of representative reviews, unveiling the characteristics of publications across various fields, periods, and journals. The newly emerging AI-generated literature reviews are also appraised, and the observed differences suggest that most AI-generated reviews still lag behind human-authored reviews in multiple aspects. Third, we briefly provide a subjective evaluation of representative PAMI reviews and introduce a paper structure-based typology of literature reviews. This typology may improve the clarity and effectiveness for scholars in reading and writing reviews, while also serving as a guide for AI systems in generating well-organized reviews. Finally, this work offers insights into the current challenges of literature reviews and envisions future directions for their development.