Media
Unsupervised Extraction of Dialogue Policies from Conversations
Sreedhar, Makesh Narsimhan, Rebedea, Traian, Parisien, Christopher
Dialogue policies play a crucial role in developing task-oriented dialogue systems, yet their development and maintenance are challenging and typically require substantial effort from experts in dialogue modeling. While in many situations, large amounts of conversational data are available for the task at hand, people lack an effective solution able to extract dialogue policies from this data. In this paper, we address this gap by first illustrating how Large Language Models (LLMs) can be instrumental in extracting dialogue policies from datasets, through the conversion of conversations into a unified intermediate representation consisting of canonical forms. We then propose a novel method for generating dialogue policies utilizing a controllable and interpretable graph-based methodology. By combining canonical forms across conversations into a flow network, we find that running graph traversal algorithms helps in extracting dialogue flows. These flows are a better representation of the underlying interactions than flows extracted by prompting LLMs. Our technique focuses on giving conversation designers greater control, offering a productivity tool to improve the process of developing dialogue policies.
Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods
Fraser, Kathleen C., Dawkins, Hillary, Kiritchenko, Svetlana
Large language models (LLMs) have advanced to a point that even humans have difficulty discerning whether a text was generated by another human, or by a computer. However, knowing whether a text was produced by human or artificial intelligence (AI) is important to determining its trustworthiness, and has applications in many domains including detecting fraud and academic dishonesty, as well as combating the spread of misinformation and political propaganda. The task of AI-generated text (AIGT) detection is therefore both very challenging, and highly critical. In this survey, we summarize state-of-the art approaches to AIGT detection, including watermarking, statistical and stylistic analysis, and machine learning classification. We also provide information about existing datasets for this task. Synthesizing the research findings, we aim to provide insight into the salient factors that combine to determine how "detectable" AIGT text is under different scenarios, and to make practical recommendations for future work towards this significant technical and societal challenge.
Detecting Synthetic Lyrics with Few-Shot Inference
Labrak, Yanis, Meseguer-Brocal, Gabriel, Epure, Elena V.
In recent years, generated content in music has gained significant popularity, with large language models being effectively utilized to produce human-like lyrics in various styles, themes, and linguistic structures. This technological advancement supports artists in their creative processes but also raises issues of authorship infringement, consumer satisfaction and content spamming. To address these challenges, methods for detecting generated lyrics are necessary. However, existing works have not yet focused on this specific modality or on creative text in general regarding machine-generated content detection methods and datasets. In response, we have curated the first dataset of high-quality synthetic lyrics and conducted a comprehensive quantitative evaluation of various few-shot content detection approaches, testing their generalization capabilities and complementing this with a human evaluation. Our best few-shot detector, based on LLM2Vec, surpasses stylistic and statistical methods, which are shown competitive in other domains at distinguishing human-written from machine-generated content. It also shows good generalization capabilities to new artists and models, and effectively detects post-generation paraphrasing. This study emphasizes the need for further research on creative content detection, particularly in terms of generalization and scalability with larger song catalogs. All datasets, pre-processing scripts, and code are available publicly on GitHub and Hugging Face under the Apache 2.0 license.
Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data
Marchal, Nahema, Xu, Rachel, Elasmar, Rasmi, Gabriel, Iason, Goldberg, Beth, Isaac, William
Generative, multimodal artificial intelligence (GenAI) offers transformative potential across industries, but its misuse poses significant risks. Prior research has shed light on the potential of advanced AI systems to be exploited for malicious purposes. However, we still lack a concrete understanding of how GenAI models are specifically exploited or abused in practice, including the tactics employed to inflict harm. In this paper, we present a taxonomy of GenAI misuse tactics, informed by existing academic literature and a qualitative analysis of approximately 200 observed incidents of misuse reported between January 2023 and March 2024. Through this analysis, we illuminate key and novel patterns in misuse during this time period, including potential motivations, strategies, and how attackers leverage and abuse system capabilities across modalities (e.g. image, text, audio, video) in the wild.
I Watched TMZ's Bizarre Bennifer Documentary. It's, Uh, Saying a Lot.
For weeks now, we've been waiting for the other Timberland-inspired Manolo to drop. About a month ago, news outlets first reported that Jennifer Lopez and Ben Affleck's marriage was in trouble--that the two, in fact, had not been seen together in 47 days. No official announcement of a separation or divorce has followed, but the updates we have gotten--he moved out; they're only sometimes wearing their rings--support the narrative that bad news is on its way. I know that it's a sign of my celebrity brain rot that I can't help but see grim parallels between the fragmentary updates on the state of their marriage and the trickle of information about the state of Jimmy Carter's health. When a newsworthy event is likely to happen--for example, the imminent death of a 99-year-old former president--journalistic outlets have a practice of prewriting the news, to have it ready to go when the time comes.
Liberal media outlets 'running cover' for Biden by calling viral clips 'cheap fakes,' critics say
There has been an avalanche of coverage recently from liberal news outlets on so-called "cheap fakes," the term being used by both the media and the White House to describe viral clips of President Biden that critics say show signs of his cognitive decline. Biden's age has been the subject of intense scrutiny in recent days, with the president facing accusations of freezing and wandering off at various events showcased online by Republicans. One prominent example was footage showing Biden stepping away from other world leaders at the G-7 Summit to give a thumbs up to parachutists off-camera, prompting Italian Prime Minister Giorgia Meloni to corral him back to the group for a photo-op. "Selective editing of video and putting spin on interpretations of events has been going on in American politics for a long time," DePauw University journalism professor Jeffrey McCall said. "What has been surprising, however, is how eager the establishment media have been to parrot the White House spin, trying to dismiss concerns about Biden's capabilities as just cheap fake editing and razzle-dazzle."
GREG GUTFELD: 'Cheap fakes' is just another hoax by the media
'Gutfeld!' panelists react to the media claiming President Biden videos are deepfakes. Welcome to the hoax hoax. Where in the run-up to the election our media circles the wagons around a dithering Joe Biden to convince us that what we see with our own eyes isn't real. This year's hoax hoax even comes with a whole new buzz term โ "cheap fake." A word that allows the left to confuse people with deep fake without actually lying.
Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics
Lee, Seungbeen, Lim, Seungwon, Han, Seungju, Oh, Giyeong, Chae, Hyungjoo, Chung, Jiwan, Kim, Minju, Kwak, Beong-woo, Lee, Yeonsoo, Lee, Dongha, Yeo, Jinyoung, Yu, Youngjae
The idea of personality in descriptive psychology, traditionally defined through observable behavior, has now been extended to Large Language Models (LLMs) to better understand their behavior. This raises a question: do LLMs exhibit distinct and consistent personality traits, similar to humans? Existing self-assessment personality tests, while applicable, lack the necessary validity and reliability for precise personality measurements. To address this, we introduce TRAIT, a new tool consisting of 8K multi-choice questions designed to assess the personality of LLMs with validity and reliability. TRAIT is built on the psychometrically validated human questionnaire, Big Five Inventory (BFI) and Short Dark Triad (SD-3), enhanced with the ATOMIC10X knowledge graph for testing personality in a variety of real scenarios. TRAIT overcomes the reliability and validity issues when measuring personality of LLM with self-assessment, showing the highest scores across three metrics: refusal rate, prompt sensitivity, and option order sensitivity. It reveals notable insights into personality of LLM: 1) LLMs exhibit distinct and consistent personality, which is highly influenced by their training data (i.e., data used for alignment tuning), and 2) current prompting techniques have limited effectiveness in eliciting certain traits, such as high psychopathy or low conscientiousness, suggesting the need for further research in this direction.
Using Multimodal Foundation Models and Clustering for Improved Style Ambiguity Loss
Teaching text-to-image models to be creative involves using style ambiguity loss, which requires a pretrained classifier. In this work, we explore a new form of the style ambiguity training objective, used to approximate creativity, that does not require training a classifier or even a labeled dataset. We then train a diffusion model to maximize style ambiguity to imbue the diffusion model with creativity and find our new methods improve upon the traditional method, based on automated metrics for human judgment, while still maintaining creativity and novelty.
Video Generation with Learned Action Prior
Sarkar, Meenakshi, Bhardwaj, Devansh, Ghose, Debasish
Stochastic video generation is particularly challenging when the camera is mounted on a moving platform, as camera motion interacts with observed image pixels, creating complex spatio-temporal dynamics and making the problem partially observable. Existing methods typically address this by focusing on raw pixel-level image reconstruction without explicitly modelling camera motion dynamics. We propose a solution by considering camera motion or action as part of the observed image state, modelling both image and action within a multi-modal learning framework. We introduce three models: Video Generation with Learning Action Prior (VG-LeAP) treats the image-action pair as an augmented state generated from a single latent stochastic process and uses variational inference to learn the image-action latent prior; Causal-LeAP, which establishes a causal relationship between action and the observed image frame at time $t$, learning an action prior conditioned on the observed image states; and RAFI, which integrates the augmented image-action state concept into flow matching with diffusion generative processes, demonstrating that this action-conditioned image generation concept can be extended to other diffusion-based models. We emphasize the importance of multi-modal training in partially observable video generation problems through detailed empirical studies on our new video action dataset, RoAM.