Media
Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement
Cheng, Zihao, Zhou, Li, Jiang, Feng, Wang, Benyou, Li, Haizhou
The rapid development of large language models (LLMs), like ChatGPT, has resulted in the widespread presence of LLM-generated content on social media platforms, raising concerns about misinformation, data biases, and privacy violations, which can undermine trust in online discourse. While detecting LLM-generated content is crucial for mitigating these risks, current methods often focus on binary classification, failing to address the complexities of real-world scenarios like human-AI collaboration. To move beyond binary classification and address these challenges, we propose a new paradigm for detecting LLM-generated content. This approach introduces two novel tasks: LLM Role Recognition (LLM-RR), a multi-class classification task that identifies specific roles of LLM in content generation, and LLM Influence Measurement (LLM-IM), a regression task that quantifies the extent of LLM involvement in content creation. To support these tasks, we propose LLMDetect, a benchmark designed to evaluate detectors' performance on these new tasks. LLMDetect includes the Hybrid News Detection Corpus (HNDC) for training detectors, as well as DetectEval, a comprehensive evaluation suite that considers five distinct cross-context variations and multi-intensity variations within the same LLM role. This allows for a thorough assessment of detectors' generalization and robustness across diverse contexts. Our empirical validation of 10 baseline detection methods demonstrates that fine-tuned PLM-based models consistently outperform others on both tasks, while advanced LLMs face challenges in accurately detecting their own generated content. Our experimental results and analysis offer insights for developing more effective detection models for LLM-generated content. This research enhances the understanding of LLM-generated content and establishes a foundation for more nuanced detection methodologies.
X updates its privacy policy to allow third parties to train AI models with its data
X is updating its privacy policy with new language that allows it to provide users' data to third-party "collaborators" in order to train AI models. The new policy, which takes effect November 15, 2024, would seem to open the door to Reddit-like arrangements in which outside companies can pay to license data from X. The updated policy shared by X includes a new section titled "third-party collaborators." Depending on your settings, or if you decide to share your data, we may share or disclose your information with third parties. If you do not opt out, in some instances the recipients of the information may use it for their own independent purposes in addition to those stated in X's Privacy Policy, including, for example, to train their artificial intelligence models, whether generative or otherwise.
Filmmakers Are Worried About AI. Big Tech Wants Them to See 'What's Possible'
When Hollywood's writers and actors went on strike last year, it was, in part, because of AI. Actors didn't care for the notion that their likenesses could be used without their permission, whether by the studios that hired them that week or by someone at home with a computer in 2040. Writers didn't want to do punch-ups on potentially crummy AI scripts or have their words (or ideas) cannibalized by large language models that didn't pay them a dime. But while some Hollywood filmmakers came out of the strikes fearful of how AI might wreck their industries, others wanted to learn more. This week, many of those filmmakers gathered in a movie theater in Culver City, California, for the inaugural Culver Cup, a generative-AI film competition sponsored by FBRC.AI and Amazon Web Services.
Netflix's The Electric State trailer shows off cartoony robots and oversized VR headsets
Netflix has released the first trailer for The Electric State, a post-apocalyptic road movie from Marvel (and Community) mainstays The Russo Brothers. The adaptation of Simon Stålenhag's 2018 graphic novel is set in a retro-futuristic version of the '90s after a robot uprising. It tells the story of Michelle, an orphaned teenager (Millie Bobby Brown) who ventures across the west of the US to look for her younger brother with a smuggler (a mustachioed Chris Pratt) and a pair of robots. The movie's look draws heavily from Stålenhag's gorgeous artwork, right down to the oversized VR helmets. The robots, in particular the one accompanying Michelle, have a cartoon-inspired aesthetic that wouldn't look out of place in Fallout.
Easy ways to make calls when vision is a challenge
The upgraded Magnifier app stands out with iOS 18. Technology can be wonderfully convenient and provide a great deal of entertainment, but it can also be a great way to improve your everyday life, too. For those who experience visual challenges, a variety of apps and features can help you. That's why we love this question about apps and features that can help visually challenged loved ones: "I am not tech savvy. I need to know if there is an app that I can download on a phone, that will allow my mother to tell the app, without needing internet services, who she wants to make a phone call to? She's losing her eyesight and can no longer see the numbers on her phone. She's 88 years old and doesn't own a computer and has limited income," writes "Sheryl" of Westminster, Colorado.
All of Our Gadgets Just Keep Talking
Everybody wants to talk to their pet. Or to try to get them to listen, anyway. So it's no wonder that some startups think the way to break through the communication barrier between you and your pooch is with a nice big helping of technology. Welcome to a world with AI-enabled dog and cat collars that try to interpret a pet's needs and then share those wishes with their human. The only problem with these devices is that the pet won't actually be a part of the conversation, as the collar is just guessing at what the pet is thinking--but still doing all the talking anyway.
Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation
Shimizu, Ryotaro, Wada, Takashi, Wang, Yu, Kruse, Johannes, O'Brien, Sean, HtaungKham, Sai, Song, Linxin, Yoshikawa, Yuya, Saito, Yuki, Tsung, Fugee, Goto, Masayuki, McAuley, Julian
Recent research on explainable recommendation generally frames the task as a standard text generation problem, and evaluates models simply based on the textual similarity between the predicted and ground-truth explanations. However, this approach fails to consider one crucial aspect of the systems: whether their outputs accurately reflect the users' (post-purchase) sentiments, i.e., whether and why they would like and/or dislike the recommended items. To shed light on this issue, we introduce new datasets and evaluation methods that focus on the users' sentiments. Specifically, we construct the datasets by explicitly extracting users' positive and negative opinions from their post-purchase reviews using an LLM, and propose to evaluate systems based on whether the generated explanations 1) align well with the users' sentiments, and 2) accurately identify both positive and negative opinions of users on the target items. We benchmark several recent models on our datasets and demonstrate that achieving strong performance on existing metrics does not ensure that the generated explanations align well with the users' sentiments. Lastly, we find that existing models can provide more sentiment-aware explanations when the users' (predicted) ratings for the target items are directly fed into the models as input. We will release our code and datasets upon acceptance.
Advancing Large Language Model Attribution through Self-Improving
Huang, Lei, Feng, Xiaocheng, Ma, Weitao, Zhao, Liang, Fan, Yuchun, Zhong, Weihong, Xu, Dongliang, Yang, Qing, Liu, Hongtao, Qin, Bing
Teaching large language models (LLMs) to generate text with citations to evidence sources can mitigate hallucinations and enhance verifiability in information-seeking systems. However, improving this capability requires high-quality attribution data, which is costly and labor-intensive. Inspired by recent advances in self-improvement that enhance LLMs without manual annotation, we present START, a Self-Taught AttRibuTion framework for iteratively improving the attribution capability of LLMs. First, to prevent models from stagnating due to initially insufficient supervision signals, START leverages the model to self-construct synthetic training data for warming up. To further self-improve the model's attribution ability, START iteratively utilizes fine-grained preference supervision signals constructed from its sampled responses to encourage robust, comprehensive, and attributable generation. Experiments on three open-domain question-answering datasets, covering long-form QA and multi-step reasoning, demonstrate significant performance gains of 25.13% on average without relying on human annotations and more advanced models. Further analysis reveals that START excels in aggregating information across multiple sources.
Temporally Consistent Factuality Probing for Large Language Models
Bajpai, Ashutosh, Goyal, Aaryan, Anwer, Atif, Chakraborty, Tanmoy
The prolific use of Large Language Models (LLMs) as an alternate knowledge base requires them to be factually consistent, necessitating both correctness and consistency traits for paraphrased queries. Recently, significant attempts have been made to benchmark datasets and metrics to evaluate LLMs for these traits. However, structural simplicity (subject-relation-object) and contemporary association in their query formulation limit the broader definition of factuality and consistency. In this study, we introduce TeCFaP, a novel Temporally Consistent Factuality Probe task to expand the consistent factuality probe in the temporal dimension. To this end, we propose TEMP-COFAC, a high-quality dataset of prefix-style English query paraphrases. Subsequently, we extend the definitions of existing metrics to represent consistent factuality across temporal dimension. We experiment with a diverse set of LLMs and find most of them performing poorly on TeCFaP. Next, we propose a novel solution CoTSeLF (Consistent-Time-Sensitive Learning Framework) combining multi-task instruction tuning (MT-IT) with consistent-time-sensitive reinforcement learning (CTSRL) to improve temporally consistent factuality in LLMs. Our experiments demonstrate the efficacy of CoTSeLF over several baselines.
Human Action Anticipation: A Survey
Lai, Bolin, Toyer, Sam, Nagarajan, Tushar, Girdhar, Rohit, Zha, Shengxin, Rehg, James M., Kitani, Kris, Grauman, Kristen, Desai, Ruta, Liu, Miao
Predicting future human behavior is an increasingly popular topic in computer vision, driven by the interest in applications such as autonomous vehicles, digital assistants and human-robot interactions. The literature on behavior prediction spans various tasks, including action anticipation, activity forecasting, intent prediction, goal prediction, and so on. Our survey aims to tie together this fragmented literature, covering recent technical innovations as well as the development of new large-scale datasets for model training and evaluation. We also summarize the widely-used metrics for different tasks and provide a comprehensive performance comparison of existing approaches on eleven action anticipation datasets. This survey serves as not only a reference for contemporary methodologies in action anticipation, but also a guideline for future research direction of this evolving landscape.