Media
Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data
Zhang, Jingyu, Marone, Marc, Li, Tianjian, Van Durme, Benjamin, Khashabi, Daniel
For humans to trust the fluent generations of large language models (LLMs), they must be able to verify their correctness against trusted, external sources. Recent efforts aim to increase verifiability through citations of retrieved documents or post-hoc provenance. However, such citations are prone to mistakes that further complicate their verifiability. To address these limitations, we tackle the verifiability goal with a different philosophy: we trivialize the verification process by developing models that quote verbatim statements from trusted sources in pre-training data. We propose Quote-Tuning, which demonstrates the feasibility of aligning LLMs to leverage memorized information and quote from pre-training data. Quote-Tuning quantifies quoting against large corpora with efficient membership inference tools, and uses the amount of quotes as an implicit reward signal to construct a synthetic preference dataset for quoting, without any human annotation. Next, the target model is aligned to quote using preference optimization algorithms. Experimental results show that Quote-Tuning significantly increases the percentage of LLM generation quoted verbatim from high-quality pre-training documents by 55% to 130% relative to untuned models while maintaining response quality. Further experiments demonstrate that Quote-Tuning generalizes quoting to out-of-domain data, is applicable in different tasks, and provides additional benefits to truthfulness. Quote-Tuning not only serves as a hassle-free method to increase quoting but also opens up avenues for improving LLM trustworthiness through better verifiability.
Analyzing Musical Characteristics of National Anthems in Relation to Global Indices
Hasan, S M Rakib, Dhakal, Aakar, Siddiqua, Ms. Ayesha, Rahman, Mohammad Mominur, Islam, Md Maidul, Chowdhury, Mohammed Arfat Raihan, Swapno, S M Masfequier Rahman, Nobel, SM Nuruzzaman
Music plays a huge part in shaping peoples' psychology and behavioral patterns. This paper investigates the connection between national anthems and different global indices with computational music analysis and statistical correlation analysis. We analyze national anthem musical data to determine whether certain musical characteristics are associated with peace, happiness, suicide rate, crime rate, etc. To achieve this, we collect national anthems from 169 countries and use computational music analysis techniques to extract pitch, tempo, beat, and other pertinent audio features. We then compare these musical characteristics with data on different global indices to ascertain whether a significant correlation exists. Our findings indicate that there may be a correlation between the musical characteristics of national anthems and the indices we investigated. The implications of our findings for music psychology and policymakers interested in promoting social well-being are discussed. This paper emphasizes the potential of musical data analysis in social research and offers a novel perspective on the relationship between music and social indices. The source code and data are made open-access for reproducibility and future research endeavors. It can be accessed at http://bit.ly/na_code.
A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-off
Meisenbacher, Stephen, Nandakumar, Nihildev, Klymenko, Alexandra, Matthes, Florian
The application of Differential Privacy to Natural Language Processing techniques has emerged in relevance in recent years, with an increasing number of studies published in established NLP outlets. In particular, the adaptation of Differential Privacy for use in NLP tasks has first focused on the $\textit{word-level}$, where calibrated noise is added to word embedding vectors to achieve "noisy" representations. To this end, several implementations have appeared in the literature, each presenting an alternative method of achieving word-level Differential Privacy. Although each of these includes its own evaluation, no comparative analysis has been performed to investigate the performance of such methods relative to each other. In this work, we conduct such an analysis, comparing seven different algorithms on two NLP tasks with varying hyperparameters, including the $\textit{epsilon ($\varepsilon$)}$ parameter, or privacy budget. In addition, we provide an in-depth analysis of the results with a focus on the privacy-utility trade-off, as well as open-source our implementation code for further reproduction. As a result of our analysis, we give insight into the benefits and challenges of word-level Differential Privacy, and accordingly, we suggest concrete steps forward for the research field.
ESPN star Stephen A Smith fires back at Hillary Clinton over remarks about voters: 'Last thing you need to do'
ESPN personality and OutKick's Clay Travis talk about who the pundit will vote for in the 2024 presidential election. ESPN star Stephen A. Smith snapped back at former Democrat presidential nominee Hillary Clinton, who told voters to "get over yourselves" when asked about Americans dreading a Trump-Biden rematch this November. Clinton made her declaration in an appearance on Monday's "The Tonight Show." She suggested it wasn't a hard choice to make for voters because "one is old, and effective, and compassionate, has a heart and really cares about people. And one is old and has been charged with 91 felonies."
Stability AI's audio generator can now crank out 3 minute 'songs'
Stability AI just unveiled Stable Audio 2.0, an upgraded version of its music-generation platform. This system lets users create up to three minutes of audio via text prompt. Just imagine the fake birthday song you could make in the style of that one Rob Thomas/Santana track. The tool is free and publicly available through the company's website, so have at it. Introducing Stable Audio 2.0 – a new model capable of producing high-quality, full tracks with coherent musical structure up to three minutes long at 44.1 kHz stereo from a single prompt.
Fox News AI Newsletter: Taco Bell's 'AI-first' mentality
Brands said it is bringing an "AI-first" mentality to fast food. FAST-FOOD INNOVATIONS: Yum! Brands, the operator of KFC, Pizza Hut, Taco Bell and The Habit Burger Grill restaurants, is embracing technology with plans for "AI-powered" fast food, according to a Wall Street Journal report. Companies will be making the move toward AI to power their restaurants. CREATIVE INFRINGEMENT: Katy Perry and Miranda Lambert are just some of the more than over 200 names who have signed a letter speaking out for musicians' rights as artificial intelligence continues to expand its reach. POWER SURGE: Global energy demand is projected to surge in coming years amid the growth of artificial intelligence, which requires massive amounts of electricity.
Billie Eilish, Nicki Minaj and Katy Perry are among 200 artists calling for a ban on 'predatory' AI in the music industry - amid fears technology could replace them
Billie Eilish, Nicki Minaj and Katy Perry are among 200 high-profile artists calling for the'predatory' use of AI in the music industry to be stopped. In an open letter, several of the world's biggest stars have warned the tech'will set in motion a race to the bottom' if left unchecked. The use of AI to steal artists' voices, likeness, and sound is an'assault on human creativity', they said, and would'destroy the music ecosystem'. Issued by the Artists Rights Alliance (ARA), the letter calls for a ban on AI tools that undermine or replace human songwriters or their work. The move is part of an industry-wide push for better regulation of generative AI, the technology behind chatbots like ChatGPT and image generators like Midjourney.
George Carlin's estate reaches settlement over AI-generated comedy special
Fox News Flash top entertainment and celebrity headlines are here. George Carlin's estate has agreed to a settlement with the media company it sued earlier this year over the use of artificial intelligence. In January, Carlin's estate sued the podcast company, Dudesy, for recreating Carlin's iconic comedic style in an hour-long special titled "George Carlin: I'm Glad I'm Dead." The settlement indicates that Dudesy is required to permanently remove the special and cannot use Carlin's image voice or likeness in the future without written consent from the estate. According to The Associated Press, the settlement agreement was approved by both sides and awaits a judge's approval.
Long-form factuality in large language models
Wei, Jerry, Yang, Chengrun, Song, Xinying, Lu, Yifeng, Hu, Nathan, Huang, Jie, Tran, Dustin, Peng, Daiyi, Liu, Ruibo, Huang, Da, Du, Cosmo, Le, Quoc V.
Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics. To benchmark a model's long-form factuality in open domains, we first use GPT-4 to generate LongFact, a prompt set comprising thousands of questions spanning 38 topics. We then propose that LLM agents can be used as automated evaluators for long-form factuality through a method which we call Search-Augmented Factuality Evaluator (SAFE). SAFE utilizes an LLM to break down a long-form response into a set of individual facts and to evaluate the accuracy of each fact using a multi-step reasoning process comprising sending search queries to Google Search and determining whether a fact is supported by the search results. Furthermore, we propose extending F1 score as an aggregated metric for long-form factuality. To do so, we balance the percentage of supported facts in a response (precision) with the percentage of provided facts relative to a hyperparameter representing a user's preferred response length (recall). Empirically, we demonstrate that LLM agents can outperform crowdsourced human annotators - on a set of ~16k individual facts, SAFE agrees with crowdsourced human annotators 72% of the time, and on a random subset of 100 disagreement cases, SAFE wins 76% of the time. At the same time, SAFE is more than 20 times cheaper than human annotators. We also benchmark thirteen language models on LongFact across four model families (Gemini, GPT, Claude, and PaLM-2), finding that larger language models generally achieve better long-form factuality. LongFact, SAFE, and all experimental code are available at https://github.com/google-deepmind/long-form-factuality.
Blessing or curse? A survey on the Impact of Generative AI on Fake News
Loth, Alexander, Kappes, Martin, Pahl, Marc-Oliver
Fake news significantly influence our society. They impact consumers, voters, and many other societal groups. While Fake News exist for a centuries, Generative AI brings fake news on a new level. It is now possible to automate the creation of masses of high-quality individually targeted Fake News. On the other end, Generative AI can also help detecting Fake News. Both fields are young but developing fast. This survey provides a comprehensive examination of the research and practical use of Generative AI for Fake News detection and creation in 2024. Following the Structured Literature Survey approach, the paper synthesizes current results in the following topic clusters 1) enabling technologies, 2) creation of Fake News, 3) case study social media as most relevant distribution channel, 4) detection of Fake News, and 5) deepfakes as upcoming technology. The article also identifies current challenges and open issues.