Media
Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMs
Singhal, Ronit, Patwa, Pransh, Patwa, Parth, Chadha, Aman, Das, Amitava
Given the widespread dissemination of misinformation on social media, implementing fact-checking mechanisms for online claims is essential. Manually verifying every claim is highly challenging, underscoring the need for an automated fact-checking system. This paper presents our system designed to address this issue. We utilize the Averitec dataset to assess the veracity of claims. In addition to veracity prediction, our system provides supporting evidence, which is extracted from the dataset. We develop a Retrieve and Generate (RAG) pipeline to extract relevant evidence sentences from a knowledge base, which are then inputted along with the claim into a large language model (LLM) for classification. We also evaluate the few-shot In-Context Learning (ICL) capabilities of multiple LLMs. Our system achieves an 'Averitec' score of 0.33, which is a 22% absolute improvement over the baseline. All code will be made available on All code will be made available on https://github.com/ronit-singhal/evidence-backed-fact-checking-using-rag-and-few-shot-in-context-learning-with-llms.
Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarization
Cheng, Luyao, Wang, Hui, Zheng, Siqi, Chen, Yafeng, Huang, Rongjie, Zhang, Qinglin, Chen, Qian, Li, Xihao
Speaker diarization, the process of segmenting an audio stream or transcribed speech content into homogenous partitions based on speaker identity, plays a crucial role in the interpretation and analysis of human speech. Most existing speaker diarization systems rely exclusively on unimodal acoustic information, making the task particularly challenging due to the innate ambiguities of audio signals. Recent studies have made tremendous efforts towards audio-visual or audio-semantic modeling to enhance performance. However, even the incorporation of up to two modalities often falls short in addressing the complexities of spontaneous and unstructured conversations. To exploit more meaningful dialogue patterns, we propose a novel multimodal approach that jointly utilizes audio, visual, and semantic cues to enhance speaker diarization. Our method elegantly formulates the multimodal modeling as a constrained optimization problem. First, we build insights into the visual connections among active speakers and the semantic interactions within spoken content, thereby establishing abundant pairwise constraints. Then we introduce a joint pairwise constraint propagation algorithm to cluster speakers based on these visual and semantic constraints. This integration effectively leverages the complementary strengths of different modalities, refining the affinity estimation between individual speaker embeddings. Extensive experiments conducted on multiple multimodal datasets demonstrate that our approach consistently outperforms state-of-the-art speaker diarization methods.
Great Memory, Shallow Reasoning: Limits of $k$NN-LMs
Geng, Shangyi, Zhao, Wenting, Rush, Alexander M
$K$-nearest neighbor language models ($k$NN-LMs), which integrate retrieval with next-word prediction, have demonstrated strong performance in language modeling as well as downstream NLP benchmarks. These results have led researchers to argue that models trained on poor quality or outdated data could perform well by employing a $k$NN extension that has access to a higher-quality datastore. In this work, we ask whether this improved ability to recall information really translates into downstream abilities. We extensively evaluate $k$NN-LMs on a diverse set of tasks, ranging from sentiment classification and commonsense reasoning to multi-hop reasoning. Results show that $k$NN-LMs excel at memory-intensive tasks, where utilizing the patterns in the input is sufficient for determining the output, but struggle with reasoning tasks that require integrating multiple pieces of information to derive new knowledge. We further demonstrate through oracle experiments and qualitative analysis that even with perfect retrieval, $k$NN-LMs still fail to determine the correct answers, placing an upper bound on their reasoning performance. Code and datastores are released at https://github.com/GSYfate/knnlm-limits/.
DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Framework
Xie, Zhifei, Tang, Daniel, Tan, Dingwei, Klein, Jacques, Bissyand, Tegawend F., Ezzini, Saad
Current video generation models excel at creating short, realistic clips, but struggle with longer, multi-scene videos. We introduce \texttt{DreamFactory}, an LLM-based framework that tackles this challenge. \texttt{DreamFactory} leverages multi-agent collaboration principles and a Key Frames Iteration Design Method to ensure consistency and style across long videos. It utilizes Chain of Thought (COT) to address uncertainties inherent in large language models. \texttt{DreamFactory} generates long, stylistically coherent, and complex videos. Evaluating these long-form videos presents a challenge. We propose novel metrics such as Cross-Scene Face Distance Score and Cross-Scene Style Consistency Score. To further research in this area, we contribute the Multi-Scene Videos Dataset containing over 150 human-rated videos.
Estimated Audio-Caption Correspondences Improve Language-Based Audio Retrieval
Primus, Paul, Schmid, Florian, Widmer, Gerhard
Dual-encoder-based audio retrieval systems are commonly optimized with contrastive learning on a set of matching and mismatching audio-caption pairs. This leads to a shared embedding space in which corresponding items from the two modalities end up close together. Since audio-caption datasets typically only contain matching pairs of recordings and descriptions, it has become common practice to create mismatching pairs by pairing the audio with a caption randomly drawn from the dataset. This is not ideal because the randomly sampled caption could, just by chance, partly or entirely describe the audio recording. However, correspondence information for all possible pairs is costly to annotate and thus typically unavailable; we, therefore, suggest substituting it with estimated correspondences. To this end, we propose a two-staged training procedure in which multiple retrieval models are first trained as usual, i.e., without estimated correspondences. In the second stage, the audio-caption correspondences predicted by these models then serve as prediction targets. We evaluate our method on the ClothoV2 and the AudioCaps benchmark and show that it improves retrieval performance, even in a restricting self-distillation setting where a single model generates and then learns from the estimated correspondences. We further show that our method outperforms the current state of the art by 1.6 pp. mAP@10 on the ClothoV2 benchmark.
Let Community Rules Be Reflected in Online Content Moderation
Xin, Wangjiaxuan, Wang, Kanlun, Fu, Zhe, Zhou, Lina
Content moderation is a typical intervention strategy for Content moderation is a widely used strategy to regulating online communities on social media prevent the dissemination of irregular information on platforms, to ensure that user-generated content social media platforms. Despite extensive research on complies with the platforms' policies and community developing automated models to support decisionmaking standards (Gillespie, 2020). in content moderation, there remains a notable With the advancement of AI technologies and the scarcity of studies that integrate the rules of online increasing workload associated with online moderation communities into content moderation. This study (Batrinca & Treleaven, 2015), online platforms are addresses this gap by proposing a community rulebased increasingly adopting machine learning and/or deep content moderation framework that directly learning-based techniques to automate content integrates community rules into the moderation of usergenerated moderation, particularly to address its scalability issue content.
Epistemic Injustice in Generative AI
Kay, Jackie, Kasirzadeh, Atoosa, Mohamed, Shakir
While traditional discussions of epistemic injustice have While algorithms have traditionally been leveraged to primarily centered on interpersonal human interactions present and organize human-generated content, the advent (McKinnon 2017; Tsosie 2012), existing research on algorithmic of generative AI has started to fundamentally shift this epistemic injustice has largely been limited to epistemic paradigm. Generative AI models can now create content - injustices produced by decision-making and classification spanning text, imagery, and beyond - that resembles that of algorithms. However, we argue that the distinctive authors, journalists, painters, or photographers. In this paper, characteristics of generative AI give rise to novel forms of we take generative AI to be the class of machine learning epistemic injustice that necessitate a dedicated analytical models trained on massive amounts of data, typically media framework. To address this, we expand upon the established such as text, images, audio or video, in order to produce philosophical discourse on epistemic injustice and introduce representative instances of such media (García-Peñalvo and an account of "generative algorithmic epistemic injustice," Vázquez-Ingelmo 2023).
San Francisco's Nocturnal Taxi Ballet
For the past few nights, I've concerned myself with the private lives of autonomous vehicles. It started when I read a news story about a San Francisco apartment complex whose residents were repeatedly awoken at 4 a.m. by honking self-driving taxis. The building overlooks an open-air parking lot that Waymo recently leased to store its vehicles. In the wee hours of the morning--between ferrying home overserved bar crawlers and picking up commuters during the morning rush hour--dozens of the autonomous white sedans fill the lot, power down, and wait to be summoned. Sometimes, too many awaken at the same time and back up while trying to make their way to the exit, only to find the lanes clogged by their brethren.
Animated video game anthology series Secret Level is coming to Prime Video
Secret Level is a gaming-inspired anthology series coming to Prime Video on December 15. The upcoming Amazon title is from the same team behind Netflix's Love, Death and Robots. From the teaser released during Gamescom, this new project will be continuing the Blur Studio specialty for creating masterful animated works for an adult audience. The creative team was looking to inspire "nerd joy" with Love, Death and Robots and from the first glimpse, Secret Level seems like a natural progression of that goal. Each of the 15 stories in the show are inspired by a different game.
OpenAI signs multi-year content partnership with Condé Nast
Condé Nast and OpenAI announced a multi-year partnership on Tuesday to display content from the publisher's brands such as the Vogue, Wired and the New Yorker within the AI startup's products, including ChatGPT and its SearchGPT prototype. The financial terms of the deal were not disclosed. The Microsoft-backed, Sam Altman-led firm has signed similar deals with Time magazine, the Financial Times, Business Insider owner Axel Springer, France's Le Monde and Spain's Prisa Media over the past few months. The deals give OpenAI access to the large archives of text owned by the publishers, which are necessary both for training large language models like ChatGPT and for finding real-time information. OpenAI launched its AI-powered search engine SearchGPT in July, with real-time access to information from the internet, making an incursion on territory long dominated by Google.