Media
I'm a teacher - here are the conspiracy theories my 6th graders believe in
A language arts teacher has shared the bizarre conspiracy theories her sixth grade students believe in and what fostered that beliefs. The teacher, who goes by the name Ms Alexanderr, said was amazed by her students' ideas and wanted to compile a list of the top five most she felt were the most bizarre. While the teacher said she wasn't surprised by one conspiracy theory that birds aren't real, she was shocked and couldn't understand others. Among them was the theory that Bill Nye the science guy is a Russian spy while another claimed Michael Jackson was still alive. The pop-star conspiracy was particularly perplexing, because her students were born after he died in 2009.
Amazon's Alexa has been spreading FAKE news on everything from MPs' expenses to the origins of the Northern Lights, shocking report reveals
It's supposed to be the reliable smart assistant that'makes your life easier' with instant titbits of information. But a shocking report has revealed that in many cases, Amazon's Alexa doesn't know the difference between right and wrong. An investigation by Full Fact has found that Alexa spouts incorrect information on topics ranging from MPs' expenses to the origins of the Northern Lights. Full Fact, the UK's independent fact checking organisation, called the findings'misleading' and'clearly a big problem'. What's more, staff at the organization have been furious to discover that Alexa was attributing the wrong answers to none other than Full Fact.
ESPN accused of 'willfully' violating emergency alert system rules in 2023 NBA promos
Fox News Flash top sports headlines are here. Check out what's clicking on Foxnews.com. The Federal Communications Commission (FCC) proposed a fine of 146,976 against ESPN Thursday over the company's alleged improper use of the emergency alert system (EAS). The FCC accused ESPN of violating the emergency alert system six times, starting Oct. 20, 2023. The agency said ESPN transmitted or caused the transmittance of emergency alert system codes "during a promotional segment for the start of the 2023-2024 NBA season."
Critical Questions Generation: Motivation and Challenges
Figueras, Blanca Calvo, Agerri, Rodrigo
The development of Large Language Models (LLMs) has brought impressive performances on mitigation strategies against misinformation, such as counterargument generation. However, LLMs are still seriously hindered by outdated knowledge and by their tendency to generate hallucinated content. In order to circumvent these issues, we propose a new task, namely, Critical Questions Generation, consisting of processing an argumentative text to generate the critical questions (CQs) raised by it. In argumentation theory CQs are tools designed to lay bare the blind spots of an argument by pointing at the information it could be missing. Thus, instead of trying to deploy LLMs to produce knowledgeable and relevant counterarguments, we use them to question arguments, without requiring any external knowledge. Research on CQs Generation using LLMs requires a reference dataset for large scale experimentation. Thus, in this work we investigate two complementary methods to create such a resource: (i) instantiating CQs templates as defined by Walton's argumentation theory and (ii), using LLMs as CQs generators. By doing so, we contribute with a procedure to establish what is a valid CQ and conclude that, while LLMs are reasonable CQ generators, they still have a wide margin for improvement in this task.
MixEval-X: Any-to-Any Evaluations from Real-World Data Mixtures
Ni, Jinjie, Song, Yifan, Ghosal, Deepanway, Li, Bo, Zhang, David Junhao, Yue, Xiang, Xue, Fuzhao, Zheng, Zian, Zhang, Kaichen, Shah, Mahir, Jain, Kabir, You, Yang, Shieh, Michael
Perceiving and generating diverse modalities are crucial for AI models to effectively learn from and engage with real-world signals, necessitating reliable evaluations for their development. We identify two major issues in current evaluations: (1) inconsistent standards, shaped by different communities with varying protocols and maturity levels; and (2) significant query, grading, and generalization biases. To address these, we introduce MixEval-X, the first any-to-any, real-world benchmark designed to optimize and standardize evaluations across diverse input and output modalities. We propose multi-modal benchmark mixture and adaptation-rectification pipelines to reconstruct real-world task distributions, ensuring evaluations generalize effectively to real-world use cases. Extensive meta-evaluations show our approach effectively aligns benchmark samples with real-world task distributions. Meanwhile, MixEval-X's model rankings correlate strongly with that of crowd-sourced real-world evaluations (up to 0.98) while being much more efficient. We provide comprehensive leaderboards to rerank existing models and organizations and offer insights to enhance understanding of multi-modal evaluations and inform future research.
FAME: Towards Factual Multi-Task Model Editing
Zeng, Li, Shan, Yingyu, Liu, Zeming, Yao, Jiashu, Guo, Yuhang
Large language models (LLMs) embed extensive knowledge and utilize it to perform exceptionally well across various tasks. Nevertheless, outdated knowledge or factual errors within LLMs can lead to misleading or incorrect responses, causing significant issues in practical applications. To rectify the fatal flaw without the necessity for costly model retraining, various model editing approaches have been proposed to correct inaccurate knowledge within LLMs in a cost-efficient way. To evaluate these model editing methods, previous work introduced a series of datasets. However, most of the previous datasets only contain fabricated data in a single format, which diverges from real-world model editing scenarios, raising doubts about their usability in practice. To facilitate the application of model editing in real-world scenarios, we propose the challenge of practicality. To resolve such challenges and effectively enhance the capabilities of LLMs, we present FAME, an factual, comprehensive, and multi-task dataset, which is designed to enhance the practicality of model editing. We then propose SKEME, a model editing method that uses a novel caching mechanism to ensure synchronization with the real world. The experiments demonstrate that SKEME performs excellently across various tasks and scenarios, confirming its practicality.
DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation
Cheng, Hanbo, Lin, Limin, Liu, Chenyu, Xia, Pengcheng, Hu, Pengfei, Ma, Jiefeng, Du, Jun, Pan, Jia
Talking head generation intends to produce vivid and realistic talking head videos from a single portrait and speech audio clip. Although significant progress has been made in diffusion-based talking head generation, almost all methods rely on autoregressive strategies, which suffer from limited context utilization beyond the current generation step, error accumulation, and slower generation speed. To address these challenges, we present DAWN (Dynamic frame Avatar With Non-autoregressive diffusion), a framework that enables all-at-once generation of dynamic-length video sequences. Specifically, it consists of two main components: (1) audio-driven holistic facial dynamics generation in the latent motion space, and (2) audio-driven head pose and blink generation. Extensive experiments demonstrate that our method generates authentic and vivid videos with precise lip motions, and natural pose/blink movements. Additionally, with a high generation speed, DAWN possesses strong extrapolation capabilities, ensuring the stable production of high-quality long videos. Furthermore, we hope that DAWN sparks further exploration of non-autoregressive approaches in diffusion models. Talking head generation aims at synthesizing a realistic and expressive talking head from a given portrait and audio clip, which is garnering growing interest due to its potential applications in virtual meetings, gaming, and film production. For talking head generation, it is essential that the lip motions in the generated video precisely match the accompanying speech, while maintaining high overall visual fidelity (Guo et al., 2021a). Furthermore, natural coordination between head pose, eye blinking, and the rhythm of the audio is also crucial for a convincing output (Liu et al., 2023).
DiscoGraMS: Enhancing Movie Screen-Play Summarization using Movie Character-Aware Discourse Graph
Chitale, Maitreya Prafulla, Bindal, Uday, Rajkumar, Rajakrishnan, Mishra, Rahul
Summarizing movie screenplays presents a unique set of challenges compared to standard document summarization. Screenplays are not only lengthy, but also feature a complex interplay of characters, dialogues, and scenes, with numerous direct and subtle relationships and contextual nuances that are difficult for machine learning models to accurately capture and comprehend. Recent attempts at screenplay summarization focus on fine-tuning transformer-based pre-trained models, but these models often fall short in capturing long-term dependencies and latent relationships, and frequently encounter the "lost in the middle" issue. To address these challenges, we introduce DiscoGraMS, a novel resource that represents movie scripts as a movie character-aware discourse graph (CaD Graph). This approach is well-suited for various downstream tasks, such as summarization, question-answering, and salience detection. The model aims to preserve all salient information, offering a more comprehensive and faithful representation of the screenplay's content. We further explore a baseline method that combines the CaD Graph with the corresponding movie script through a late fusion of graph and text modalities, and we present very initial promising results.
Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation
Tang, Shuo, Pang, Xianghe, Liu, Zexi, Tang, Bohan, Ye, Rui, Dong, Xiaowen, Wang, Yanfeng, Chen, Siheng
We conducted experiments comparing the effectiveness of using simpler versus more complex dataset in different stages of the post-training process to better understand the optimal post-training strategy for large language models. Here we conduct comparison experiment on two kinds of instructions: simple instructions and specialized instructions, denoted as type 1 and type 2. As showen in Table 10, we observe that performing SFT on simpler instructions helps the model to establish a foundational level of instruction-following ability. This is reflected in moderate performance on AlpacaEval 2 (LC 16.25%, WR 17.62%) but lower performance on the more challenging Arena-Hard benchmark (WR 10.7%). When the model is fine-tuned on more specialized and complex data, there is a marginal improvement (LC 14.70%, WR 16.01%, Arena-Hard WR 14.7%), and the significant performance gains are achieved when DPO is applied after SFT. For example, SFT followed by DPO with complex, specialized instructions yields substantial improvements (LC 21.64%, WR 30.06%,
Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning
He, Pengfei, Li, Zitao, Xing, Yue, Li, Yaling, Tang, Jiliang, Ding, Bolin
Zero-shot reasoning methods with Large Language Models (LLMs) offer significant advantages including great generalization to novel tasks and reduced dependency on human-crafted examples. However, the current zero-shot methods still have limitations in complex tasks, e.g., answering questions that require multi-step reasoning. In this paper, we address this limitation by introducing a novel structure-oriented analysis method to help LLMs better understand the question and guide the problem-solving process of LLMs. We first demonstrate how the existing reasoning strategies, Chain-of-Thought and ReAct, can benefit from our structure-oriented analysis. In addition to empirical investigations, we leverage the probabilistic graphical model to theoretically explain why our structure-oriented analysis can improve the LLM reasoning process. To further improve the reliability in complex question-answering tasks, we propose a multi-agent reasoning system, Structure-oriented Autonomous Reasoning Agents (SARA), that can better enforce the reasoning process following our structure-oriented analysis by refinement techniques and is equipped with external knowledge retrieval capability to reduce factual errors. Extensive experiments verify the effectiveness of the proposed reasoning system. Surprisingly, in some cases, the system even surpasses few-shot methods. Finally, the system not only improves reasoning accuracy in complex tasks but also demonstrates robustness against potential attacks that corrupt the reasoning process.