Media
Chris Pratt blasts Hollywood stars with 'bad' attitudes on set, says it 'ruins everything for everyone'
Pratt, Charlie Day and Jack Black on new animated film, playing iconic characters, being fans of the game and more. Chris Pratt had harsh words for Hollywood stars who bring negativity to movie sets. On Thursday, the 45-year-old actor joined co-star Millie Bobby Brown and directors Joe and Anthony Russo on the New York Comic Con panel for their upcoming Netflix sci-fi film "The Electric State." During the panel, Pratt slammed actors who had "bad" attitudes while filming. "Look, these guys can attest to this, because they're the same way. Like, there's no room for s---ty attitudes there," the Marvel star said, via People magazine.
SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
Chen, Jingxuan, Yuen, Derek, Xie, Bin, Yang, Yuhao, Chen, Gongwei, Wu, Zhihao, Yixing, Li, Zhou, Xurui, Liu, Weiwen, Wang, Shuai, Zhou, Kaiwen, Shao, Rui, Nie, Liqiang, Wang, Yasheng, Hao, Jianye, Wang, Jun, Shao, Kun
Smartphone agents are increasingly important for helping users control devices efficiently, with (Multimodal) Large Language Model (MLLM)-based approaches emerging as key contenders. Fairly comparing these agents is essential but challenging, requiring a varied task scope, the integration of agents with different implementations, and a generalisable evaluation pipeline to assess their strengths and weaknesses. In this paper, we present SPA-Bench, a comprehensive SmartPhone Agent Benchmark designed to evaluate (M)LLM-based agents in an interactive environment that simulates real-world conditions. SPA-Bench offers three key contributions: (1) A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines; (2) A plug-and-play framework enabling real-time agent interaction with Android devices, integrating over ten agents with the flexibility to add more; (3) A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption. Our extensive experiments across tasks and agents reveal challenges like interpreting mobile user interfaces, action grounding, memory retention, and execution costs. We propose future research directions to ease these difficulties, moving closer to real-world smartphone agent applications.
Bias Amplification: Language Models as Increasingly Biased Media
Wang, Ze, Wu, Zekun, Zhang, Jeremy, Jain, Navya, Guan, Xin, Koshiyama, Adriano
As Large Language Models (LLMs) become increasingly integrated into various facets of society, a significant portion of online text consequently become synthetic. This raises concerns about bias amplification, a phenomenon where models trained on synthetic data amplify the pre-existing biases over successive training iterations. Previous literature seldom discusses bias amplification as an independent issue from model collapse. In this work, we address the gap in understanding the bias amplification of LLMs with four main contributions. Firstly, we propose a theoretical framework, defining the necessary and sufficient conditions for its occurrence, and emphasizing that it occurs independently of model collapse. Using statistical simulations with weighted maximum likelihood estimation, we demonstrate the framework and show how bias amplification arises without the sampling and functional form issues that typically drive model collapse. Secondly, we conduct experiments with GPT-2 to empirically demonstrate bias amplification, specifically examining open-ended generational political bias with a benchmark we developed. We observe that GPT-2 exhibits a right-leaning bias in sentence continuation tasks and that the bias progressively increases with iterative fine-tuning on synthetic data generated by previous iterations. Thirdly, we explore three potential mitigation strategies: Overfitting, Preservation, and Accumulation. We find that both Preservation and Accumulation effectively mitigate bias amplification and model collapse. Finally, using novel mechanistic interpretation techniques, we demonstrate that in the GPT-2 experiments, bias amplification and model collapse are driven by distinct sets of neurons, which aligns with our theoretical framework.
The Computational Anatomy of Humility: Modeling Intellectual Humility in Online Public Discourse
Guo, Xiaobo, Potnis, Neil, Yu, Melody, Gillani, Nabeel, Vosoughi, Soroush
The ability for individuals to constructively engage with one another across lines of difference is a critical feature of a healthy pluralistic society. This is also true in online discussion spaces like social media platforms. To date, much social media research has focused on preventing ills -- like political polarization and the spread of misinformation. While this is important, enhancing the quality of online public discourse requires not just reducing ills but also promoting foundational human virtues. In this study, we focus on one particular virtue: ``intellectual humility'' (IH), or acknowledging the potential limitations in one's own beliefs. Specifically, we explore the development of computational methods for measuring IH at scale. We manually curate and validate an IH codebook on 350 posts about religion drawn from subreddits and use them to develop LLM-based models for automating this measurement. Our best model achieves a Macro-F1 score of 0.64 across labels (and 0.70 when predicting IH/IA/Neutral at the coarse level), higher than an expected naive baseline of 0.51 (0.32 for IH/IA/Neutral) but lower than a human annotator-informed upper bound of 0.85 (0.83 for IH/IA/Neutral). Our results both highlight the challenging nature of detecting IH online -- opening the door to new directions in NLP research -- and also lay a foundation for computational social science researchers interested in analyzing and fostering more IH in online public discourse.
Augmenting the Veracity and Explanations of Complex Fact Checking via Iterative Self-Revision with LLMs
Zhang, Xiaocheng, Wang, Xi, Lu, Yifei, Ye, Zhuangzhuang, Wang, Jianing, Bao, Mengjiao, Yan, Peng, Su, Xiaohong
Explanation generation plays a more pivotal role than fact verification in producing interpretable results and facilitating comprehensive fact-checking, which has recently garnered considerable attention. However, previous studies on explanation generation has shown several limitations, such as being confined to English scenarios, involving overly complex inference processes, and not fully unleashing the potential of the mutual feedback between veracity labels and explanation texts. To address these issues, we construct two complex fact-checking datasets in the Chinese scenarios: CHEF-EG and TrendFact. These datasets involve complex facts in areas such as health, politics, and society, presenting significant challenges for fact verification methods. In response to these challenges, we propose a unified framework called FactISR (Augmenting Fact-Checking via Iterative Self-Revision) to perform mutual feedback between veracity and explanations by leveraging the capabilities of large language models(LLMs). FactISR uses a single model to address tasks such as fact verification and explanation generation. Its self-revision mechanism can further revision the consistency between veracity labels, explanation texts, and evidence, as well as eliminate irrelevant noise. We conducted extensive experiments with baselines and FactISR on the proposed datasets. The experimental results demonstrate the effectiveness of our method.
Are LLMs Good Zero-Shot Fallacy Classifiers?
Pan, Fengjun, Wu, Xiaobao, Li, Zongrui, Luu, Anh Tuan
Fallacies are defective arguments with faulty reasoning. Detecting and classifying them is a crucial NLP task to prevent misinformation, manipulative claims, and biased decisions. However, existing fallacy classifiers are limited by the requirement for sufficient labeled data for training, which hinders their out-of-distribution (OOD) generalization abilities. In this paper, we focus on leveraging Large Language Models (LLMs) for zero-shot fallacy classification. To elicit fallacy-related knowledge and reasoning abilities of LLMs, we propose diverse single-round and multi-round prompting schemes, applying different task-specific instructions such as extraction, summarization, and Chain-of-Thought reasoning. With comprehensive experiments on benchmark datasets, we suggest that LLMs could be potential zero-shot fallacy classifiers. In general, LLMs under single-round prompting schemes have achieved acceptable zero-shot performances compared to the best full-shot baselines and can outperform them in all OOD inference scenarios and some open-domain tasks. Our novel multi-round prompting schemes can effectively bring about more improvements, especially for small LLMs. Our analysis further underlines the future research on zero-shot fallacy classification. Codes and data are available at: https://github.com/panFJCharlotte98/Fallacy_Detection.
Audio Processing using Pattern Recognition for Music Genre Classification
Chatterjee, Sivangi, Ganguly, Srishti, Bose, Avik, Prasad, Hrithik Raj, Ghosal, Arijit
This project explores the application of machine learning techniques for music genre classification using the GTZAN dataset, which contains 100 audio files per genre. Motivated by the growing demand for personalized music recommendations, we focused on classifying five genres--Blues, Classical, Jazz, Hip Hop, and Country--using a variety of algorithms including Logistic Regression, K-Nearest Neighbors (KNN), Random Forest, and Artificial Neural Networks (ANN) implemented via Keras. The ANN model demonstrated the best performance, achieving a validation accuracy of 92.44%. We also analyzed key audio features such as spectral roll-off, spectral centroid, and MFCCs, which helped enhance the model's accuracy. Future work will expand the model to cover all ten genres, investigate advanced methods like Long Short-Term Memory (LSTM) networks and ensemble approaches, and develop a web application for real-time genre classification and playlist generation. This research aims to contribute to improving music recommendation systems and content curation.
Penguin Random House amends its copyright rules to protect authors from AI
Artificial intelligence makers have faced a mountain of criticism for borrowing from the work of others to train its models. Now the world's largest publishing house is taking steps to ensure its authors don't have their work plagiarized in the name of progress. Now the wording states: "No part of this book may be used or reproduced in any manner for the purpose of training artificial intelligence technologies or systems." The new wording also protects against data absorption by noting the publisher "expressly reserves [the titles] from the text and data mining exception." This part of the amended text comes from a recent European Parliament directive regarding text and data mining exceptions and ownership.
Canon EOS R5 II review: Canon's most powerful camera yet puts Sony on notice
Move over Sony, Canon is trying to take the lead in bleeding-edge tech for mirrorless cameras. The company's new 4,300, 45-megapixel EOS R5 II offers advanced features like eye-tracking autofocus (AF) that can't be found on any recent Sony model. The new camera is also pushing Sony's A1 and other models in the key areas of speed, video and autofocus. And it's arguably more desirable than Canon's own upcoming flagship R1 as it has nearly double the resolution. I've had the R5 II for a few weeks, evaluating not only its practicality and speed for both professionals and serious amateurs, but also how it stacks up against Sony's A1, the gold standard for high-resolution mirrorless cameras.
I'm a teacher - here are the conspiracy theories my 6th graders believe in
A language arts teacher has shared the bizarre conspiracy theories her sixth grade students believe in and what fostered that beliefs. The teacher, who goes by the name Ms Alexanderr, said was amazed by her students' ideas and wanted to compile a list of the top five most she felt were the most bizarre. While the teacher said she wasn't surprised by one conspiracy theory that birds aren't real, she was shocked and couldn't understand others. Among them was the theory that Bill Nye the science guy is a Russian spy while another claimed Michael Jackson was still alive. The pop-star conspiracy was particularly perplexing, because her students were born after he died in 2009.