Generative AI
Elon Musk warns he will BAN Apple devices from his firms after new AI deal to integrate ChatGPT into iPhones and iPads - calling it a 'security violation'
Elon Musk announced Monday that he'll ban Apple devices from his companies' premises if the iPhone creator goes forward with its planned OpenAI integration. Apple held its annual Worldwide Developer's Conference Monday, and the biggest takeaway was OpenAI's ChatGPT will be paired with Apple's assistant Siri. In one example shown, Siri recommended that the iPhone user consult ChatGPT for further dinner recipe ideas, flagging all the way that these new answers were coming from OpenAI's chatbot and advising users to'check important info for mistakes'. Musk responded to the partnership on X, saying employees and visitors to his companies would'have to check their Apple devices at the door' where they will be stored in a'Faraday cage' – a cage that blocks electromagnetic fields. He continued: 'If Apple integrates OpenAI at the OS level, then Apple devices will be banned at my companies.
Danish Media Threatens to Sue OpenAI
In the latest battle between AI and the media, major Danish newspapers and TV stations are threatening to sue OpenAI unless the company compensates the country's press for allegedly using their content to train its models. "We want remuneration for our work [which] they have used to train their model," says Karen Rønde, CEO of the Danish Press Publications' Collective Management Organization (DPCMO), which represents 99 percent of Danish media outlets, including state broadcaster DR and TV 2. Rønde says the DPCMO plans to sue if a deal is not reached in the next year. Soon after those lawsuits, OpenAI struck a series of licensing deals with major publishers, enabling the company to train its future iterations of ChatGPT on their content. Financial terms for the deals have not been disclosed. Now, Danish media is attempting to force OpenAI to negotiate with them as a collective, an unusual tactic that could provide a model for other small countries if successful.
Apple unveils long-awaited AI strategy, partnership with OpenAI
Apple has revealed a slew of new artificial intelligence-powered features backed by a partnership with OpenAI, as the iPhone maker battles perceptions that it is falling behind in the race to capitalise on the technology. Apple executives including CEO Tim Cook unveiled "Apple Intelligence" on Monday during a nearly two-hour-long presentation at the company's annual Worldwide Developers Conference in Cupertino, California. "All of this goes beyond artificial intelligence, it's personal intelligence, and it is the next big step for Apple," CEO Tim Cook said. The upgrades include an overhaul of the virtual assistant Siri, which will be capable of hundreds of more tasks with the help of ChatGPT. Apple users will also be able to create their own emojis based on language prompts and generate summaries of emails in the mailbox via the tech giant's in-house technology.
Evaluating Contextually Personalized Programming Exercises Created with Generative AI
Logacheva, Evanfiya, Hellas, Arto, Prather, James, Sarsa, Sami, Leinonen, Juho
Programming skills are typically developed through completing various hands-on exercises. Such programming problems can be contextualized to students' interests and cultural backgrounds. Prior research in educational psychology has demonstrated that context personalization of exercises stimulates learners' situational interests and positively affects their engagement. However, creating a varied and comprehensive set of programming exercises for students to practice on is a time-consuming and laborious task for computer science educators. Previous studies have shown that large language models can generate conceptually and contextually relevant programming exercises. Thus, they offer a possibility to automatically produce personalized programming problems to fit students' interests and needs. This article reports on a user study conducted in an elective introductory programming course that included contextually personalized programming exercises created with GPT-4. The quality of the exercises was evaluated by both the students and the authors. Additionally, this work investigated student attitudes towards the created exercises and their engagement with the system. The results demonstrate that the quality of exercises generated with GPT-4 was generally high. What is more, the course participants found them engaging and useful. This suggests that AI-generated programming problems can be a worthwhile addition to introductory programming courses, as they provide students with a practically unlimited pool of practice material tailored to their personal interests and educational needs.
Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?
Fu, Xingyu, He, Muyu, Lu, Yujie, Wang, William Yang, Roth, Dan
We present a novel task and benchmark for evaluating the ability of text-to-image(T2I) generation models to produce images that fit commonsense in real life, which we call Commonsense-T2I. Given two adversarial text prompts containing an identical set of action words with minor differences, such as "a lightbulb without electricity" v.s. "a lightbulb with electricity", we evaluate whether T2I models can conduct visual-commonsense reasoning, e.g. produce images that fit "the lightbulb is unlit" vs. "the lightbulb is lit" correspondingly. Commonsense-T2I presents an adversarial challenge, providing pairwise text prompts along with expected outputs. The dataset is carefully hand-curated by experts and annotated with fine-grained labels, such as commonsense type and likelihood of the expected outputs, to assist analyzing model behavior. We benchmark a variety of state-of-the-art (sota) T2I models and surprisingly find that, there is still a large gap between image synthesis and real life photos--even the DALL-E 3 model could only achieve 48.92% on Commonsense-T2I, and the stable diffusion XL model only achieves 24.92% accuracy. Our experiments show that GPT-enriched prompts cannot solve this challenge, and we include a detailed analysis about possible reasons for such deficiency. We aim for Commonsense-T2I to serve as a high-quality evaluation benchmark for T2I commonsense checking, fostering advancements in real life image generation.
Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities
Pandiani, Delfina Sol Martinez, Sang, Erik Tjong Kim, Ceolin, Davide
Internet memes, channels for humor, social commentary, and cultural expression, are increasingly used to spread toxic messages. Studies on the computational analyses of toxic memes have significantly grown over the past five years, and the only three surveys on computational toxic meme analysis cover only work published until 2022, leading to inconsistent terminology and unexplored trends. Our work fills this gap by surveying content-based computational perspectives on toxic memes, and reviewing key developments until early 2024. Employing the PRISMA methodology, we systematically extend the previously considered papers, achieving a threefold result. First, we survey 119 new papers, analyzing 158 computational works focused on content-based toxic meme analysis. We identify over 30 datasets used in toxic meme analysis and examine their labeling systems. Second, after observing the existence of unclear definitions of meme toxicity in computational works, we introduce a new taxonomy for categorizing meme toxicity types. We also note an expansion in computational tasks beyond the simple binary classification of memes as toxic or non-toxic, indicating a shift towards achieving a nuanced comprehension of toxicity. Third, we identify three content-based dimensions of meme toxicity under automatic study: target, intent, and conveyance tactics. We develop a framework illustrating the relationships between these dimensions and meme toxicities. The survey analyzes key challenges and recent trends, such as enhanced cross-modal reasoning, integrating expert and cultural knowledge, the demand for automatic toxicity explanations, and handling meme toxicity in low-resource languages. Also, it notes the rising use of Large Language Models (LLMs) and generative AI for detecting and generating toxic memes. Finally, it proposes pathways for advancing toxic meme detection and interpretation.
Haptic Repurposing with GenAI
Mixed Reality aims to merge the digital and physical worlds to create immersive human-computer interactions. Despite notable advancements, the absence of realistic haptic feedback often breaks the immersive experience by creating a disconnect between visual and tactile perceptions. This paper introduces Haptic Repurposing with GenAI, an innovative approach to enhance MR interactions by transforming any physical objects into adaptive haptic interfaces for AI-generated virtual assets. Utilizing state-of-the-art generative AI models, this system captures both 2D and 3D features of physical objects and, through user-directed prompts, generates corresponding virtual objects that maintain the physical form of the original objects. Through model-based object tracking, the system dynamically anchors virtual assets to physical props in real time, allowing objects to visually morph into any user-specified virtual object. This paper details the system's development, presents findings from usability studies that validate its effectiveness, and explores its potential to significantly enhance interactive MR environments. The hope is this work can lay a foundation for further research into AI-driven spatial transformation in immersive and haptic technologies.
From Complexity to Clarity: How AI Enhances Perceptions of Scientists and the Public's Understanding of Science
This paper evaluated the effectiveness of using generative AI to simplify science communication and enhance the public's understanding of science. By comparing lay summaries of journal articles from PNAS, yoked to those generated by AI, this work first assessed linguistic simplicity across such summaries and public perceptions. Study 1a analyzed simplicity features of PNAS abstracts (scientific summaries) and significance statements (lay summaries), observing that lay summaries were indeed linguistically simpler, but effect size differences were small. Study 1b used a large language model, GPT-4, to create significance statements based on paper abstracts and this more than doubled the average effect size without fine-tuning. Study 2 experimentally demonstrated that simply-written GPT summaries facilitated more favorable perceptions of scientists (they were perceived as more credible and trustworthy, but less intelligent) than more complexly-written human PNAS summaries. Crucially, Study 3 experimentally demonstrated that participants comprehended scientific writing better after reading simple GPT summaries compared to complex PNAS summaries. In their own words, participants also summarized scientific papers in a more detailed and concrete manner after reading GPT summaries compared to PNAS summaries of the same article. AI has the potential to engage scientific communities and the public via a simple language heuristic, advocating for its integration into scientific dissemination for a more informed society.
A Survey on Diffusion Models for Time Series and Spatio-Temporal Data
Yang, Yiyuan, Jin, Ming, Wen, Haomin, Zhang, Chaoli, Liang, Yuxuan, Ma, Lintao, Wang, Yi, Liu, Chenghao, Yang, Bin, Xu, Zenglin, Bian, Jiang, Pan, Shirui, Wen, Qingsong
The study of time series is crucial for understanding trends and anomalies over time, enabling predictive insights across various sectors. Spatio-temporal data, on the other hand, is vital for analyzing phenomena in both space and time, providing a dynamic perspective on complex system interactions. Recently, diffusion models have seen widespread application in time series and spatio-temporal data mining. Not only do they enhance the generative and inferential capabilities for sequential and temporal data, but they also extend to other downstream tasks. In this survey, we comprehensively and thoroughly review the use of diffusion models in time series and spatio-temporal data, categorizing them by model category, task type, data modality, and practical application domain. In detail, we categorize diffusion models into unconditioned and conditioned types and discuss time series and spatio-temporal data separately. Unconditioned models, which operate unsupervised, are subdivided into probability-based and score-based models, serving predictive and generative tasks such as forecasting, anomaly detection, classification, and imputation. Conditioned models, on the other hand, utilize extra information to enhance performance and are similarly divided for both predictive and generative tasks. Our survey extensively covers their application in various fields, including healthcare, recommendation, climate, energy, audio, and transportation, providing a foundational understanding of how these models analyze and generate data. Through this structured overview, we aim to provide researchers and practitioners with a comprehensive understanding of diffusion models for time series and spatio-temporal data analysis, aiming to direct future innovations and applications by addressing traditional challenges and exploring innovative solutions within the diffusion model framework.
Estimating the Hallucination Rate of Generative AI
Jesson, Andrew, Beltran-Velez, Nicolas, Chu, Quentin, Karlekar, Sweta, Kossen, Jannik, Gal, Yarin, Cunningham, John P., Blei, David
This work is about estimating the hallucination rate for in-context learning (ICL) with Generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and asked to make a prediction based on that dataset. The Bayesian interpretation of ICL assumes that the CGM is calculating a posterior predictive distribution over an unknown Bayesian model of a latent parameter and data. With this perspective, we define a \textit{hallucination} as a generated prediction that has low-probability under the true latent parameter. We develop a new method that takes an ICL problem -- that is, a CGM, a dataset, and a prediction question -- and estimates the probability that a CGM will generate a hallucination. Our method only requires generating queries and responses from the model and evaluating its response log probability. We empirically evaluate our method on synthetic regression and natural language ICL tasks using large language models.