Goto

Collaborating Authors

 emu


Evaluating and comparing gender bias across four text-to-image models

Hammad, Zoya, Sowah, Nii Longdon

arXiv.org Artificial Intelligence

SUMMARY As we increasingly use Artificial Intelligence (AI) in decision-making for industries like healthcare, finance, e-commerce, and even entertainment, it is crucial to also reflect on the ethical aspects of AI, for example the inclusivity and fairness of the information it provides. In this work, we aimed to evaluate different text-to-image AI models and compare the degree of gender bias they present. The evaluated models were Stable Diffusion XL (SDXL), Stable Diffusion Cascade (SC), DALL-E and Emu. We hypothesized that DALL-E and Stable Diffusion, which are comparatively older models, would exhibit a noticeable degree of gender bias towards men, while Emu, which was recently released by Meta AI, would have more balanced results. As hypothesized, we found that both Stable Diffusion models exhibit a noticeable degree of gender bias while Emu demonstrated more balanced results (i.e less gender bias). However, interestingly, Open AI's DALL-E exhibited almost opposite results, such that the ratio of women to men was significantly higher in most cases tested. Here, although we still observed a bias, the bias favored females over males. This bias may be explained by the fact that OpenAI changed the prompts at its backend, as observed during our experiment. We also observed that Emu from Meta AI utilized user information while generating images via WhatsApp. We also proposed some potential solutions to avoid such biases, including ensuring diversity across AI research teams and having diverse datasets. INTRODUCTION Artificial Intelligence (AI) has been growing remarkably in recent years, impacting numerous aspects of our daily lives. One such area of significant advancement is text-to-image generation.


Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation

Da, Jeff, Forbes, Maxwell, Zellers, Rowan, Zheng, Anthony, Hwang, Jena D., Bosselut, Antoine, Choi, Yejin

arXiv.org Artificial Intelligence

Multimodal disinformation, from 'deepfakes' to simple edits that deceive, is an important societal problem. Yet at the same time, the vast majority of media edits are harmless -- such as a filtered vacation photo. The difference between this example, and harmful edits that spread disinformation, is one of intent. Recognizing and describing this intent is a major challenge for today's AI systems. We present the task of Edited Media Understanding, requiring models to answer open-ended questions that capture the intent and implications of an image edit. We introduce a dataset for our task, EMU, with 48k question-answer pairs written in rich natural language. We evaluate a wide variety of vision-and-language models for our task, and introduce a new model PELICAN, which builds upon recent progress in pretrained multimodal representations. Our model obtains promising results on our dataset, with humans rating its answers as accurate 40.35% of the time. At the same time, there is still much work to be done -- humans prefer human-annotated captions 93.56% of the time -- and we provide analysis that highlights areas for further progress.


Pixel-Space Post-Training of Latent Diffusion Models

Zhang, Christina, Motwani, Simran, Yu, Matthew, Hou, Ji, Juefei-Xu, Felix, Tsai, Sam, Vajda, Peter, He, Zijian, Wang, Jialiang

arXiv.org Artificial Intelligence

Latent diffusion models (LDMs) have made significant advancements in the field of image generation in recent years. One major advantage of LDMs is their ability to operate in a compressed latent space, allowing for more efficient training and deployment. However, despite these advantages, challenges with LDMs still remain. For example, it has been observed that LDMs often generate high-frequency details and complex compositions imperfectly. We hypothesize that one reason for these flaws is due to the fact that all pre- and post-training of LDMs are done in latent space, which is typically $8 \times 8$ lower spatial-resolution than the output images. To address this issue, we propose adding pixel-space supervision in the post-training process to better preserve high-frequency details. Experimentally, we show that adding a pixel-space objective significantly improves both supervised quality fine-tuning and preference-based post-training by a large margin on a state-of-the-art DiT transformer and U-Net diffusion models in both visual quality and visual flaw metrics, while maintaining the same text alignment quality.


AI-Driven Strategies for Reducing Student Withdrawal -- A Study of EMU Student Stopout

Zhao, Yan, Otteson, Amy

arXiv.org Artificial Intelligence

Not everyone who enrolls in college will leave with a certificate or degree, but the number of people who drop out or take a break is much higher than experts previously believed. In December 2013, there were 29 million people with some college education but no degree. That number jumped to 36 million by December of 2018, according to a new report from the National Student Clearinghouse Research Center[1]. It is imperative to understand the underlying factors contributing to student withdrawal and to assist decision-makers to identify effective strategies to prevent it. By analyzing the characteristics and educational pathways of the stopout student population, our aim is to provide actionable insights that can benefit institutions facing similar challenges. Eastern Michigan University (EMU) faces significant challenges in student retention, with approximately 55% of its undergraduate students not completing their degrees within six years. As an institution committed to student success, EMU conducted a comprehensive study of student withdrawals to understand the influencing factors. And the paper revealed a high correlation between certain factors and withdrawals, even in the early stages of university attendance. Based on these findings, we developed a predictive model that employs artificial intelligence techniques to assess the potential risk that students abandon their studies. These models enable universities to implement early intervention strategies, support at-risk students, and improve overall higher education success.


Expectable Motion Unit: Avoiding Hazards From Human Involuntary Motions in Human-Robot Interaction

Kirschner, Robin Jeanne, Mayer, Henning, Burr, Lisa, Mansfeld, Nico, Abdolshah, Saeed, Haddadin, Sami

arXiv.org Artificial Intelligence

In robotics, many control and planning schemes have been developed to ensure human physical safety in human-robot interaction. The human psychological state and the expectation towards the robot, however, are typically neglected. Even if the robot behaviour is regarded as biomechanically safe, humans may still react with a rapid involuntary motion (IM) caused by a startle or surprise. Such sudden, uncontrolled motions can jeopardize safety and should be prevented by any means. In this letter, we propose the Expectable Motion Unit (EMU), which ensures that a certain probability of IM occurrence is not exceeded in a typical HRI setting. Based on a model of IM occurrence generated through an experiment with 29 participants, we establish the mapping between robot velocity, robot-human distance, and the relative frequency of IM occurrence. This mapping is processed towards a real-time capable robot motion generator that limits the robot velocity during task execution if necessary. The EMU is combined in a holistic safety framework that integrates both the physical and psychological safety knowledge. A validation experiment showed that the EMU successfully avoids human IM in five out of six cases.


Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learning

Na, Hyungho, Seo, Yunkyeong, Moon, Il-chul

arXiv.org Artificial Intelligence

In cooperative multi-agent reinforcement learning (MARL), agents aim to achieve a common goal, such as defeating enemies or scoring a goal. Existing MARL algorithms are effective but still require significant learning time and often get trapped in local optima by complex tasks, subsequently failing to discover a goal-reaching policy. To address this, we introduce Efficient episodic Memory Utilization (EMU) for MARL, with two primary objectives: (a) accelerating reinforcement learning by leveraging semantically coherent memory from an episodic buffer and (b) selectively promoting desirable transitions to prevent local convergence. To achieve (a), EMU incorporates a trainable encoder/decoder structure alongside MARL, creating coherent memory embeddings that facilitate exploratory memory recall. To achieve (b), EMU introduces a novel reward structure called episodic incentive based on the desirability of states. This reward improves the TD target in Q-learning and acts as an additional incentive for desirable transitions. We provide theoretical support for the proposed incentive and demonstrate the effectiveness of EMU compared to conventional episodic control. The proposed method is evaluated in StarCraft II and Google Research Football, and empirical results indicate further performance improvement over state-of-the-art methods. Our code is available at: https://github.com/HyunghoNa/EMU.


Google buys Emu, opening the door to make money off your chats

AITopics Original Links

Google is looking to become a leader in instant-messaging with its purchase of Emu. The Emu instant-message system, Wired reports, can monitor chats, determine what people are discussing, and then insert links it deems helpful to users. Of course, Google could easily use this feature to get ads in front of users. In a blog post, Emu announced that it would be joining the search giant and closing down its app as of August 25. Starting then, Emu will no longer be available in the App Store and current users will no longer be able to send, receive, or download messages using the app. Although the purchase of Emu has been confirmed, it's currently unclear how much Google paid in the deal.