Goto

Collaborating Authors

 Generative AI


Auditing and Mitigating Cultural Bias in LLMs

arXiv.org Artificial Intelligence

Culture fundamentally shapes people's reasoning, behavior, and communication. Generative artificial intelligence (AI) technologies may cause a shift towards a dominant culture. As people increasingly use AI to expedite and even automate various professional and personal tasks, cultural values embedded in AI models may bias authentic expression. We audit large language models for cultural bias, comparing their responses to nationally representative survey data, and evaluate country-specific prompting as a mitigation strategy. We find that GPT-4, 3.5 and 3 exhibit cultural values resembling English-speaking and Protestant European countries. Our mitigation strategy reduces cultural bias in recent models but not for all countries/territories. To avoid cultural bias in generative AI, especially in high-stakes contexts, we suggest using culture matching and ongoing cultural audits.


PortfolioMentor: Multimodal Generative AI Companion for Learning and Crafting Interactive Digital Art Portfolios

arXiv.org Artificial Intelligence

Digital art portfolios serve as impactful mediums for artists to convey their visions, weaving together visuals, audio, interactions, and narratives. However, without technical backgrounds, design students often find it challenging to translate creative ideas into tangible codes and designs, given the lack of tailored resources for the non-technical, academic support in art schools, and a comprehensive guiding tool throughout the mentally demanding process. Recognizing the role of companionship in code learning and leveraging generative AI models' capabilities in supporting creative tasks, we present PortfolioMentor, a coding companion chatbot for IDEs. This tool guides and collaborates with students through proactive suggestions and responsible Q&As for learning, inspiration, and support. In detail, the system starts with the understanding of the task and artist's visions, follows the co-creation of visual illustrations, audio or music suggestions and files, click-scroll effects for interactions, and creative vision conceptualization, and finally synthesizes these facets into a polished interactive digital portfolio.


Exploration with Principles for Diverse AI Supervision

arXiv.org Artificial Intelligence

Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. Even state-of-the-art AI models like ChatGPT depend on fine-tuning through human demonstrations, demanding extensive human input and domain expertise. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. To address this limitation, we propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data. Drawing inspiration from unsupervised reinforcement learning (RL) pretraining, EAI achieves exploration within the natural language space. We accomplish this by harnessing large language models to assess the novelty of generated content. Our approach employs two key components: an actor that generates novel content following exploration principles and a critic that evaluates the generated content, offering critiques to guide the actor. Empirical evaluations demonstrate that EAI significantly boosts model performance on complex reasoning tasks, addressing the limitations of human-intensive supervision.


HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models

arXiv.org Artificial Intelligence

In recent years, Text-to-Image (T2I) models have been extensively studied, especially with the emergence of diffusion models that achieve state-of-the-art results on T2I synthesis tasks. However, existing benchmarks heavily rely on subjective human evaluation, limiting their ability to holistically assess the model's capabilities. Furthermore, there is a significant gap between efforts in developing new T2I architectures and those in evaluation. To address this, we introduce HRS-Bench, a concrete evaluation benchmark for T2I models that is Holistic, Reliable, and Scalable. Unlike existing bench-marks that focus on limited aspects, HRS-Bench measures 13 skills that can be categorized into five major categories: accuracy, robustness, generalization, fairness, and bias. In addition, HRS-Bench covers 50 scenarios, including fashion, animals, transportation, food, and clothes. We evaluate nine recent large-scale T2I models using metrics that cover a wide range of skills. A human evaluation aligned with 95% of our evaluations on average was conducted to probe the effectiveness of HRS-Bench. Our experiments demonstrate that existing models often struggle to generate images with the desired count of objects, visual text, or grounded emotions. We hope that our benchmark help ease future text-to-image generation research. The code and data are available at https://eslambakr.github.io/hrsbench.github.io


Sam Altman's Second Coming Sparks New Fears of the AI Apocalypse?

WIRED

Open AI's new boss is the same as the old boss. But the company--and the artificial intelligence industry--may have been profoundly changed by the past five days of high-stakes soap opera. Sam Altman, OpenAI's CEO, cofounder and figurehead, was removed by the board of directors on Friday. By Tuesday night, after a mass protest by the majority of the startup's staff, Altman was on his way back, and most of the existing board was gone. But that board, mostly independent of OpenAI's operations, bound to a "for the good of humanity" mission statement, was critical to the company's uniqueness.


The Failed OpenAI Coup Changes Everything

Slate

This article is from Big Technology, a newsletter by Alex Kantrowitz. Improbably and dramatically, the ex-OpenAI CEO returned as CEO late Tuesday. Altman's counter-coup swept out three board members who sparked his firing and included an agreement to investigate what went down this past weekend. The new board--which now includes Larry Summers and Bret Taylor--will expand to up to nine members and likely include someone from Microsoft. The A.I. field will not go back to "normal" after this. OpenAI was already vulnerable coming into the chaos and will now have to work harder to maintain its lead while facing inspired competition.



What the Firing and Rehiring of Sam Altman Actually Means

Slate

Folks, if you predicted on Friday that the closely watched OpenAI power struggle would end in the most pointless-seeming way possible โ€ฆ well, just look. Late Tuesday night, four days after CEO Sam Altman's shocking ouster from the A.I. company, we found ourselves (mostly) back where we started: Altman is returning to OpenAI as its CEO, albeit not to its board of directors; Greg Brockman is once again president of OpenAI, but also will not be a member of the board; Mira Murati, who briefly took the helm as interim CEO, is just regular ol' CTO again; the three researchers who'd stepped down Friday in solidarity with Altman and Brockman are either back at the company or requesting to return; Altman & co. will once again operate with the backing of Microsoft, not as direct employees of the Big Tech pioneer. When it comes to the Main Characters of this saga and their loyalists, it seems most everyone's pretty happy. "[W]e are so back," Brockman exclaimed, sharing a selfie with his smiling team (who celebrated, according to the Information's Erin Woo, by setting off a false fire alarm at OpenAI HQ). Twitch co-founder Emmett Shear is no longer interim CEO but is "deeply pleased by this result, after 72 very intense hours of work," and is "glad to have been a part of the solution."


Mods Are Asleep. Quick, Everyone Release AI Products

WIRED

The turmoil at OpenAI over the past five days has captivated the tech industry and kept entrepreneurs, journalists, and anyone who still has an X account glued to their timelines for the latest emoji updates and lower-case missives. In the meantime, some of the most prominent AI companies--including OpenAI--continued to do what Silicon Valley is known for: Drop new products. The unexpected firing of Sam Altman, OpenAI's CEO, was followed by an avalanche of new AI features from competitors, including Anthropic and Stable Diffusion. On Tuesday afternoon, in the midst of turmoil, OpenAI rolled out ChatGPT with voice capabilities for free to all users. OpenAI had pre-released this in late September, but only for paid users.


Fox News AI Newsletter: Ousted CEO returns to ChatGPT maker OpenAI

FOX News

Sam Altman, chief executive officer of OpenAI, during a fireside chat at University College London (UCL) in London, UK, on Wednesday, May 24, 2023. Altman said part of the reason for his current tour of European cities is to discover a suitable location for a new office. ALTMAN RETURNS: OpenAI brings back former CEO, establishes new board days after ouster. EGG ON FACE: OpenAI board's days numbered as $90B company plunges into chaos. NO OVERSIGHT: Tech CEO's ouster demonstrates need for better regulation.