Goto

Collaborating Authors

 Media


Taxonomy and Analysis of Sensitive User Queries in Generative AI Search

arXiv.org Artificial Intelligence

Although there has been a growing interest among industries to integrate generative LLMs into their services, limited experiences and scarcity of resources acts as a barrier in launching and servicing large-scale LLM-based conversational services. In this paper, we share our experiences in developing and operating generative AI models within a national-scale search engine, with a specific focus on the sensitiveness of user queries. We propose a taxonomy for sensitive search queries, outline our approaches, and present a comprehensive analysis report on sensitive queries from actual users.


Deciphering Political Entity Sentiment in News with Large Language Models: Zero-Shot and Few-Shot Strategies

arXiv.org Artificial Intelligence

Sentiment analysis plays a pivotal role in understanding public opinion, particularly in the political domain where the portrayal of entities in news articles influences public perception. In this paper, we investigate the effectiveness of Large Language Models (LLMs) in predicting entity-specific sentiment from political news articles. Leveraging zero-shot and few-shot strategies, we explore the capability of LLMs to discern sentiment towards political entities in news content. Employing a chain-of-thought (COT) approach augmented with rationale in few-shot in-context learning, we assess whether this method enhances sentiment prediction accuracy. Our evaluation on sentiment-labeled datasets demonstrates that LLMs, outperform fine-tuned BERT models in capturing entity-specific sentiment. We find that learning in-context significantly improves model performance, while the self-consistency mechanism enhances consistency in sentiment prediction. Despite the promising results, we observe inconsistencies in the effectiveness of the COT prompting method. Overall, our findings underscore the potential of LLMs in entity-centric sentiment analysis within the political news domain and highlight the importance of suitable prompting strategies and model architectures.


ClickDiffusion: Harnessing LLMs for Interactive Precise Image Editing

arXiv.org Artificial Intelligence

Recently, researchers have proposed powerful systems for generating and manipulating images using natural language instructions. However, it is difficult to precisely specify many common classes of image transformations with text alone. For example, a user may wish to change the location and breed of a particular dog in an image with several similar dogs. This task is quite difficult with natural language alone, and would require a user to write a laboriously complex prompt that both disambiguates the target dog and describes the destination. We propose ClickDiffusion, a system for precise image manipulation and generation that combines natural language instructions with visual feedback provided by the user through a direct manipulation interface. We demonstrate that by serializing both an image and a multi-modal instruction into a textual representation it is possible to leverage LLMs to perform precise transformations of the layout and appearance of an image. Code available at https://github.com/poloclub/ClickDiffusion.


The NES Video-Music Database: A Dataset of Symbolic Video Game Music Paired with Gameplay Videos

arXiv.org Artificial Intelligence

Neural models are one of the most popular approaches for music generation, yet there aren't standard large datasets tailored for learning music directly from game data. To address this research gap, we introduce a novel dataset named NES-VMDB, containing 98,940 gameplay videos from 389 NES games, each paired with its original soundtrack in symbolic format (MIDI). NES-VMDB is built upon the Nintendo Entertainment System Music Database (NES-MDB), encompassing 5,278 music pieces from 397 NES games. Our approach involves collecting long-play videos for 389 games of the original dataset, slicing them into 15-second-long clips, and extracting the audio from each clip. Subsequently, we apply an audio fingerprinting algorithm (similar to Shazam) to automatically identify the corresponding piece in the NES-MDB dataset. Additionally, we introduce a baseline method based on the Controllable Music Transformer to generate NES music conditioned on gameplay clips. We evaluated this approach with objective metrics, and the results showed that the conditional CMT improves musical structural quality when compared to its unconditional counterpart. Moreover, we used a neural classifier to predict the game genre of the generated pieces. Results showed that the CMT generator can learn correlations between gameplay videos and game genres, but further research has to be conducted to achieve human-level performance.


Evaluating Frontier Models for Dangerous Capabilities

arXiv.org Artificial Intelligence

To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning. We do not find evidence of strong dangerous capabilities in the models we evaluated, but we flag early warning signs. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.


Holon: a cybernetic interface for bio-semiotics

arXiv.org Artificial Intelligence

This paper presents an interactive artwork, "Holon", a collection of 130 autonomous, cybernetic organisms that listen and make sound in collaboration with the natural environment. The work was developed for installation on water at a heritage-listed dock in Melbourne, Australia. Conceptual issues informing the work are presented, along with a detailed technical overview of the implementation. Individual holons are of three types, inspired by biological models of animal communication: composer/generators, collector/critics and disruptors. Collectively, Holon integrates and occupies elements of the acoustic spectrum in collaboration with human and non-human agents.


AI Has Lost Its Magic

The Atlantic - Technology

I frequently ask ChatGPT to write poems in the style of the American modernist poet Hart Crane. It does an admirable job of delivering. But the other day, when I instructed the software to give the Crane treatment to a plate of ice-cream sandwiches, I felt bored before I even saw the answer. "The oozing cream, like time, escapes our grasp, / Each moment slipping with a silent gasp." I read the poem, Slacked part of it to a colleague, and closed the window.


'AI-assisted genocide': Israel reportedly used database for Gaza kill lists

Al Jazeera

The Israeli military's reported use of an untested and undisclosed artificial intelligence-powered database to identify targets for its bombing campaign in Gaza has alarmed human rights and technology experts who said it could amount to "war crimes". The Israeli-Palestinian publication 972 Magazine and Hebrew-language media outlet Local Call reported recently that the Israeli army was isolating and identifying thousands of Palestinians as potential bombing targets using an AI-assisted targeting system called Lavender. "That database is responsible for drawing up kill lists of as many as 37,000 targets," Al Jazeera's Rory Challands, reporting from occupied East Jerusalem, said on Thursday. The unnamed Israeli intelligence officials who spoke to the media outlets said Lavender had an error rate of about 10 percent. "But that didn't stop the Israelis from using it to fast-track the identification of often low-level Hamas operatives in Gaza and bombing them," Challands said.


Chinese mourners turn to AI to remember and 'revive' loved ones

The Guardian

As millions of people across China travel to the graves of their ancestors to pay their respects for the annual tomb-sweeping festival, a new way of remembering, and reviving, their beloved relatives is being born. For as little as 20 yuan ( 2.20), Chinese netizens can create a moving digital avatar of their loved one, according to some services advertised online. So this year, to mark tomb-sweeping festival on Thursday, innovative mourners are turning to artificial intelligence to commune with the departed. At the more sophisticated end of the spectrum, the Taiwanese singer Bao Xiaobai used AI to "resurrect" his 22-year-old daughter, who died in 2022. Despite having only an audio recording of her speaking three sentences of English, Bao reportedly spent more than a year experimenting with AI technology before managing to create a video of his daughter singing happy birthday to her mother, which he published in January.


AgentGroupChat: An Interactive Group Chat Simulacra For Better Eliciting Emergent Behavior

arXiv.org Artificial Intelligence

Language significantly influences the formation and evolution of Human emergent behavior, which is crucial in understanding collective intelligence within human societies. Considering that the study of how language affects human behavior needs to put it into the dynamic scenarios in which it is used, we introduce AgentGroupChat in this paper, a simulation that delves into the complex role of language in shaping collective behavior through interactive debate scenarios. Central to this simulation are characters engaging in dynamic conversation interactions. To enable simulation, we introduce the Verbal Strategist Agent, utilizing large language models to enhance interaction strategies by incorporating elements of persona and action. We set four narrative scenarios based on AgentGroupChat to demonstrate the simulation's capacity to mimic complex language use in group dynamics. Evaluations focus on aligning agent behaviors with human expectations and the emergence of collective behaviors within the simulation. Results reveal that emergent behaviors materialize from a confluence of factors: a conducive environment for extensive information exchange, characters with diverse traits, high linguistic comprehension, and strategic adaptability. During discussions on ``the impact of AI on humanity'' in AgentGroupChat simulation, philosophers commonly agreed that ``AI could enhance societal welfare with judicious limitations'' and even come to a conclusion that ``the essence of true intelligence encompasses understanding the necessity to constrain self abilities''. Additionally, in the competitive domain of casting for primary roles in films in AgentGroupChat, certain actors were ready to reduce their remuneration or accept lesser roles, motivated by their deep-seated desire to contribute to the project.