Goto

Collaborating Authors

 elena


Anatomy of a Feeling: Narrating Embodied Emotions via Large Vision-Language Models

Saim, Mohammad, Duong, Phan Anh, Luong, Cat, Bhanderi, Aniket, Jiang, Tianyu

arXiv.org Artificial Intelligence

The embodiment of emotional reactions from body parts contains rich information about our affective experiences. We propose a framework that utilizes state-of-the-art large vision-language models (LVLMs) to generate Embodied LVLM Emotion Narratives (ELENA). These are well-defined, multi-layered text outputs, primarily comprising descriptions that focus on the salient body parts involved in emotional reactions. We also employ attention maps and observe that contemporary models exhibit a persistent bias towards the facial region. Despite this limitation, we observe that our employed framework can effectively recognize embodied emotions in face-masked images, outperforming baselines without any fine-tuning. ELENA opens a new trajectory for embodied emotion analysis across the modality of vision and enriches modeling in an affect-aware setting.


ELENA: Epigenetic Learning through Evolved Neural Adaptation

Kriuk, Boris, Sulamanidze, Keti, Kriuk, Fedor

arXiv.org Artificial Intelligence

Optimization of complex networks is one of the fundamental challenges in computer science research. With the progression of computational resources availability, a great variety of conceptually different algorithms have been presented over the past decades to achieve competitive results in the domain of network optimization. Many approaches, such as Lin-Kernighan-Helsgaun heuristic [1], Genetic Algorithm variations [2,3,4], Ant Colony Optimization [5], k-opt local search [6,7] with sequential improvements have gained acknowledgment from both research community and industry across logistics, telecommunications, and biotechnology verticals. The Traveling Salesman Problem (TSP) [8], first formalized by Karl Menger in 1930, remains a cornerstone problem that has driven network optimization algorithmic innovations for decades. The Vehicle Routing Problem (VRP) [9,10], introduced by Dantzig and Ramser in 1959, extends TSP's complexity by incorporating multiple vehicles and capacity constraints, finding direct applications in logistics and delivery. The Maximum Clique Problem (MCP) [11], important for social network analysis, computational biochemistry and wireless network allocation, focuses on finding the largest complete subgraph within a network.


BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model

Yu, Yeyong, Yu, Runsheng, Wei, Haojie, Zhang, Zhanqiu, Qian, Quan

arXiv.org Artificial Intelligence

The rapid advancement of large language models (LLMs) has revolutionized role-playing, enabling the development of general role-playing models. However, current role-playing training has two significant issues: (I) Using a predefined role profile to prompt dialogue training for specific scenarios usually leads to inconsistencies and even conflicts between the dialogue and the profile, resulting in training biases. (II) The model learns to imitate the role based solely on the profile, neglecting profile-dialogue alignment at the sentence level. In this work, we propose a simple yet effective framework called BEYOND DIALOGUE, designed to overcome these hurdles. This framework innovatively introduces "beyond dialogue" tasks to align dialogue with profile traits based on each specific scenario, thereby eliminating biases during training. Furthermore, by adopting an innovative prompting mechanism that generates reasoning outcomes for training, the framework allows the model to achieve fine-grained alignment between profile and dialogue at the sentence level. The aforementioned methods are fully automated and low-cost. Additionally, the integration of automated dialogue and objective evaluation methods forms a comprehensive framework, paving the way for general role-playing. Experimental results demonstrate that our model excels in adhering to and reflecting various dimensions of role profiles, outperforming most proprietary general and specialized role-playing baselines. All code and datasets are available at https://github.com/yuyouyu32/BeyondDialogue.


Collaborative Quest Completion with LLM-driven Non-Player Characters in Minecraft

Rao, Sudha, Xu, Weijia, Xu, Michael, Leandro, Jorge, Lobb, Ken, DesGarennes, Gabriel, Brockett, Chris, Dolan, Bill

arXiv.org Artificial Intelligence

The use of generative AI in video game development is on the rise, and as the conversational and other capabilities of large language models continue to improve, we expect LLM-driven non-player characters (NPCs) to become widely deployed. In this paper, we seek to understand how human players collaborate with LLM-driven NPCs to accomplish in-game goals. We design a minigame within Minecraft where a player works with two GPT4-driven NPCs to complete a quest. We perform a user study in which 28 Minecraft players play this minigame and share their feedback. On analyzing the game logs and recordings, we find that several patterns of collaborative behavior emerge from the NPCs and the human players. We also report on the current limitations of language-only models that do not have rich game-state or visual understanding. We believe that this preliminary study and analysis will inform future game developers on how to better exploit these rapidly improving generative AI models for collaborative roles in games.


College of Engineering Awards

University of Washington Computer Science

The College of Engineering Awards acknowledge the extraordinary efforts of the college's teaching and research assistants, staff, and faculty members. The College of Engineering Awards ceremony scheduled for April 20 has been canceled. Since joining UW in 2014, Cole DeForest has established himself as an innovative researcher, an effective teacher and a collaborative colleague, holding appointments in Chemical Engineering, Bioengineering, and the Institute for Stem Cell & Regenerative Medicine. His research focuses on the development of (de)polymerization reactions that can be triggered using light in the presence of cells, and "represents a major advancement in cell culture niches that allow unprecedented control of the cellular microenvironment, and is enabling him to conduct newfound experiments that were previously impossible." Cole has received numerous honors, including an NSF Career Award, a Young Investigator Award through the American Chemical Society, and a UW Presidential Distinguished Teaching Award.


Afterwork event – Women in Big Data in collaboration with intel @intel Munich

#artificialintelligence

The WiBD Afterwork event took place at Intel in Munich during the Oktoberfest, the famous celebration joined by millions of people from around the world who come to Bavaria to enjoy beer and a unique atmosphere. We had around one hundred attendees who enjoyed our event with us while the rest of the city was celebrating. The event kicked off as usual with an introduction about Women in Big Data from the organisers Nahia Orduna and Pat Piritburana, who presented the vision of the organisation and the Munich Chapter. The first speaker was Oliver Buschmann, Client Business and Platform Planning at Intel Corporation. He delivered a presentation around "AI: from Buzzword to reality".


Syntactic Analysis Based on Morphological Characteristic Features of the Romanian Language

Patrut, Bogdan

arXiv.org Artificial Intelligence

This paper refers to the syntactic analysis of phrases in Romanian, as an important process of natural language processing. We will suggest a real-time solution, based on the idea of using some words or groups of words that indicate grammatical category; and some specific endings of some parts of sentence. Our idea is based on some characteristics of the Romanian language, where some prepositions, adverbs or some specific endings can provide a lot of information about the structure of a complex sentence. Such characteristics can be found in other languages, too, such as French. Using a special grammar, we developed a system (DIASEXP) that can perform a dialogue in natural language with assertive and interogative sentences about a "story" (a set of sentences describing some events from the real life).