Traditional rule-based conversational robots, constrained by predefined scripts and static response mappings, fundamentally lack adaptability for personalized, long-term human interaction. While Large Language Models (LLMs) like GPT-4 have revolutionized conversational AI through open-domain capabilities, current social robots implementing LLMs still lack emotional awareness and continuous personalization. This dual limitation hinders their ability to sustain engagement across multiple interaction sessions. We bridge this gap with PERCY (Personal Emotional Robotic Conversational sYstem), a system designed to enable open-domain, multi-turn dialogues by dynamically analyzing users' real-time facial expressions and vocabulary to tailor responses based on their emotional state. Built on a ROS-based multimodal framework, PERCY integrates a fine-tuned GPT-4 reasoning engine, combining textual sentiment analysis with visual emotional cues to accurately assess and respond to user emotions. We evaluated PERCY's performance through various dialogue quality metrics, showing strong coherence, relevance, and diversity. Human evaluations revealed PERCY's superior personalization and comparable naturalness to other models. This work highlights the potential for integrating advanced multimodal perception and personalization in social robot dialogue systems.

The integration of conversational agents into our daily lives has become increasingly common, yet many of these agents cannot engage in deep interactions with humans. Despite this, there is a noticeable shortage of datasets that capture multimodal information from human-robot interaction dialogues. To address this gap, we have developed a Personal Emotional Robotic Conversational sYstem (PERCY) and recorded a novel multimodal dataset that encompasses rich embodied interaction data. The process involved asking participants to complete a questionnaire and gathering their profiles on ten topics, such as hobbies and favourite music. Subsequently, we initiated conversations between the robot and the participants, leveraging GPT-4 to generate contextually appropriate responses based on the participant's profile and emotional state, as determined by facial expression recognition and sentiment analysis. Automatic and user evaluations were conducted to assess the overall quality of the collected data. The results of both evaluations indicated a high level of naturalness, engagement, fluency, consistency, and relevance in the conversation, as well as the robot's ability to provide empathetic responses. It is worth noting that the dataset is derived from genuine interactions with the robot, involving participants who provided personal information and conveyed actual emotions.

When Virginia Woolf wrote this innocuous sentence in "A Room of One's Own," her foundational work of feminist criticism, she opened the door to another field, still decades in the future--that of queer literary criticism. Do not blush," Woolf cautioned her audience. "Let us admit in the privacy of our own society that these things sometimes happen. Sometimes women do like women." Chloe and Olivia are characters in a book that Woolf has invented, a mediocre novel by a writer she names Mary Carmichael. Ostensibly, the women are friends and colleagues, not lovers, but Woolf drops clues for attentive readers. At one point, she interrupts her train of thought to ask for reassurance that Sir Chartres Biron is not lurking somewhere in the room. When she gave her original talks, Biron had recently been appointed the chief magistrate in an obscenity case that had been brought against the publisher of Radclyffe Hall's "The Well of Loneliness," a novel about a girl named Stephen who wants to be ...