Nadine: An LLM-driven Intelligent Social Robot with Affective Capabilities and Human-like Memory

In recent decades, robotics research has expanded significantly, encompassing a wide array of applications spanning from industrial robotics [1, 2] to social robotics[3, 4, 5]. In this work, we focus on social robots, specifically designed for human interaction, which have gained increasing attention due to their potential roles in healthcare, elderly care, and various service industries [6, 7, 8] such as education, museum and finance. This surge in interest has led to extensive research endeavours aimed at enhancing the quality of human-robot interaction (HRI) by refining robot systems to mimic human-like behaviours [9, 10, 11], and exploring the impact of social attributes on HRI dynamics [12, 13, 14] and the influence of the physical appearance of social robots [15, 16, 17, 18]. Large Language Models (LLMs) represent a pivotal advancement in artificial intelligence, trained on vast corpora of textual data, possessing the ability to comprehend intricate linguistic nuances, infer contextual meanings, and generate coherent responses. Their versatility has rendered them indispensable across a spectrum of industries, from healthcare to finance to entertainment. Notably, the integrations of LLMs and robotics have emerged as a transformative trend, offering new avenues for enhancing robot capabilities??. This paper presents a novel robotics system deployed in the social robot Nadine, which comprises three key modules: perception, interaction, and robot control modules. The perception module takes the role of understanding multiple modalities encompassing the user's query and the environmental visual cues.