A Multimodal Symphony: Integrating Taste and Sound through Generative AI

Over recent years, the rapid evolution and progress of generative models have opened new possibilities in manipulating images, audio, and text, both independently and in a multimodal context. These AI advancements have ignited considerable debate about the essence of these human-engineered "intelligences". Critics have termed large language models (LLMs) as "statistical parrots" (Bender et al., 2021) due to their reliance on data. However, others view them as advanced tools capable of emulating and exploring the intricate structures of the human brain (Zhao et al., 2023; Abbasiantaeb et al., 2024; Fayyaz et al., 2024). Despite this division, it has become increasingly clear that limiting these models to a few specialized areas greatly restricts their potential to fully grasp and portray the complexity of the world. Therefore the integration of sensory modalities through technology, particularly using AI, has emerged as a compelling frontier in computer science and cognitive research (Murari et al., 2020; Turato et al., 2022). As multimodal AI models advance, they increasingly offer innovative solutions for bridging human experiences and machine understanding across diverse sensory domains. These models, which merge information from different modalities enable machines to interpret complex real-world scenarios and provide more nuanced outputs. While recent research has predominantly focused on the intersection of audio and visual modalities, the potential for integrating taste and sound remains relatively unexplored.