SynVAE AI translates visual artwork into melodies

Synesthesia, which an estimated 1% to 25% of the world's population experience, is a phenomenon in which stimulation of one sensory pathway (such as hearing) leads to involuntary experiences in a second sensory pathway (such as sight). Those with chromesthesia "hear" sounds as color, for instance, while those with lexical-gustatory synesthesia have associations between words and tastes. In a recent study that appears to be at least partly inspired by this area, researchers at the University of Amsterdam investigated an AI system -- Synesthetic Variational Autoencoder, or SynVAE -- capable of mapping characteristics of paintings and other visual art to musical phrases (in the form of MIDI files). They say in a qualitative trial, human evaluators were able to match MIDI files to their muse with accuracies of up to 73%. "Art is experienced as a flow of information between an artist and an observer. Should the latter be visually impaired, however, a barrier appears," wrote the researchers.