Earphone tracks facial expressions, even with a face mask

Cornell researchers have invented an earphone that can continuously track full facial expressions by observing the contour of the cheeks – and can then translate expressions into emojis or silent speech commands. With the ear-mounted device, called C-Face, users could express emotions to online collaborators without holding cameras in front of their faces – an especially useful communication tool as much of the world engages in remote work or learning. "This device is simpler, less obtrusive and more capable than any existing ear-mounted wearable technologies for tracking facial expressions," said Cheng Zhang, assistant professor of information science and senior author of "C-Face: Continuously Reconstructing Facial Expressions by Deep Learning Contours of the Face With Ear-Mounted Miniature Cameras." The paper will be presented at the Association for Computing Machinery Symposium on User Interface Software and Technology, to be held virtually Oct. 20-23. "In previous wearable technology aiming to recognize facial expressions, most solutions needed to attach sensors on the face," said Zhang, director of Cornell's SciFi Lab, "and even with so much instrumentation, they could only recognize a limited set of discrete facial expressions."