AI-powered app gives voice to people with speech deficits：The Asahi Shimbun

An app powered by artificial intelligence (AI) that can read speakers' lips even if they do not enunciate words is available for those willing to collaborate in a study aimed at facilitating smooth conversation for people with speech difficulties. The lip-reading AI technology, developed by Takeshi Saito, associate professor of intelligence information studies at the Kyushu Institute of Technology, can recognize numbers from zero to nine as well as 15 words or phrases such as "a-ri-ga-to-u" (thank you) and "ha-ji-me-ma-shi-te" (nice to meet you) in the study. The app uses video to determine a user's words based combinations of estimated syllables analyzed by movements of 20 locations around the mouth of the speaker. For example, when the app interprets syllables as "a, i, ga, to, u," it can figure out that the correct word is "a-ri-ga-to-u" because "a, i, ga, to, u" has no meaning in Japanese. As it only analyzes lip movements, the app can determine particular words even when no sounds are produced.