USING A computer used to mean bashing away at a keyboard. Then it meant tapping on a touchscreen. Increasingly, it means simply speaking. Over 100m devices powered by Alexa, Amazon's voice assistant, rest on the world's shelves. Apple's offering, Siri, processes 25bn requests a month. By 2025 the market for such technology could be worth more than $27bn.

Millions of people communicate using sign language, but so far projects to capture its complex gestures and translate them to verbal speech have had limited success. A new advance in real-time hand tracking from Google's AI labs, however, could be the breakthrough some have been waiting for. The new technique uses a few clever shortcuts and, of course, the increasing general efficiency of machine learning systems to produce, in real time, a highly accurate map of the hand and all its fingers, using nothing but a smartphone and its camera. "Whereas current state-of-the-art approaches rely primarily on powerful desktop environments for inference, our method achieves real-time performance on a mobile phone, and even scales to multiple hands," write Google researchers Valentin Bazarevsky and Fan Zhang in a blog post. "Robust real-time hand perception is a decidedly challenging computer vision task, as hands often occlude themselves or each other (e.g.

Translating is difficult work, the more so the further two languages are from one another. But sign language is a unique case, and translating it uniquely difficult, because it is fundamentally different from spoken and written languages. All the same, SignAll has been working hard for years to make accurate, real-time machine translation of ASL a reality. One would think that with all the advances in AI and computer vision happening right now, a problem as interesting and beneficial to solve as this would be under siege by the best of the best. Even thinking about it from a cynical market-expansion point of view, an Echo or TV that understands sign language could attract millions of new (and very thankful) customers.

Machine translation systems that convert sign language into text and back again are helping people who are deaf or have difficulty hearing to communicate with those who cannot sign. KinTrans, a start-up based in Dallas, Texas, is trialling its technology in a bank and government offices in the United Arab Emirates, and plans to install it in more places over the next couple of months. SignAll, a company based in Budapest, Hungary, will begin its own trials next year. KinTrans uses a 3D camera to track the movement of a person's hands as they sign words. A sign language user can approach a bank teller and sign to the KinTrans camera that they'd like assistance, for example.