Robot learns to use hand gestures including pointing to portray 'me' or 'you' by watching 52 hours of TED talk videos on YouTube

  • Scientists taught a robot how to use human-like hand gestures while speaking
  • The android learned to use a crooked arm action to suggest holding something 
  • Robots that copy human gestures promise to offer more natural interactions

Scientists have taught a robot how to use human-like hand gestures while speaking by feeding it footage of people giving presentations.

The android learned to use a pointing gesture to portray 'you' or 'me', as well as a crooked arm action to suggest holding something.

Building robots that gesticulate like humans will make interactions with them feel more natural, the Korean team behind the technology said.

They built the robot around machine learning software that they showed 52 hours of TED talks - presentations given by expert speakers on various topics.

Scroll down for video

Pictured is a Pepper robot that scientists taught to give human-like hand gestures during speech. The android learned how to give a pointing gesture to portray 'you' or 'me', as well as a crooked arm action to suggest holding something

Pictured is a Pepper robot that scientists taught to give human-like hand gestures during speech. The android learned how to give a pointing gesture to portray 'you' or 'me', as well as a crooked arm action to suggest holding something

'People use co-speech gestures when they talk to others to emphasize speech, show intention, or describe something vividly,' researchers at Electronics and Telecommunications Research Institute in Daejeon, Korea, wrote in their paper.

'Robots can learn co-speech gestures from human behaviours as we ourselves do.

'Mimicking human gestures is a viable strategy for humanoid robots since they have similar appearances and control joints.'

Scientists trained a machine learning programme to match hand gestures to words and phrases used by Ted talk speakers.

Machine learning programmes work by sifting through reams of data to learn patterns, such as movements in a video.

Once the software had learned to accurately match gestures with phrases, the team installed it into a humanoid robot.

Many androids already use hand gestures, but these actions are pre-programmed by humans, which takes hours to code.

The robot's repertoire is also limited to those programmed by its designers, leaving a tiny gesture set compared to the broad range used by humans. 

To avoid this, researchers programmed the new robot to learn the natural gestures of humans used during speech.

Experts built the robot around machine learning software that they showed 52 hours of TED talks - presentations given by expert speakers on various topics. Pictured is a still from the software that mapped gestures given by speakers

Experts built the robot around machine learning software that they showed 52 hours of TED talks - presentations given by expert speakers on various topics. Pictured is a still from the software that mapped gestures given by speakers

They mapped the speaker's pose for each frame of the video, including the position of the head, arms and hands, and matched this to what they were saying.

The result was a robot capable of giving complex gestures, including an open armed action to suggest inclusiveness, and point to delineate 'me' and 'you'.

To test the system, researchers had 46 volunteers compare the gestures of the new robot to those produced by current technology.

Participants judged the learned actions to be more friendly, more human-like and a better match to the words spoken than those of other modern robots.

Researchers plan to personalise the hand movements so the robots don't all use the same gestures in future. 

HOW DOES ARTIFICIAL INTELLIGENCE LEARN?

AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn.

ANNs can be trained to recognise patterns in information - including speech, text data, or visual images - and are the basis for a large number of the developments in AI over recent years.

Conventional AI uses input to 'teach' an algorithm about a particular subject by feeding it massive amounts of information.   

AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn. ANNs can be trained to recognise patterns in information - including speech, text data, or visual images

AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn. ANNs can be trained to recognise patterns in information - including speech, text data, or visual images

Practical applications include Google's language translation services, Facebook's facial recognition software and Snapchat's image altering live filters.

The process of inputting this data can be extremely time consuming, and is limited to one type of knowledge. 

A new breed of ANNs called Adversarial Neural Networks pits the wits of two AI bots against each other, which allows them to learn from each other. 

This approach is designed to speed up the process of learning, as well as refining the output created by AI systems. 

Advertisement

Robot learns to use hand gestures including pointing by watching 52 hours of TED talk videos