Goto

Collaborating Authors

 onishi


Voice Activity Projection Model with Multimodal Encoders

Saga, Takeshi, Pelachaud, Catherine

arXiv.org Artificial Intelligence

Turn-taking management is crucial for any social interaction. Still, it is challenging to model human-machine interaction due to the complexity of the social context and its multimodal nature. Unlike conventional systems based on silence duration, previous existing voice activity projection (VAP) models successfully utilized a unified representation of turn-taking behaviors as prediction targets, which improved turn-taking prediction performance. Recently, a multimodal VAP model outperformed the previous state-of-the-art model by a significant margin. In this paper, we propose a multimodal model enhanced with pre-trained audio and face encoders to improve performance by capturing subtle expressions. Our model performed competitively, and in some cases, even better than state-of-the-art models on turn-taking metrics. All the source codes and pretrained models are available at https://github.com/sagatake/VAPwithAudioFaceEncoders.


Japanese astronaut Onishi snags first Antares cargo delivery to ISS in two years

The Japan Times

CAPE CANAVERAL, FLORIDA/MIAMI – The International Space Station received its first shipment from a private, Virginia-based company in more than two years Sunday following a sensational nighttime launch observed 250 miles up and down the East Coast. Orbital ATK's cargo ship pulled up at the space station bearing 5,000 pounds of food, equipment and research. "What a beautiful vehicle," said Japanese astronaut Takuya Onishi, who used the station's big robot arm to grab the vessel. The capture occurred as the spacecraft soared 250 miles above Kyrgyzstan; Onishi likened it to the last 195 meters of a marathon. Last Monday's liftoff from Wallops Island was the first by an Antares rocket since a 2014 launch explosion.


Mitsubishi Heavy unveils robot for use when flammable gas has leaked

The Japan Times

Mitsubishi Heavy Industries Ltd. on Tuesday unveiled a robot that can operate in the presence of flammable gases, such as after a gas leak following a disaster. A joint project with Chiba Institute of Technology, the Sakura No. 2 is the country's first mobile inspection unit that can operate in the presence of high concentrations of explosive gases such as methane and hydrogen. There is an increasing need for an inspection robot that is not a fire hazard as Japan steers toward becoming a hydrogen-based society, said Ken Onishi, a senior engineer in charge of the project for Mitsubishi Heavy. "There was a debate over whether to develop robots that can operate near hydrogen gas, as doing so requires an extremely high level of technology," Onishi said. "As we may encounter accidents such as collisions involving hydrogen cars or a truck loaded with hydrogen tanks rolling over inside a road tunnel, we decided to develop a robot that can deal with such situations."