TL;DR: At SmashLab, we're creating an intelligent assistant that uses the sensors in a smartwatch to support physical tasks such as cooking and DIY. This blog post explores how we use less intrusive scene understanding--compared to cameras--to enable helpful, context-aware interactions for task execution in their daily lives. Every day, we perform many tasks, including cooking, crafting, and medical self-care (like the COVID-19 self-test kit), which involve a series of discrete steps. Accurately executing all the steps can be difficult; when we try a new recipe, for example, we might have questions at any step and might make mistakes by skipping important steps or doing them in the wrong order. This project, Procedural Interaction from Sensing Module (PrISM), aims to support users in executing these kinds of tasks through dialogue-based interactions.

Playing a mob boss in a video game might not seem all that serious for a veteran actor who has appeared on feature films and television. For George Takei, however, performing the role of Masumi Arakawa in "Yakuza: Like a Dragon," is akin to his life coming full circle. Takei was only six years old when he was first introduced to the concept of "benshi" -- Japanese performers who narrated for silent films. It was a discovery that happened to coincide with a dark period in American history. At the time, Takei and his family were staying in Arkansas, part of some 120,000 Japanese-Americans who were gathered into internment camps after Pearl Harbor was bombed on Dec. 7, 1941.