Collaborating Authors

5 reasons robots aren't going to take over the world


There is no technical match for the magnificent human and animal skin that encompasses a variety of tactile sensors. This perception is required for complex manipulation. Also, the software that processes the input from the sensors in robots is nowhere near as sophisticated as the human brain when it comes to interpretation and reaction to the messages received from the tactile sensors.

This carpet can feel exactly what you're doing on it using only tactile sensors


MIT CSAIL are testing an intelligent carpet that accurately estimates three-dimensional human poses using only tactile sensors. The long-term goal is to use this system for health monitoring purposes, like rehabilitation and fall detection.

DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation Machine Learning

Despite decades of research, general purpose in-hand manipulation remains one of the unsolved challenges of robotics. One of the contributing factors that limit current robotic manipulation systems is the difficulty of precisely sensing contact forces -- sensing and reasoning about contact forces are crucial to accurately control interactions with the environment. As a step towards enabling better robotic manipulation, we introduce DIGIT, an inexpensive, compact, and high-resolution tactile sensor geared towards in-hand manipulation. DIGIT improves upon past vision-based tactile sensors by miniaturizing the form factor to be mountable on multi-fingered hands, and by providing several design improvements that result in an easier, more repeatable manufacturing process, and enhanced reliability. We demonstrate the capabilities of the DIGIT sensor by training deep neural network model-based controllers to manipulate glass marbles in-hand with a multi-finger robotic hand. To provide the robotic community access to reliable and low-cost tactile sensors, we open-source the DIGIT design at

Interactive Perception at Toyota Research Institute


Dr. Carolyn Matl, Research Scientist at Toyota Research Institute, explains why Interactive Perception and soft tactile sensors are critical for manipulating challenging objects such as liquids, grains, and dough. She also dives into "StRETcH" a Soft to Resistive Elastic Tactile Hand, a variable stiffness soft tactile end-effector, presented by her research group. Carolyn Matl is a research scientist at the Toyota Research Institute, where she works on robotic perception and manipulation with the Mobile Manipulation Team. She received her B.S.E in Electrical Engineering from Princeton University in 2016, and her Ph.D. in Electrical Engineering and Computer Sciences at the University of California, Berkeley in 2021. At Berkeley, she was awarded the NSF Graduate Research Fellowship and was advised by Ruzena Bajcsy. Her dissertation work focused on developing and leveraging non-traditional sensors for robotic manipulation of complicated objects and substances like liquids and doughs. Would you mind introducing yourself? Thank you so much for having me on the podcast. I'm Carolyn Matl and I'm a research scientist at the Toyota research Institute where I work with a really great group of people on the mobile manipulation team on fun and challenging robotic perception and manipulation problems.

Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear Artificial Intelligence

Robotic touch, particularly when using soft optical tactile sensors, suffers from distortion caused by motion-dependent shear. The manner in which the sensor contacts a stimulus is entangled with the tactile information about the geometry of the stimulus. In this work, we propose a supervised convolutional deep neural network model that learns to disentangle, in the latent space, the components of sensor deformations caused by contact geometry from those due to sliding-induced shear. The approach is validated by reconstructing unsheared tactile images from sheared images and showing they match unsheared tactile images collected with no sliding motion. In addition, the unsheared tactile images give a faithful reconstruction of the contact geometry that is not possible from the sheared data, and robust estimation of the contact pose that can be used for servo control sliding around various 2D shapes. Finally, the contact geometry reconstruction in conjunction with servo control sliding were used for faithful full object reconstruction of various 2D shapes. The methods have broad applicability to deep learning models for robots with a shear-sensitive sense of touch.