Researchers use AI to identify similar materials in images

A robot manipulating objects while, say, working in a kitchen, will benefit from understanding which items are composed of the same materials. With this knowledge, the robot would know to exert a similar amount of force whether it picks up a small pat of butter from a shadowy corner of the counter or an entire stick from inside the brightly lit fridge. Identifying objects in a scene that are composed of the same material, known as material selection, is an especially challenging problem for machines because a material's appearance can vary drastically based on the shape of the object or lighting conditions. Scientists at MIT and Adobe Research have taken a step toward solving this challenge. They developed a technique that can identify all pixels in an image representing a given material, which is shown in a pixel selected by the user. The method is accurate even when objects have varying shapes and sizes, and the machine-learning model they developed isn't tricked by shadows or lighting conditions that can make the same material appear different.