Goto

Collaborating Authors

 computer vision algorithm identify image


MIT's newest computer vision algorithm identifies images down to the pixel

Engadget

For humans, identifying items in a scene -- whether that's an avocado or an Aventador, a pile of mashed potatoes or an alien mothership -- is as simple as looking at them. But for artificial intelligence and computer vision systems, developing a high-fidelity understanding of their surroundings takes a bit more effort. To help machines better see the way people do, a team of researchers at MIT CSAIL in collaboration with Cornell University and Microsoft have developed STEGO, an algorithm able to identify images down to the individual pixel. Normally, creating CV training data involves a human drawing boxes around specific objects within an image -- say, a box around the dog sitting in a field of grass -- and labeling those boxes with what's inside ("dog"), so that the AI trained on it will be able to tell the dog from the grass. STEGO (Self-supervised Transformer with Energy-based Graph Optimization), conversely, uses a technique known as semantic segmentation, which applies a class label to each pixel in the image to give the AI a more accurate view of the world around it.