Researchers Demonstrate AI Can Be Fooled

The artificial intelligence systems used by image recognition tools, such as those that certain connected cars use to identify street signs, can be tricked to make an incorrect identification by a low-cost but effective attack using a camera, a projector and a PC, according to Purdue University researchers. A research paper describes an Optical Adversarial Attack, or OPAD, which uses a projector to project calculated patterns that alter the appearance of the 3D objects to AI-based image recognition systems. The paper will be presented in October at an ICCV 2021 Workshop. In an experiment, a pattern was projected onto a stop sign, causing the image recognition to read the sign as a speed limit sign instead. The researchers say this attack method could also work with image recognition tools in applications ranging from military drones to facial recognition systems, potentially undermining their reliability.