Thermal cameras like FLIR, or Forward Looking Infrared, sensors are actively deployed on aerial and ground vehicles, in watch towers and at check points for surveillance purposes. More recently, thermal cameras are becoming available for use as body-worn cameras. The ability to perform automatic face recognition at nighttime using such thermal cameras is beneficial for informing a Soldier that an individual is someone of interest, like someone who may be on a watch list. The motivations for this technology -- developed by Drs. Benjamin S. Riggan, Nathaniel J. Short and Shuowen "Sean" Hu, from the U.S. Army Research Laboratory -- are to enhance both automatic and human-matching capabilities.
US army researchers have developed a convolutional neural network and a range of algorithms to recognise faces in the dark. "This technology enables matching between thermal face images and existing biometric face databases or watch lists that only contain visible face imagery," explained Benjamin Riggan on Monday, co-author of the study and an electronics engineer at the US army laboratory. "The technology provides a way for humans to visually compare visible and thermal facial imagery through thermal-to-visible face synthesis." The thermal images are processed and passed to a convolutional neural network to extract facial features using landmarks that mark the corners of the eyes, nose and lips to determine its overall shape. The system, dubbed "multi-region synthesis" is trained with a loss function so that the error between the thermal images and the visible ones is minimized, creating an accurate portrayal of what someone's face looks like despite only glimpsing it in the dark.
The US Army is developing a machine learning method for identifying faces from thermal imagery. Soon the American government will be able to film people from outside of buildings, using cameras that can see through walls in near-total darkness, and an AI will recognize the people in the images. Army Research Laboratory (ARL) scientists Benjamin S. Riggan, Nathaniel J. Short, and Shuowen Hu recently released a white paper detailing military efforts to develop a method for applying facial recognition technology to images taken using thermal imaging devices. When using thermal cameras to capture facial imagery, the main challenge is that the captured thermal image must be matched against a watch list or gallery that only contains conventional visible imagery from known persons of interest. Such devices are common, especially in military use.
With the increased attention on thermal imagery for Covid-19 screening, the public sector may believe there are new opportunities to exploit thermal as a modality for computer vision and AI. Thermal physiology research has been ongoing since the late nineties. This research lies at the intersections of medicine, psychology, machine learning, optics, and affective computing. We will review the known factors of thermal vs. RGB imaging for facial emotion recognition. But we also propose that thermal imagery may provide a semi-anonymous modality for computer vision, over RGB, which has been plagued by misuse in facial recognition. However, the transition to adopting thermal imagery as a source for any human-centered AI task is not easy and relies on the availability of high fidelity data sources across multiple demographics and thorough validation. This paper takes the reader on a short review of machine learning in thermal FER and the limitations of collecting and developing thermal FER data for AI training. Our motivation is to provide an introductory overview into recent advances for thermal FER and stimulate conversation about the limitations in current datasets.
Is thermal imagery detailed enough to enable an AI model to recognize people's facial features? That's the question Intel and Gánsk University of Technology researchers sought to answer in a study recently presented at the Institute of Electrical and Electronics Engineers' 12th International Conference on Human System Interaction. These researchers investigated the performance of a model trained on visible light data that was subsequently retrained on thermal images. As the researchers point out in a paper describing their work, thermal imagery is often used in lieu of RGB camera data within environments where privacy is preferred or otherwise mandated, like medical facilities. That's because it's able to obscure personally identifying details like eye color and jaw line.