Multispectral Transfer Network: Unsupervised Depth Estimation for All-Day Vision

Kim, Namil (NAVER LABS Corp.) | Choi, Yukyung (Clova NAVER Corp.) | Hwang, Soonmin (Korea Advanced Institute of Science and Technology (KAIST)) | Kweon, In So (Korea Advanced Institute of Science and Technology (KAIST))

AAAI Conferences 

To understand the real-world, it is essential to perceive in all-day conditions including cases which are not suitable for RGB sensors, especially at night. Beyond these limitations, the innovation introduced here is a multispectral solution in the form of depth estimation from a thermal sensor without an additional depth sensor.Based on an analysis of multispectral properties and the relevance to depth predictions, we propose an efficient and novel multi-task framework called the Multispectral Transfer Network (MTN) to estimate a depth image from a single thermal image. By exploiting geometric priors and chromaticity clues, our model can generate a pixel-wise depth image in an unsupervised manner. Moreover, we propose a new type of multitask module called Interleaver as a means of incorporating the chromaticity and fine details of skip-connections into the depth estimation framework without sharing feature layers. Lastly, we explain a novel technical means of stably training and covering large disparities and extending thermal images to data-driven methods for all-day conditions. In experiments, we demonstrate the better performance and generalization of depth estimation through the proposed multispectral stereo dataset, including various driving conditions.

Duplicate Docs Excel Report

None found

Similar Docs  Excel Report  more

None found