3D reconstruction of hidden branch structures made by using image analysis and AI tech

Three-dimensional (3D) reconstruction from multiple images obtained from different viewpoints has been actively examined. However, it was difficult to reconstruct the structure of objects which have hidden portions, such as plants with branch structures hidden under their leaves. By combining the original image-to-image translation approach in a Bayesian deep learning framework and 3D reconstruction, a group of researchers led by Fumio Okura estimated the existence probability of branches that are hidden under leaves in images obtained. Using these estimated branch positions, they achieved 3D reconstruction of plant structure, i.e., accurate reconstruction of branch structures, including those hidden under leaves. Specifically, they converted images of leafy plants to images showing branch existence probability, thereby achieving 3D reconstruction. The results of this study will be presented at the EEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018) to be held from June 18 through June 22, 2018.