Generating high-quality 3DMPCs by adaptive data acquisition and NeREF-based radiometric calibration with UGV plant phenotyping system

An efficient method for next-best-view (NBV) estimation with high accuracy in the case of the limited camera field of view (FOV) was proposed. Abstract: Fusion of three-dimensional (3D) and multispectral (MS) imaging data has a great potential for high-throughput plant phenotyping of structural and biochemical as well as physiological traits simultaneously, which is important for decision support in agriculture and for crop breeders in selecting the best genotypes. However, lacking of 3D data integrity of various plant canopy structures and low-quality of MS images caused by the complex illumination effects make a great challenge, especially at the proximal imaging scale. Therefore, this study proposed a novel approach for adaptive data acquisition and radiometric calibration to generate high-quality 3D multispectral point clouds (3DMPCs) of plants. An efficient next-best-view (NBV) planning method based on an unmanned ground vehicle (UGV) plant phenotyping system with a multisensor-equipped robotic arm was proposed to achieve adaptive data acquisition. The neural reference field (NeREF) was employed to predict the digital number (DN) values of the hemispherical reference for radiometric calibration. For NBV planning, the average total time for single plant at a joint speed of 1.55 rad/s was about 62.8 s, with an average reduction of 18.0% compared to the unplanned. The integrity of the wholeplant data was improved by an average of 23.6% compared to the fixed viewpoints alone. Compared with the ASD measurements, the average root mean square error (RMSE) of the reflectance spectra obtained from 3DMPCs at different regions of interest was 0.08 with an average decrease of 58.93% compared to the results obtained from the single-frame of MS images without 3D radiometric calibration. The 3Dcalibrated plant 3DMPCs improved the predictive accuracy of partial least squares regression (PLSR) for chlorophyll content, with an average increase of 0.07 in the coefficient of determination (R Our approach introduced a fresh perspective on generating high-quality 3DMPCs of plants under the natural light condition, enabling more precise analysis of plant morphological and physiological parameters. Keywords: adaptive data acquisition; 3DMPC; NBV planning; radiometric calibration; NeREF; chlorophyll content 1. Introduction High-throughput plant phenotyping provides an unprecedented way to systematically evaluate plant development and functionality with the precise quantification of morphological, physiological, biochemical, and performance traits over the whole growth period. It can help on decision support in agriculture, for ecological diversity studies, and for crop breeding in the selection of superior genotypes to improve crop performance, and thus revolutionize the agriculture and breeding strategies to meet the future need of agricultural sustainable development (Freschet et al. 2018; Hu and Schmidhalter 2023).