Scharr, Hanno
MRI Reconstruction with Regularized 3D Diffusion Model (R3DM)
Bangun, Arya, Cao, Zhuo, Quercia, Alessio, Scharr, Hanno, Pfaehler, Elisabeth
In order to speed up the acquisition time, MRI instruments acquire sub-sampled k-space data, a technique where only a fraction of the total k-space data points are sampled during the imaging process. Several attempts have been proposed to develop two-dimensional (2D) and three-dimensional (3D) image reconstruction techniques for sub-sampled k-space, as discussed in [11, 13, 31]. Advancements in 3D MR imaging methods can address the challenges posed by complex anatomical structures of human organs and plant growths. Consequently, the demand for developing 3D MR image reconstruction methods has intensified. Currently, most works reconstruct a 3D volumetric image by stacking 2D reconstructions because MR images are acquired slice by slice. This method doesn't consider the inter-dependency between the slices, thus can lead to inconsistencies and artifacts, as discussed in [4, 8, 50]. This particularly affects datasets that have equally distributed information and structures with high continuity on all dimensions, such as roots and vessels [4, 38, 50]. Before the deep learning-based models, which learn the data-driven prior, the model-based iterative reconstruction method proved its effectiveness in the 3D MRI reconstruction problem [15, 54]. The problem is formulated as an optimization problem where a data consistency term is applied to ensure fidelity, and a regularisation term, such as the Total Variation (TV) penalty [24] is utilized to provide general prior knowledge of MRI data.
Retrieval of sun-induced plant fluorescence in the O$_2$-A absorption band from DESIS imagery
Buffat, Jim, Pato, Miguel, Alonso, Kevin, Auer, Stefan, Carmona, Emiliano, Maier, Stefan, Müller, Rupert, Rademske, Patrick, Rascher, Uwe, Scharr, Hanno
We provide the first method allowing to retrieve spaceborne SIF maps at 30 m ground resolution with a strong correlation ($r^2=0.6$) to high-quality airborne estimates of sun-induced fluorescence (SIF). SIF estimates can provide explanatory information for many tasks related to agricultural management and physiological studies. While SIF products from airborne platforms are accurate and spatially well resolved, the data acquisition of such products remains science-oriented and limited to temporally constrained campaigns. Spaceborne SIF products on the other hand are available globally with often sufficient revisit times. However, the spatial resolution of spaceborne SIF products is too small for agricultural applications. In view of ESA's upcoming FLEX mission we develop a method for SIF retrieval in the O$_2$-A band of hyperspectral DESIS imagery to provide first insights for spaceborne SIF retrieval at high spatial resolution. To this end, we train a simulation-based self-supervised network with a novel perturbation based regularizer and test performance improvements under additional supervised regularization of atmospheric variable prediction. In a validation study with corresponding HyPlant derived SIF estimates at 740 nm we find that our model reaches a mean absolute difference of 0.78 mW / nm / sr / m$^2$.
Untrained Perceptual Loss for image denoising of line-like structures in MR images
Pfaehler, Elisabeth, Pflugfelder, Daniel, Scharr, Hanno
In the acquisition of Magnetic Resonance (MR) images shorter scan times lead to higher image noise. Therefore, automatic image denoising using deep learning methods is of high interest. MR images containing line-like structures such as roots or vessels yield special characteristics as they display connected structures and yield sparse information. For this kind of data, it is important to consider voxel neighborhoods when training a denoising network. In this paper, we translate the Perceptual Loss to 3D data by comparing feature maps of untrained networks in the loss function as done previously for 2D data. We tested the performance of untrained Perceptual Loss (uPL) on 3D image denoising of MR images displaying brain vessels (MR angiograms - MRA) and images of plant roots in soil. We investigate the impact of various uPL characteristics such as weight initialization, network depth, kernel size, and pooling operations on the results. We tested the performance of the uPL loss on four Rician noise levels using evaluation metrics such as the Structural Similarity Index Metric (SSIM). We observe, that our uPL outperforms conventional loss functions such as the L1 loss or a loss based on the Structural Similarity Index Metric (SSIM). The uPL network's initialization is not important, while network depth and pooling operations impact denoising performance. E.g. for both datasets a network with five convolutional layers led to the best performance while a network with more layers led to a performance drop. We also find that small uPL networks led to better or comparable results than using large networks such as VGG. We observe superior performance of our loss for both datasets, all noise levels, and three network architectures. In conclusion, for images containing line-like structures, uPL is an alternative to other loss functions for 3D image denoising.
Parameter-efficient Bayesian Neural Networks for Uncertainty-aware Depth Estimation
Paul, Richard D., Quercia, Alessio, Fortuin, Vincent, Nöh, Katharina, Scharr, Hanno
State-of-the-art computer vision tasks, like monocular depth estimation (MDE), rely heavily on large, modern Transformer-based architectures. However, their application in safety-critical domains demands reliable predictive performance and uncertainty quantification. While Bayesian neural networks provide a conceptually simple approach to serve those requirements, they suffer from the high dimensionality of the parameter space. Parameter-efficient fine-tuning (PEFT) methods, in particular low-rank adaptations (LoRA), have emerged as a popular strategy for adapting large-scale models to down-stream tasks by performing parameter inference on lower-dimensional subspaces. In this work, we investigate the suitability of PEFT methods for subspace Bayesian inference in large-scale Transformer-based vision models. We show that, indeed, combining BitFit, DiffFit, LoRA, and CoLoRA, a novel LoRA-inspired PEFT method, with Bayesian inference enables more robust and reliable predictive performance in MDE.