Hierarchical Spatial-Frequency Aggregation for Spectral Deconvolution Imaging

Abstract--Computational spectral imaging (CSI) achieves real-time hyperspectral imaging through co-designed optics and algorithms, but typical CSI methods suffer from a bulky footprint and limited fidelity. Therefore, Spectral Deconvolution imaging (SDI) methods based on PSF engineering have been proposed to achieve high-fidelity compact CSI design recently. However, the composite convolution-integration operations of SDI render the normal-equation coefficient matrix scene-dependent, which hampers the efficient exploitation of imaging priors and poses challenges for accurate reconstruction. T o tackle the inherent data-dependent operators in SDI, we introduce a Hierarchical Spatial-Spectral Aggregation Unfolding Framework (HSF AUF). By decomposing subproblems and projecting them into the frequency domain, HSF AUF transforms nonlinear processes into linear mappings, thereby enabling efficient solutions. Furthermore, to integrate spatial-spectral priors during iterative refinement, we propose a Spatial-Frequency Aggregation Transformer (SF A T), which explicitly aggregates information across spatial and frequency domains. By integrating SF A T into HSF AUF, we develop a Transformer-based deep unfolding method, Hierarchical Spatial-Frequency Aggregation Unfolding Transformer (HSF AUT), to solve the inverse problem of SDI. Systematic simulated and real experiments show that HSF AUT surpasses SOT A methods with cheaper memory and computational costs, while exhibiting optimal performance on different SDI systems. Hyperspectral images (HSIs) capture high-resolution spectra at each spatial location, providing a spectral representation that reveals the rich characteristics of various components and materials, offering a high-dimensional visual capability beyond human vision. Thus, HSIs have found widespread applications in fields such as medical diagnosis [1], remote sensing [2], [3], agricultural inspection [4], and machine vision [5]. However, early hyperspectral imaging techniques were constrained by 2D sensor, requiring spatial or spectral scanning that sacrificed temporal resolution for spectral resolution, restricting their use in dynamic scenes. To overcome these challenges, computational spectral imaging (CSI) [6] integrates optics, electronics, and algorithms to enhance imaging capabilities [7], [8], [9].