Neumann Networks for Inverse Problems in Imaging

Many challenging image processing tasks can be described by an ill-posed linear inverse problem: deblurring, deconvolution, inpainting, compressed sensing, and superresolution all lie in this framework. Traditional inverse problem solvers minimize a cost function consisting of a data-fit term, which measures how well an image matches the observations, and a regularizer, whichreflects prior knowledge and promotes images with desirable properties like smoothness. Recent advances in machine learning and image processing have illustrated that it is often possible to learn a regularizer from training data that can outperform more traditional regularizers.We present an end-to-end, data-driven method of solving inverse problems inspired by the Neumann series, which we call a Neumann network. Rather than unroll an iterative optimization algorithm, we truncate a Neumann series which directly solves the linear inverseproblem with a data-driven nonlinear regularizer. Finally, when the images belong to a union of subspaces and under appropriate assumptions on the forward model, we prove there exists a Neumann network configuration that well-approximates the optimal oracle estimator for the inverse problem and demonstrate empirically that the trained Neumann network has the form predicted by theory. D. Gilton is with the Department of Electrical and Computer Engineering, University of Wisconsin, Madison, WI, 53706 USA (email: gilton@wisc.edu). G. Ongie is with the Department of Statistics, University of Chicago, Chicago, IL, 60637 USA (email: gongie@uchicago.edu). R. Willett is with the Department of Statistics and Computer Science, University of Chicago, Chicago, IL, 60637 USA (email: willett@uchicago.edu). In general, a regularization function r(β) measures the lack of conformity of β to this prior knowledge and β is selected so that r( β) is as small as possible while still providing a good fit to the data. However, recent work in computer vision using deep neural networks has leveraged large collections of"training" images to yield unprecedented image recognition performance [32, 33, 38], and an emerging body of research is exploring whether this training data can also be used to improve thequality of image reconstruction. In other words, can training data be used to learn how to regularize inverse problems? As we detail below, existing methods include using training images to learn a low-dimensional image manifold and constraining β to lie on this manifold [9] or learning a denoising autoencoder that can be treated as a regularization step (i.e., proximal operator) within an iterative reconstruction scheme [47].