With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics and present a fair performance evaluation across standardized evaluation metrics. Finally, we discuss the strengths and limitations of existing studies and present potential future directions.

Both scenarios have some advantages which are, unfortunately, overweighed with severe limitations that hinder implementations of BCI technology in real-world tasks. Thus, in synchronous BCI paradigms, a wide variety of stimuli, including visual categories, can be utilized to explore and measure the evoked responses of a particular subject [1]. However, the whole set of stimuli has to be successively presented to the subject each time to determine his intention, which makes such approach inconvenient for the applications requiring fast, real-time control of an external device. Motor-imagery or other asynchronous BCIs do not require any external stimuli presentation, which allows a subject to produce voluntary mental commands at his own wish. At the same time, the ability of different subjects to perform various mental tasks is variable and depends on their personal physiological parameters and experience [2].

Both scenarios have some advantages which are, unfortunately, overweighed with severe limitations that hinder implementations of BCI technology in real-world tasks. Thus, in synchronous BCI paradigms, a wide variety of stimuli, including visual categories, can be utilized to explore and measure the evoked responses of a particular subject [1]. However, the whole set of stimuli has to be successively presented to the subject each time to determine his intention, which makes such approach inconvenient for the applications requiring fast, real-time control of an external device. Motor-imagery or other asynchronous BCIs do not require any external stimuli presentation, which allows a subject to produce voluntary mental commands at his own wish. At the same time, the ability of different subjects to perform various mental tasks is variable and depends on their personal physiological parameters and experience [2].

In compressed sensing, we wish to reconstruct a sparse signal $x$ from observed data $y$. In sparse coding, on the other hand, we wish to find a representation of an observed signal $y$ as a sparse linear combination, with coefficients $x$, of elements from an overcomplete dictionary. While many algorithms are competitive at both problems when $x$ is very sparse, it can be challenging to recover $x$ when it is less sparse. We present the Difference Map, which excels at sparse recovery when sparseness is lower and noise is higher. The Difference Map out-performs the state of the art with reconstruction from random measurements and natural image reconstruction via sparse coding.