Developing amethod forhigh-quality reconstruction ofseenimages fromthecorresponding brain activity is an important milestone towards decoding the contents of dreams and mental imagery (Fig 1a). In this task, one attempts to solve for the mapping between fMRI recordings and their corresponding natural images, using many "labeled"{Image, fMRI} pairs (i.e., images and their corresponding fMRIresponses).

With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics and present a fair performance evaluation across standardized evaluation metrics. Finally, we discuss the strengths and limitations of existing studies and present potential future directions.

Imagine a reality where computers can visualise what you are thinking. In late December, Guohua Shen, Tomoyasu Horikawa, Kei Majima and Yukiyasu Kamitani released the results of their recent research on using artificial intelligence to decode thoughts on the scientific platform, BioRxiv. Machine learning has previously been used to study brain scans (MRIs, or magnetic resonance imaging) and generate visualisations of what a person is thinking when referring to simple, binary images like black and white letters or simple geographic shapes (as shown in Figure 2 here). But the scientists from Kyoto developed new techniques of "decoding" thoughts using deep neural networks (artificial intelligence). The new technique allows the scientists to decode more sophisticated "hierarchical" images, which have multiple layers of colour and structure, like a picture of a bird or a man wearing a cowboy hat, for example.

As computer scientists attempt to make machines think and learn like humans, the middle ground is being taken up by researchers attempting to use AI to read our minds. In the latest breakthrough, scientists at Kyoto University, Japan, have studied deep neural networks (AI) and discovered that computers wield the capacity to at least visualise what humans are thinking. Before we get ahead of ourselves, it's worth noting that the technology is nascent, and applies in only optimal conditions. If you recoil at someone's dubious new choice of profile picture on Facebook, your laptop isn't going to start registering your distaste and broadcasting it to the world. That being said, the new technology certainly has seemingly impressive – if ominous – potential applications.

Imagine a reality where computers can visualize what you are thinking. In late December, Guohua Shen, Tomoyasu Horikawa, Kei Majima and Yukiyasu Kamitani released the results of their recent research on using artificial intelligence to decode thoughts on the scientific platform, BioRxiv. Machine learning has previously been used to study brain scans (MRIs, or magnetic resonance imaging) and generate visualizations of what a person is thinking when referring to simple, binary images like black and white letters or simple geographic shapes (as shown in Figure 2 here). But the scientists from Kyoto developed new techniques of "decoding" thoughts using deep neural networks (artificial intelligence). The new technique allows the scientists to decode more sophisticated "hierarchical" images, which have multiple layers of color and structure, like a picture of a bird or a man wearing a cowboy hat, for example.

There have been significant limitations, however, beginning with a necessity to extensively catalog each subject's unique brain patterns, which are then matched with a small number of pre-programmed images. These procedures require that subjects undergo lengthy and expensive fMRI testing. Now a team of researchers in Kyoto has used neural network-based artificial intelligence to decode and predict what a person is seeing or imagining, referring to a significantly larger catalog of images. Their results are reported in Nature Communications. "When we gaze at an object, our brains process these patterns hierarchically, starting with the simplest and progressing to more complex features," explains team leader Yukiyasu Kamitani of Kyoto University.

Scientists in Japan have developed an AI that can decode patterns in the brain to predict what a person is seeing or imagining. In a new study, researchers used signal patterns derived from a deep neural network to predict visual features from fMRI scans. Their'decoder' was able to identify objects with a high degree of accuracy, and the researchers say the breakthrough could pave the way for more advanced'brain-machine interfaces.' In a new study, researchers used signal patterns derived from a deep neural network to predict visual features from fMRI scans. Their'decoder' was able to identify objects with a high degree of accuracy.