It is widely known that Boltzmann machines are capable of representing arbitrary probability distributions over the values of their visible neurons, given enough hidden ones. However, sampling -- and thus training -- these models can be numerically hard. Recently we proposed a regularisation of the connections of Boltzmann machines, in order to control the energy landscape of the model, paving a way for efficient sampling and training. Here we formally prove that such regularised Boltzmann machines preserve the ability to represent arbitrary distributions. This is in conjunction with controlling the number of energy local minima, thus enabling easy \emph{guided} sampling and training. Furthermore, we explicitly show that regularised Boltzmann machines can store exponentially many arbitrarily correlated visible patterns with perfect retrieval, and we connect them to the Dense Associative Memory networks.

Most existing training algorithms for SNNs assume spiking neuron models in which a neuron outputs individual spikes as a function of the dynamics of an internal state variable known as membrane potential. This paper explores a more general model in which each spiking neuron contains multiple compartments, each tracking the dynamics of a distinct membrane potential, while sharing the same synaptic weights across compartments. It is demonstrated that learning rules based on probabilistic generalized linear neural models can leverage the presence of multiple compartments through modern variational inference based on importance weighting or generalized expectation-maximization. The key idea is to use the neural compartments to sample multiple independent spiking signals from hidden neurons so as to obtain better statistical estimates of the likelihood training criterion. The derived online learning algorithms follow three-factor rules with global learning signals. Experimental results on a structured output memorization task and classification task with a standard neuromorphic data set demonstrate significant improvements in terms of accuracy and calibration with an increasing number of compartments.

Spiking neural networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by energy-efficient hardware implementations, which can offer significant energy reductions as compared to conventional artificial neural networks (ANNs). The design of training algorithms lags behind the hardware implementations. Most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pretrained ANNs via rate encoding. This article provides an introduction to SNNs by focusing on a probabilistic signal processing methodology that enables the direct derivation of learning rules by leveraging the unique time-encoding capabilities of SNNs. We adopt discrete-time probabilistic models for networked spiking neurons and derive supervised and unsupervised learning rules from first principles via variational inference. Examples and open research problems are also provided.

Recently, Deep Learning has been applied in the techniques of artificial intelligence. Especially, Deep Learning performed good results in the field of image recognition. Most new Deep Learning architectures are naturally developed in image recognition. For this reason, not only the numerical data and text data but also the time-series data are transformed to the image data format. Multi-modal data consists of two or more kinds of data such as picture and text. The arrangement in a general method is formed in the squared array with no specific aim. In this paper, the data arrangement are modified according to the similarity of input-output pattern in Adaptive Structural Learning method of Deep Belief Network. The similarity of output signals of hidden neurons is made by the order rearrangement of hidden neurons. The experimental results for the data rearrangement in squared array showed the shortening time required for DBN learning.

Through the success of deep learning in various domains, artificial neural networks are currently among the most used artificial intelligence methods. Taking inspiration from the network properties of biological neural networks (e.g. sparsity, scale-freeness), we argue that (contrary to general practice) artificial neural networks, too, should not have fully-connected layers. Here we propose sparse evolutionary training of artificial neural networks, an algorithm which evolves an initial sparse topology (Erd\H{o}s-R\'enyi random graph) of two consecutive layers of neurons into a scale-free topology, during learning. Our method replaces artificial neural networks fully-connected layers with sparse ones before training, reducing quadratically the number of parameters, with no decrease in accuracy. We demonstrate our claims on restricted Boltzmann machines, multi-layer perceptrons, and convolutional neural networks for unsupervised and supervised learning on 15 datasets. Our approach has the potential to enable artificial neural networks to scale up beyond what is currently possible.

Machine learning, the field that's driving a revolution in artificial intelligence, has cemented its role in modern technology. Its tools and techniques have led to rapid improvements in everything from self-driving cars and speech recognition to the digital mastery of an ancient board game. Now, physicists are beginning to use machine learning tools to tackle a different kind of problem, one at the heart of quantum physics. In a paper published recently in Physical Review X, researchers from JQI and the Condensed Matter Theory Center (CMTC) at the University of Maryland showed that certain neural networks--abstract webs that pass information from node to node like neurons in the brain--can succinctly describe wide swathes of quantum systems . Dongling Deng, a JQI Postdoctoral Fellow who is a member of CMTC and the paper's first author, says that researchers who use computers to study quantum systems might benefit from the simple descriptions that neural networks provide.

Researchers have yet to explore these quantum systems fully. Realizing the need for better tools to do so, physicists from the Joint Quantum Institute (JQI) and the University of Maryland's Condensed Matter Theory Center (CMTC) have turned to artificial neural networks, which are constructed to function and pass information like neurons in the brain. "If we want to numerically tackle some quantum problem, we first need to find an efficient representation," JQI researcher Dongling Deng said in a press release. He got the idea after hearing about DeepMind's Go-playing artificial intelligence (AI) AlphaGo famously defeated human professional players in 2016. Machine learning, which is behind the achievements of current AI systems, seemed like a plausible tool.

We consider a statistical framework in which recurrent networks of spiking neurons learn to generate spatio-temporal spike patterns. Given biologically realistic stochastic neuronal dynamics we derive a tractable learning rule for the synaptic weights towards hidden and visible neurons that leads to optimal recall of the training sequences. We show that learning synaptic weights towards hidden neurons significantly improves the storing capacity of the network. Furthermore, we derive an approximate online learning rule and show that our learning rule is consistent with Spike-Timing Dependent Plasticity in that if a presynaptic spike shortly precedes a postynaptic spike, potentiation is induced and otherwise depression is elicited.