Deep Learning
DifNet: Semantic Segmentation by Diffusion Networks
Peng Jiang, Fanglin Gu, Yunhai Wang, Changhe Tu, Baoquan Chen
Deep Neural Networks (DNNs) have recently shown state of the art performance on semantic segmentation tasks, however, they still suffer from problems of poor boundary localization and spatial fragmented predictions . The difficulties lie in the requirement of making dense predictions from a long path model all at once, since details are hard to keep when data goes through deeper l ayers. Instead, in this work, we decompose this difficult task into two relative simple sub-tasks: seed detection which is required to predict initial predictions without the need of wholeness and preciseness, and similarity estimation which meas ures the possibility of any two nodes belong to the same class without the need of know ing which class they are. We use one branch network for one sub-task each, and apply a cascade of random walks base on hierarchical semantics to approximate a complex diffusion process which propagates seed information to the whole imag e according to the estimated similarities. The proposed DifNet consistently produces improvements ov er the baseline models with the same depth and with the equivalent number of para meters, and also achieves promising performance on Pascal VOC and Pascal Con text dataset. Our DifNet is trained end-to-end without complex loss function s.
Long short-term memory and Learning-to-learn in networks of spiking neurons
Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, Wolfgang Maass
Recurrent networks of spiking neurons (RSNNs) underlie the astounding computing and learning capabilities of the brain. But computing and learning capabilities of RSNN models have remained poor, at least in comparison with artificial neural networks (ANNs). We address two possible reasons for that. One is that RSNNs in the brain are not randomly connected or designed according to simple rules, and they do not start learning as a tabula rasa network. Rather, RSNNs in the brain were optimized for their tasks through evolution, development, and prior experience. Details of these optimization processes are largely unknown. But their functional contribution can be approximated through powerful optimization methods, such as backpropagation through time (BPTT). A second major mismatch between RSNNs in the brain and models is that the latter only show a small fraction of the dynamics of neurons and synapses in the brain. We include neurons in our RSNN model that reproduce one prominent dynamical process of biological neurons that takes place at the behaviourally relevant time scale of seconds: neuronal adaptation.