When Can Neural Networks Learn Connected Decision Regions?

arXiv.org Machine Learning

Previous work has questioned the conditions under which the decision regions of a neural network are connected and further showed the implications of the corresponding theory to the problem of adversarial manipulation of classifiers. It has been proven that for a class of activation functions including leaky ReLU, neural networks having a pyramidal structure, that is no layer has more hidden units than the input dimension, produce necessarily connected decision regions. In this paper, we advance this important result by further developing the sufficient and necessary conditions under which the decision regions of a neural network are connected. We then apply our framework to overcome the limits of existing work and further study the capacity to learn connected regions of neural networks for a much wider class of activation functions including those widely used, namely ReLU, sigmoid, tanh, softlus, and exponential linear function.

Neural network learns to reproduce what your brain sees


Scientists dream of recreating mental images through brain scans, but current techniques produce results that are... fuzzy, to put it mildly. A trio of Chinese researchers might just solve that. They've developed neural network algorithms that do a much better job of reproducing images taken from functional MRI scans. The team trains its network to recreate images by feeding it the visual cortex scans of someone looking at a picture and asking the network to recreate the original image based on that data. After enough practice, it's off to the races -- the system knows how to correlate voxels (3D pixels) in scans so that it can generate accurate, noise-free images without having to see the original.

A Beginner's Guide to Neural Networks with Python and SciKit Learn 0.18!


I'm Jose Portilla and I teach thousands of students on Udemy about Data Science and Programming and I also conduct in-person programming and data science training. Check out the end of the article for discount coupons on my courses! The most popular machine learning library for Python is SciKit Learn. The newest version (0.18) was just released a few days ago and now has built in support for Neural Network models. In this article we will learn how Neural Networks work and how to implement them with the Python programming language and latest version of SciKit-Learn!

On decision regions of narrow deep neural networks

arXiv.org Machine Learning

We show that for neural network functions that have width less or equal to the input dimension all connected components of decision regions are unbounded. The result holds for continuous and strictly monotonic activation functions as well as for ReLU activation. This complements recent results on approximation capabilities of [Hanin 2017 Approximating] and connectivity of decision regions of [Nguyen 2018 Neural] for such narrow neural networks. Further, we give an example that negatively answers the question posed in [Nguyen 2018 Neural] whether one of their main results still holds for ReLU activation. Our results are illustrated by means of numerical experiments.