A Tropical Approach to Neural Networks with Piecewise Linear Activations

Traditional literature on pattern recognition and neural networks utilizes the linear Perceptron, a multiply-accumulate architecture fed into an (optional) activation function introduced by Rosenblatt [40], as the building block of a multitude of complex architectures modelling neural computation. In recent years, multilayered, complex architectures of neural networks have enjoyed an unprecedented growth in popularity, with the introduction of the paradigm of deep learning [4]. An illustrative example of the power of deep learning is Convolutional Neural Networks; although they were the state of the art when they were introduced, two decades ago [24], it wasn't until recently that they were systematically applied to image recognition challenges[23], achieving results comparable to humans (e.g.