Investigation of Maxout Activations on Convolutional Neural Networks for Big Data Text Sentiment Analysis

We explore the performance of multiple maxout activation variants on the big data text sentiment analysis task using convolutional neural networks. Maxout networks have gained great success in many computer vision tasks, but there is limited work on other classification tasks. Our experiments compare ReLU, LReLU, SeLU and tanh to four maxout variants. We evaluate the effectiveness of the activation functions on five datasets, including two datasets collected from the Amazon product reviews corpus, two datasets collected from the Yelp corpus, and the Sentiment140 dataset. Throughout the experiments, we found that maxout networks are slow to train compared to the traditional activation functions. We find that on average across all datasets, ReLU’s classification performance is better than any maxout activation if the number of convolutional filters is doubled. Our experiments suggest that adding more filters enhance the classification accuracy of ReLU, without affecting its comparatively low training time.