Goto

Collaborating Authors

Perceptrons: Instructional Materials


gMLP: Winning over Transformers?

#artificialintelligence

Alright, we all know that transformers are cool. At least in terms of NLP, these architectures are considered to be state-of-the-art (SOTA) for language modelling, and help us perform beautifully on various downtream tasks, such as named-entity-recognition (NER), question answering (QA), part of speech tagging (POS) etc. But in this tutorial, we will dive into another architecture called Gated Multilayer Perceptron (gMLP), proposed by Google Research team. As I mentioned above, transformer architectures are very powerful, and if you want to achieve a really high performance in your particular task, you should consider using some pre-trained transformers. You could usually find them on Huggingface.


Collaborative Reflection-Augmented Autoencoder Network for Recommender Systems

arXiv.org Artificial Intelligence

As the deep learning techniques have expanded to real-world recommendation tasks, many deep neural network based Collaborative Filtering (CF) models have been developed to project user-item interactions into latent feature space, based on various neural architectures, such as multi-layer perceptron, auto-encoder and graph neural networks. However, the majority of existing collaborative filtering systems are not well designed to handle missing data. Particularly, in order to inject the negative signals in the training phase, these solutions largely rely on negative sampling from unobserved user-item interactions and simply treating them as negative instances, which brings the recommendation performance degradation. To address the issues, we develop a Collaborative Reflection-Augmented Autoencoder Network (CRANet), that is capable of exploring transferable knowledge from observed and unobserved user-item interactions. The network architecture of CRANet is formed of an integrative structure with a reflective receptor network and an information fusion autoencoder module, which endows our recommendation framework with the ability of encoding implicit user's pairwise preference on both interacted and non-interacted items. Additionally, a parametric regularization-based tied-weight scheme is designed to perform robust joint training of the two-stage CRANet model. We finally experimentally validate CRANet on four diverse benchmark datasets corresponding to two recommendation tasks, to show that debiasing the negative signals of user-item interactions improves the performance as compared to various state-of-the-art recommendation techniques. Our source code is available at https://github.com/akaxlh/CRANet.


Trap of Feature Diversity in the Learning of MLPs

arXiv.org Artificial Intelligence

In this paper, we discover a two-phase phenomenon in the learning of multi-layer perceptrons (MLPs). I.e., in the first phase, the training loss does not decrease significantly, but the similarity of features between different samples keeps increasing, which hurts the feature diversity. We explain such a two-phase phenomenon in terms of the learning dynamics of the MLP. Furthermore, we propose two normalization operations to eliminate the two-phase phenomenon, which avoids the decrease of the feature diversity and speeds up the training process.


A Tutorial on Spiking Neural Networks for Beginners

#artificialintelligence

Despite being quite effective in a variety of tasks across industries, deep learning is constantly evolving, proposing new neural network (NN) architectures, deep learning (DL) tasks, and even brand new concepts of the next generation of NNs, such as the Spiking Neural Network (SNN). SNN was introduced by the researchers at Heidelberg University and the University of Bern developing as a fast and energy-efficient technique for computing using spiking neuromorphic substrates. In this article, we will mostly discuss Spiking Neural Network as a variant of neural network. We will also try to understand how is it different from the traditional neural networks. Below is a list of the important topics to be tackled.


Singular learning of deep multilayer perceptrons for EEG-based emotion recognition

#artificialintelligence

Human emotion recognition is an important issue in human-computer interactions and electroencephalograph (EEG) has been widely applied to emotion recognition due to its high reliability. In recent years, methods based deep learning technology have reached the state of art performance in EEG-based emotion recognition. However, there exist singularities in the parameter space of deep neural networks, which may dramatically slow down the training process. It is very worthy to investigate the specific influence of singularities when applying deep neural networks to EEG-based emotion recognition. In this paper, we mainly focus on this problem, and analyse the singular learning dynamics of deep multilayer perceptrons theoretically and numerically. The results can help us to design better algorithms to overcome the serious influence of singularities in deep neural networks for EEG-based emotion recognition.


BotSpot: Deep Learning Classification of Bot Accounts within Twitter

arXiv.org Artificial Intelligence

The openness feature of Twitter allows programs to generate and control Twitter accounts automatically via the Twitter API. These accounts, which are known as bots, can automatically perform actions such as tweeting, re-tweeting, following, unfollowing, or direct messaging other accounts, just like real people. They can also conduct malicious tasks such as spreading of fake news, spams, malicious software and other cyber-crimes. In this paper, we introduce a novel bot detection approach using deep learning, with the Multi-layer Perceptron Neural Networks and nine features of a bot account. A web crawler is developed to automatically collect data from public Twitter accounts and build the testing and training datasets, with 860 samples of human and bot accounts. After the initial training is done, the Multilayer Perceptron Neural Networks achieved an overall accuracy rate of 92%, which proves the performance of the proposed approach.


Under the Hood of Modern Machine and Deep Learning

#artificialintelligence

In this chapter, we investigate whether unique, optimal decision boundaries can be found. In order to do so, we first have to revisit several fundamental mathematical principles. Regularization is a mathematical tool, which allows us to find unique solutions even for highly ill-posed problems. In order to use this trick, we review norms and how they can be used to steer regression problems. Rosenblatt's Perceptron and Multi-Layer Perceptrons which are also called Artificial Neural Networks inherently suffer from this ill-posedness.


Under the Hood of Modern Machine and Deep Learning

#artificialintelligence

In this chapter, we investigate whether unique, optimal decision boundaries can be found. In order to do so, we first have to revisit several fundamental mathematical principles. Regularization is a mathematical tool, which allows us to find unique solutions even for highly ill-posed problems. In order to use this trick, we review norms and how they can be used to steer regression problems. Rosenblatt's Perceptron and Multi-Layer Perceptrons which are also called Artificial Neural Networks inherently suffer from this ill-posedness.


On the Computational Intelligibility of Boolean Classifiers

arXiv.org Artificial Intelligence

In this paper, we investigate the computational intelligibility of Boolean classifiers, characterized by their ability to answer XAI queries in polynomial time. The classifiers under consideration are decision trees, DNF formulae, decision lists, decision rules, tree ensembles, and Boolean neural nets. Using 9 XAI queries, including both explanation queries and verification queries, we show the existence of large intelligibility gap between the families of classifiers. On the one hand, all the 9 XAI queries are tractable for decision trees. On the other hand, none of them is tractable for DNF formulae, decision lists, random forests, boosted decision trees, Boolean multilayer perceptrons, and binarized neural networks.


From perceptrons to deep learning

#artificialintelligence

Have you ever wondered if it's possible to learn all there is to know about machine learning and deep learning from a book? Machine Learning--A Journey to Deep Learning, with Exercises and Answers is designed to give the self-taught student a solid foundation in machine learning with step-by-step solutions to the formative exercises and many concrete examples. By going through this text, readers should become able to apply and understand machine learning algorithms as well as create new ones. The statistical approach leads to the definition of regularization out of the example of regression. Building on regression, we develop the theory of perceptrons and logistic regression.