Assumed Density Filtering Methods for Learning Bayesian Neural Networks

AAAI Conferences

Buoyed by the success of deep multilayer neural networks, there is renewed interest in scalable learning of Bayesian neural networks. Here, we study algorithms that utilize recent advances in Bayesian inference to efficiently learn distributions over network weights. In particular, we focus on recently proposed assumed density filtering based methods for learning Bayesian neural networks -- Expectation and Probabilistic backpropagation. Apart from scaling to large datasets, these techniques seamlessly deal with non-differentiable activation functions and provide parameter (learning rate, momentum) free learning. In this paper, we first rigorously compare the two algorithms and in the process develop several extensions, including a version of EBP for continuous regression problems and a PBP variant for binary classification. Next, we extend both algorithms to deal with multiclass classification and count regression problems. On a variety of diverse real world benchmarks, we find our extensions to be effective, achieving results competitive with the state-of-the-art.

When to use different machine learning algorithms: a simple guide


If you've been at machine learning long enough, you know that there is a "no free lunch" principle -- there's no one-size-fits-all algorithm that will help you solve every problem and tackle every dataset. I work for Springboard -- we've put a lot of research into machine learning training and resources. At Springboard, we offer the first online course with a machine learning job guarantee. What helps a lot when confronted with a new problem is to have a primer for what algorithm might be the best fit for certain situations. Here, we talk about different problems and data types and discuss what might be the most effective algorithm to try for each one, along with a resource that can help you implement that particular model.

Machine Learning Algorithms: Which One to Choose for Your Problem - DZone AI


When I was beginning my journey in data science, I often faced the problem of choosing the most appropriate algorithm for my specific problem. If you're like me, when you open some article about machine learning algorithms, you see dozens of detailed descriptions. The paradox is that this doesn't make it easier to choose which one to use. In this article for Statsbot, I will try to explain basic concepts and give some intuition of using different kinds of machine learning algorithms for different tasks. At the end of the article, you'll find a structured overview of the main features of described algorithms.

10 Algorithms Every Machine Learning Enthusiast Should Know


It is very crucial for the machine learning enthusiasts to know and understands the basic and important machine learning algorithms in order to keep themselves up with the current trend. In this article, we list down 10 basic algorithms which play very important roles in the machine learning era. Logistic regression, also known as the logit classifier is a popular mathematical modelling procedure used in the analysis of data. Regression Analysis is used to conduct when the dependent variable is binary i.e. 0 and 1. In Logistic Regression, logistic function is used to describe the mathematical form on which the logistic model is based.

Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks Artificial Intelligence

In recent years, graph neural networks (GNNs) have emerged as a powerful neural architecture to learn vector representations of nodes and graphs in a supervised, end-to-end fashion. Up to now, GNNs have only been evaluated empirically---showing promising results. The following work investigates GNNs from a theoretical point of view and relates them to the $1$-dimensional Weisfeiler-Leman graph isomorphism heuristic ($1$-WL). We show that GNNs have the same expressiveness as the $1$-WL in terms of distinguishing non-isomorphic (sub-)graphs. Hence, both algorithms also have the same shortcomings. Based on this, we propose a generalization of GNNs, so-called $k$-dimensional GNNs ($k$-GNNs), which can take higher-order graph structures at multiple scales into account. These higher-order structures play an essential role in the characterization of social networks and molecule graphs. Our experimental evaluation confirms our theoretical findings as well as confirms that higher-order information is useful in the task of graph classification and regression.