GPflow is a package for building Gaussian process models in Python. It implements modern Gaussian process inference for composable kernels and likelihoods. GPflow builds on TensorFlow 2.4 and TensorFlow Probability for running computations, which allows fast execution on GPUs. The online documentation (latest release)/(develop) contains more details. It was originally created by James Hensman and Alexander G. de G. Matthews.

This series is a brief introduction to modeling uncertainty using TensorFlow Probability library. I wrote it as a supplementary material to my PyData Global 2021 talk on uncertainty estimation in neural networks. We went a long way so far! We're going to use all the knowledge we've gained and apply it to a new -- more challenging -- dataset. Let's get our hands dirty!

Bayesian neural networks utilize probabilistic layers that capture uncertainty over weights and activations, and are trained using Bayesian inference. Since these probabilistic layers are designed to be drop-in replacement of their deterministic counter parts, Bayesian neural networks provide a direct and natural way to extend conventional deep neural networks to support probabilistic deep learning. However, it is nontrivial to understand, design and train Bayesian neural networks due to their complexities. We discuss the essentials of Bayesian neural networks including duality (deep neural networks, probabilistic models), approximate Bayesian inference, Bayesian priors, Bayesian posteriors, and deep variational learning. We use TensorFlow Probability APIs and code examples for illustration. The main problem with Bayesian neural networks is that the architecture of deep neural networks makes it quite redundant, and costly, to account for uncertainty for a large number of successive layers. Hybrid Bayesian neural networks, which use few probabilistic layers judicially positioned in the networks, provide a practical solution.

Machine learning models are usually developed from data as deterministic machines that map input to output using a point estimate of parameter weights calculated by maximum-likelihood methods. However, there is a lot of statistical fluke going on in the background. For instance, a dataset itself is a finite random set of points of arbitrary size from a unknown distribution superimposed by additive noise, and for such a particular collection of points, different models (i.e. Hence, there is some uncertainty about the parameters and predictions being made. Bayesian statistics provides a framework to deal with the so-called aleoteric and epistemic uncertainty, and with the release of TensorFlow Probability, probabilistic modeling has been made a lot easier, as I shall demonstrate with this post.

As described in [1][2], time series data includes many kinds of real experimental data taken from various domains such as finance, medicine, scientific research (e.g., global warming, speech analysis, earthquakes), etc. Time series forecasting has many real applications in various areas such as forecasting of business (e.g., sales, stock), weather, decease, and others [2]. Statistical modeling and inference (e.g., ARIMA model) [1][2] is one of the popular methods for time series analysis and forecasting. The philosophy of Bayesian inference is to consider probability as a measure of believability in an event [3][4][5] and use Bayes' theorem to update the probability as more evidence or information becomes available, while the philosophy of frequentist inference considers probability as the long-run frequency of events [3]. Generally speaking, we can use the Frequentist inference only when a large number of data samples are available.

Machine learning models are usually developed from data as deterministic machines that map input to output using a point estimate of parameter weights calculated by maximum-likelihood methods. However, there is a lot of statistical fluke going on in the background. For instance, a dataset itself is a finite random set of points of arbitrary size from a unknown distribution superimposed by additive noise, and for such a particular collection of points, different models (i.e. Hence, there is some uncertainty about the parameters and predictions being made. Bayesian statistics provides a framework to deal with the so-called aleoteric and epistemic uncertainty, and with the release of TensorFlow Probability, probabilistic modeling has been made a lot easier, as I shall demonstrate with this post (be aware that no theoretical background will be provided).

Let us focus on all the subsequent discussion in this article with Warren Buffet's point of view. As our data has been collected from YouTube and Twitter, we have to benchmark the authenticity and genuinity of the text data with the thoughts being as close to the financial world. This is necessary because we are going to train our models to predict the future of the economy and our text data transcripts have to be related to finance and economics. While collecting the data we assumed that these financial investors are quite dedicated to their field and will mostly talk publicly every time related to finance and economics. But still, we have to validate our assumed heuristic.

TensorFlow Probability (TFP) is a Python library built on TensorFlow that makes it easy to combine probabilistic models and deep learning on modern hardware (TPU, GPU). It's for data scientists, statisticians, and ML researchers/practitioners who want to encode domain knowledge to understand data and make predictions with uncertainty estimates. In this talk we focus on the "layers" module and demonstrate how TFP "distributions" fit naturally with Keras to enable estimating aleatoric and/or epistemic uncertainty. See the revamped dev site https://www.tensorflow.org/

In the first part, we explored how Bayesian Statistics might be used to make reinforcement learning less data-hungry. Now we execute this idea in a simple example, using Tensorflow Probability to implement our model. When it comes to games, it is difficult to imagine something simpler than rock, paper, scissors. Despite the simplicity, googling the game reveals a remarkable body of literature. We want to use Bayesian Statistics to play this game and exploit the biases of a human opponent.

While the overall trend in crack size with respect to cycles is increasing, it's not strictly monotonic. At certain instances, the observed crack is less than what was measured at the previous instant. This may very well be due to measurement error, a common feature in field data. In other words, the data is noisy.