I wrote a series of blog posts on Bayesian modeling with R and Stan. Stan is a growing platform for MC(MC) computing implemented with C . Compared to WinBUGS or OpenBUGS, it is very fast and programmable intuitively. This series of the posts show how to install Stan on R, how to run it, and how to apply it to actual datasets. I hope you'll find it to practice Bayesian modeling easier than ever.

Many retailers today employ inventory management systems based on Re-Order Point Policies, most of which rely on the assumption that all decreases in product inventory levels result from product sales. Unfortunately, it usually happens that small but random quantities of the product get lost, stolen or broken without record as time passes, e.g., as a consequence of shoplifting. This is usual for retailers handling large varieties of inexpensive products, e.g., grocery stores. In turn, over time these discrepancies lead to stock freezing problems, i.e., situations where the system believes the stock is above the re-order point but the actual stock is at zero, and so no replenishments or sales occur. Motivated by these issues, we model the interaction between sales, losses, replenishments and inventory levels as a Dynamic Bayesian Network (DBN), where the inventory levels are unobserved (i.e., hidden) variables we wish to estimate. We present an Expectation-Maximization (EM) algorithm to estimate the parameters of the sale and loss distributions, which relies on solving a one-dimensional dynamic program for the E-step and on solving two separate one-dimensional nonlinear programs for the M-step.

This is part 3/3 of a series on deep belief networks. Part 1 focused on the building blocks of deep neural nets – logistic regression and gradient descent. Part 2 focused on how to use logistic regression as a building block to create neural networks, and how to train them. Part 3 will focus on answering the question: "What is a deep belief network?" and the algorithms we use to do training and prediction. In its simplest form, a deep belief network looks exactly like the artificial neural networks we learned about in part 2! As long as there is at least 1 hidden layer, the model is considered to be "deep".

The virtual brain will be an exceptional tool giving neuroscientists a new understanding of the brain and a better understanding of neurological diseases. The Blue Brain project began in 2005 with an agreement between the EPFL and IBM.The Blue Brain Project goal is to create a synthetic brain by reverse engineering the mammalian brain down to the molecular level. The project, founded in May 2005 in Switzerland, is to study the brain's architectural and functional principles. There are a number of sub-projects, including the Cajal Blue Brain, and others run by universities and independent laboratories. HNeT Application Development System which was released in 1990 contained a number of example applications, based on the complex valued phase coherence/decoherence process including complex valued Hopfield network or complex associative memory.

Reading the academic literature Text Analytics seems difficult. However, applying it in practice has shown us that Text Classification is much easier than it looks. Most of the Classifiers consist of only a few lines of code.In this three-part blog series we will examine the three well-known Classifiers; the Naive Bayes, Maximum Entropy and Support Vector Machines. From the introductionary blog we know that the Naive Bayes Classifier is based on the bag-of-words model. With the bag-of-words model we check which word of the text-document appears in a positive-words-list or a negative-words-list.

Most random number generators use an algorithm a(k 1) f(a(k)) to produce a sequence of integers a(1), a(2), etc. that behaves like random numbers. The function f is integer-valued and bounded; because of these two conditions, the sequence a(k) eventually becomes periodic for k large enough. This is an undesirable property, and many public random number generators (those built in Excel, Python, and other languages) are poor and not suitable for cryptographic applications, Markov Chains Monte-Carlo associated with hierarchical Bayesian models, or large-scale Monte-Carlo simulations to detect extreme events (example: fraud detection, big data context). It's easy to build a sequence of non-periodic numbers or digits, by using irrational numbers. By construction, this decimal number has no period.

So you know the Bayes rule. How does it relate to machine learning? It can be quite difficult to grasp how the puzzle pieces fit together – we know it took us a while. This article is an introduction we wish we had back then. While we have some grasp on the matter, we're not experts, so the following might contain inaccuracies or even outright errors.

We study the robustness of active learning (AL) algorithms against prior misspecification: whether an algorithm achieves similar performance using a perturbed prior as compared to using the true prior. In both the average and worst cases of the maximum coverage setting, we prove that all $\alpha$-approximate algorithms are robust (i.e., near $\alpha$-approximate) if the utility is Lipschitz continuous in the prior. We further show that robustness may not be achieved if the utility is non-Lipschitz. This suggests we should use a Lipschitz utility for AL if robustness is required. For the minimum cost setting, we can also obtain a robustness result for approximate AL algorithms. Our results imply that many commonly used AL algorithms are robust against perturbed priors. We then propose the use of a mixture prior to alleviate the problem of prior misspecification. We analyze the robustness of the uniform mixture prior and show experimentally that it performs reasonably well in practice.

Quantitatively assessing relationships between latent variables and observed variables is important for understanding and developing generative models and representation learning. In this paper, we propose latent-observed dissimilarity (LOD) to evaluate the dissimilarity between the probabilistic characteristics of latent and observed variables. We also define four essential types of generative models with different independence/conditional independence configurations. Experiments using tractable real-world data show that LOD can effectively capture the differences between models and reflect the capability for higher layer learning. They also show that the conditional independence of latent variables given observed variables contributes to improving the transmission of information and characteristics from lower layers to higher layers.

You know that you want to build a predictive model. You've framed your problem in terms of classification or regression. You've prepared some training data (which took an age). You've heard or experienced first hand that Random Forests, Elastic Net Regression or Deep Belief Networks are "the business" and so you're going to use one of these (you've probably already verified that these algorithms are appropriate to your problem based on their general capabilities: whether it be their ability to deal with real valued data, "big" streaming data, multiple classes and so on). However, no two algorithms are the same (if they were we'd simply have fewer to choose from). As such there are a host of questions that you may not have even thought to ask which could make or break your choice.