We present an algorithmic solution to the problem of incremental belief updating in the context of Monte Carlo inference in Bayesian statistical models represented by probabilistic programs. Given a model and a sample-approximated posterior, our solution constructs a set of weighted observations to condition the model such that inference would result in the same posterior. This problem arises e.g. in multi-level modelling, incremental inference, inference in presence of privacy constraints. First, a set of virtual observations is selected, then, observation weights are found through a computationally efficient optimization procedure such that the reconstructed posterior coincides with or closely approximates the original posterior. We implement and apply the solution to a number of didactic examples and case studies, showing efficiency and robustness of our approach. The provided reference implementation is agnostic to the probabilistic programming language or the inference algorithm, and can be applied to most mainstream probabilistic programming environments.

The'planning as inference' paradigm extends Bayesian inference to future observations. The agent in the environment is modelled as a Bayesian generative model, but the belief about the distribution of agent's actions is updated based on future goals rather than on past facts. This allows to use common modelling and inference tools, notably probabilistic programming, to represent computer agents and explore their behavior. Representing agents as general programs provides flexibility compared to restricted approaches, such as Markov decision processes and their variants and extensions, and allows to model a broad range of complex behaviors in a unified and natural way. Planning as inference models agent preferences through conditioning agents on preferred future behaviors. Often, the conditioning is achieved through the Boltzmann distribution: the probability of a realization of agent's behavior is proportional to the exponent of the agent's reward. The motivation of using the Boltzmann distribution is not clear though. A'rational' agent should behave in a way that maximizes the agent's expected utility, shouldn't it? One argument is that the Boltzmann distribution models human errors and irrationality.

AI planning can be cast as inference in probabilistic models, and probabilistic programming was shown to be capable of policy search in partially observable domains. Prior work introduces policy search through Markov chain Monte Carlo in deterministic domains, as well as adapts black-box variational inference to stochastic domains, however not in the strictly Bayesian sense. In this work, we cast policy search in stochastic domains as a Bayesian inference problem and provide a scheme for encoding such problems as nested probabilistic programs. We argue that probabilistic programs for policy search in stochastic domains should involve nested conditioning, and provide an adaption of Lightweight Metropolis-Hastings (LMH) for robust inference in such programs. We apply the proposed scheme to stochastic domains and show that policies of similar quality are learned, despite a simpler and more general inference algorithm. We believe that the proposed variant of LMH is novel and applicable to a wider class of probabilistic programs with nested conditioning.

We propose to distinguish between deterministic conditioning, that is, conditioning on a sample from the joint data distribution, and stochastic conditioning, that is, conditioning on the distribution of the observable variable. Mostly, probabilistic programs follow the Bayesian approach by choosing a prior distribution of parameters and conditioning on observations. In a basic setting, individual observations are In a basic setting, individual observations are samples from the joint data distribution. However, observations may also be independent samples from marginal data distributions of each observable variable, summary statistics, or even data distributions themselves . These cases naturally appear in real life scenarios: samples from marginal distributions arise when different observations are collected by different parties, summary statistics (mean, variance, and quantiles) are often used to represent data collected over a large population, and data distributions may represent uncertainty during inference about future states of the world, that is, in planning. Probabilistic programming languages and frameworks which support conditioning on samples from the joint data distribution are not directly capable of expressing such models. We define the notion of stochastic conditioning and describe extensions of known general inference algorithms to probabilistic programs with stochastic conditioning. In case studies we provide probabilistic programs for several problems of statistical inference which are impossible or difficult to approach otherwise, perform inference on the programs, and analyse the results.

We propose design guidelines for a probabilistic programming facility suitable for deployment as a part of a production software system. As a reference implementation, we introduce Infergo, a probabilistic programming facility for Go, a modern programming language of choice for server-side software development. We argue that a similar probabilistic programming facility can be added to most modern general-purpose programming languages. Probabilistic programming enables automatic tuning of program parameters and algorithmic decision making through probabilistic inference based on the data. To facilitate addition of probabilistic programming capabilities to other programming languages, we share implementation choices and techniques employed in development of Infergo. We illustrate applicability of Infergo to various use cases on case studies, and evaluate Infergo's performance on several benchmarks, comparing Infergo to dedicated inference-centric probabilistic programming frameworks.

We introduce an algorithmic method for population anomaly detection based on gaussianization through an adversarial autoencoder. This method is applicable to detection of `soft' anomalies in arbitrarily distributed highly-dimensional data. A soft, or population, anomaly is characterized by a shift in the distribution of the data set, where certain elements appear with higher probability than anticipated. Such anomalies must be detected by considering a sufficiently large sample set rather than a single sample. Applications include, but not limited to, payment fraud trends, data exfiltration, disease clusters and epidemics, and social unrests. We evaluate the method on several domains and obtain both quantitative results and qualitative insights.

We present a probabilistic model of an intrusion in a marked renewal process. Given a process and a sequence of events, an intrusion is a subsequence of events that is not produced by the process. Applications of the model are, for example, online payment fraud with the fraudster taking over a user's account and performing payments on the user's behalf, or unexpected equipment failures due to unintended use. We adopt Bayesian approach to infer the probability of an intrusion in a sequence of events, a MAP subsequence of events constituting the intrusion, and the marginal probability of each event in a sequence to belong to the intrusion. We evaluate the model for intrusion detection on synthetic data, as well as on anonymized data from an online payment system.

We introduce a methodology for efficient monitoring of processes running on hosts in a corporate network. The methodology is based on collecting streams of system calls produced by all or selected processes on the hosts, and sending them over the network to a monitoring server, where machine learning algorithms are used to identify changes in process behavior due to malicious activity, hardware failures, or software errors. The methodology uses a sequence of system call count vectors as the data format which can handle large and varying volumes of data. Unlike previous approaches, the methodology introduced in this paper is suitable for distributed collection and processing of data in large corporate networks. We evaluate the methodology both in a laboratory setting on a real-life setup and provide statistics characterizing performance and accuracy of the methodology.

In this work, we explore how probabilistic programs can be used to represent policies in sequential decision problems. In this formulation, a probabilistic program is a black-box stochastic simulator for both the problem domain and the agent. We relate classic policy gradient techniques to recently introduced black-box variational methods which generalize to probabilistic program inference. We present case studies in the Canadian traveler problem, Rock Sample, and a benchmark for optimal diagnosis inspired by Guess Who. Each study illustrates how programs can efficiently represent policies using moderate numbers of parameters.

We introduce an adaptive output-sensitive Metropolis-Hastings algorithm for probabilistic models expressed as programs, Adaptive Lightweight Metropolis-Hastings (AdLMH). The algorithm extends Lightweight Metropolis-Hastings (LMH) by adjusting the probabilities of proposing random variables for modification to improve convergence of the program output. We show that AdLMH converges to the correct equilibrium distribution and compare convergence of AdLMH to that of LMH on several test problems to highlight different aspects of the adaptation scheme. We observe consistent improvement in convergence on the test problems.