Pyro: Deep Universal Probabilistic Programming

arXiv.org Machine Learning

Pyro is a probabilistic programming language built on Python as a platform for developing advanced probabilistic models in AI research. To scale to large datasets and high-dimensional models, Pyro uses stochastic variational inference algorithms and probability distributions built on top of PyTorch, a modern GPU-accelerated deep learning framework. To accommodate complex or model-specific algorithmic behavior, Pyro leverages Poutine, a library of composable building blocks for modifying the behavior of probabilistic programs.


Extending Stan for Deep Probabilistic Programming

arXiv.org Artificial Intelligence

Deep probabilistic programming combines deep neural networks (for automatic hierarchical representation learning) with probabilistic models (for principled handling of uncertainty). Unfortunately, it is difficult to write deep probabilistic models, because existing programming frameworks lack concise, high-level, and clean ways to express them. To ease this task, we extend Stan, a popular high-level probabilistic programming language, to use deep neural networks written in PyTorch. Training deep probabilistic models works best with variational inference, so we also extend Stan for that. We implement these extensions by translating Stan programs to Pyro. Our translation clarifies the relationship between different families of probabilistic programming languages. Overall, our paper is a step towards making deep probabilistic programming easier.


Learning Probabilistic Programs

arXiv.org Artificial Intelligence

We develop a technique for generalising from data in which models are samplers represented as program text. We establish encouraging empirical results that suggest that Markov chain Monte Carlo probabilistic programming inference techniques coupled with higher-order probabilistic programming languages are now sufficiently powerful to enable successful inference of this kind in nontrivial domains. We also introduce a new notion of probabilistic program compilation and show how the same machinery might be used in the future to compile probabilistic programs for efficient reusable predictive inference.


New AI programming language goes beyond deep learning

#artificialintelligence

A team of MIT researchers is making it easier for novices to get their feet wet with artificial intelligence, while also helping experts advance the field. In a paper presented at the Programming Language Design and Implementation conference this week, the researchers describe a novel probabilistic-programming system named "Gen." Users write models and algorithms from multiple fields where AI techniques are applied -- such as computer vision, robotics, and statistics -- without having to deal with equations or manually write high-performance code. Gen also lets expert researchers write sophisticated models and inference algorithms -- used for prediction tasks -- that were previously infeasible. In their paper, for instance, the researchers demonstrate that a short Gen program can infer 3-D body poses, a difficult computer-vision inference task that has applications in autonomous systems, human-machine interactions, and augmented reality.


New AI programming language goes beyond deep learning

#artificialintelligence

A team of MIT researchers is making it easier for novices to get their feet wet with artificial intelligence, while also helping experts advance the field. In a paper presented at the Programming Language Design and Implementation conference this week, the researchers describe a novel probabilistic-programming system named "Gen." Users write models and algorithms from multiple fields where AI techniques are applied -- such as computer vision, robotics, and statistics -- without having to deal with equations or manually write high-performance code. Gen also lets expert researchers write sophisticated models and inference algorithms -- used for prediction tasks -- that were previously infeasible. In their paper, for instance, the researchers demonstrate that a short Gen program can infer 3-D body poses, a difficult computer-vision inference task that has applications in autonomous systems, human-machine interactions, and augmented reality.