SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the non-i.i.d.

We consider the problem of fitting variational posterior approximations using stochastic optimization methods.

We consider the problem of fitting variational posterior approximations using stochastic optimization methods.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. This paper presents a Bayesian approach to state and parameter estimation in nonlinear state-space models, while also learning the transition dynamics through the use of a Gaussian process (GP) prior. The inference mechanism is based on particle Markov chain Monte Carlo (PMCMC) with the recently-introduced idea of ancestor sampling. The paper also discusses computational efficiencies to be had with respect to sparsity and low-rank Cholesky updates. This is a technically sound and strong paper with clear and accessible presentation.

Neural Information Processing Systems http://nips.cc/

Experiments are usually performed in one environment (e.g., in a lab, on Earth) with the

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. This study investigates two algorithms for fast inference in generative models of olfaction. Their goal is to compute the most likely, linear mixture of odors comprising an olfactory stimulus. One of the algorithms employs variational inference, while the other is based on a sampling scheme. Simulations demonstrate that both algorithms perform suitably well, and the authors claim that inference is performed rapidly within the first 100 ms, while eliminating false positives (detection of odors not present in a particular stimulus) takes much longer and is difficult when more than two odors are present.

Ideally, we would prefer an estimator that learns any distribution.

Neural Information Processing Systems http://nips.cc/