More recently, there has been a growing interest in automated analysis of high-dimensional video data collected during experiments. Here we introduce a probabilistic framework for the analysis of behavioral video and neural activity.

It is important to also demonstrate the robustness of the proposed algorithm on data relative to the naive model.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The authors present a hierarchical extension of the IRM for network modelling using the key ideas from the Bayesian rose tree paper: 1) that the hierarchy is used to specify a mixture over consistent partitions of the nodes 2) that this hierarchy can be learnt using an efficient greedy agglomerative procedure. Qualitative results on the Sampson's monks dataset, and full NIPS dataset, and quantitative results on the NIPS-234 dataset are presented. The proposed inference is computational much cheaper than the IRM, whilst obtaining similar predictive performance. The paper is very well written and the exposition of the key ideas is clear.

We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The paper proposes an estimation strategy for recovering the parameters of a finite state Markov chain given observed stationary frequencies of some states. Although the problem proposed is potentially very interesting, the paper does not appear to be in a mature state. Some fundamental issues are not adequately addressed, while the evaluation is limited, and the writing quality is not strong. Note that there is an uncountable set of ergodic transition models that can exactly match a given subset of stationary frequencies when the number of observed stationary state frequencies is small relative to the total number of states.

Neural Information Processing Systems http://nips.cc/

SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the non-i.i.d.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. This paper extends the stochastic gradient langevin dynamics by using the Reimannian structure and applies it into probability simplex. The idea appears to be quite interesting. But there are several confusing parts that I don't quite get. Maybe the authors can elaborate those a bit.

A unique benefit to this approach is that each transition's TD error can be