The Information Bottleneck (IB) objective uses information theory to formulate a task-performance versus robustness trade-off. It has been successfully applied in the standard discriminative classification setting. We pose the question whether the IB can also be used to train generative likelihood models such as normalizing flows. Since normalizing flows use invertible network architectures (INNs), they are information-preserving by construction. This seems contradictory to the idea of a bottleneck.

Some commonly used objective loss function such as cross-entropy loss for classification and squared loss for regression can be regarded as special cases of the logarithmic loss function.

Equation 9. Therefore if we define a standard "policy" loss L This is the "soft" version of an analogous statement made for "hard" optimality first shown in [32]. This argument is the direct counterpart to Theorem 2 in [32]--which uses argmax instead of softmax. From this point onwards, the same strategy for Proposition 2 again applies, completing the proof. Environments used for experiments are from OpenAI gym [56]. Each environment is associated with a true reward function (unknown to all imitation algorithms).

We argue that a good solution should be able to explicitly parameterize a policy (i.e.

Neural Information Processing Systems http://nips.cc/

Neural population activity in cortical circuits is not solely driven by external inputs, but is also modulated by endogenous states which vary on multiple time-scales. To understand information processing in cortical circuits, we need to understand the statistical structure of internal states and their interaction with sensory inputs. Here, we present a statistical model for extracting hierarchically organised neural population states from multi-channel recordings of neural spiking activity. Population states are modelled using a hidden Markov decision tree with state-dependent tuning parameters and a generalised linear observation model. We present a varia-tional Bayesian inference algorithm for estimating the posterior distribution over parameters from neural population recordings. On simulated data, we show that we can identify the underlying sequence of population states and reconstruct the ground truth parameters. Using population recordings from visual cortex, we find that a model with two levels of population states outperforms both a one-state and a two-state generalised linear model. Finally, we find that modelling of state-dependence also improves the accuracy with which sensory stimuli can be decoded from the population response.

Video summarization is a challenging problem with great application potential. Whereas prior approaches, largely unsupervised in nature, focus on sampling useful frames and assembling them as summaries, we consider video summarization as a supervised subset selection problem. Our idea is to teach the system to learn from human-created summaries how to select informative and diverse subsets, so as to best meet evaluation metrics derived from human-perceived quality. To this end, we propose the sequential determinantal point process (seqDPP), a probabilistic model for diverse sequential subset selection. Our novel seqDPP heeds the inherent sequential structures in video data, thus overcoming the deficiency of the standard DPP, which treats video frames as randomly permutable items. Meanwhile, seqDPP retains the power of modeling diverse subsets, essential for summarization. Our extensive results of summarizing videos from 3 datasets demonstrate the superior performance of our method, compared to not only existing unsupervised methods but also naive applications of the standard DPP model.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. The authors present a novel non-parametric Bayesian model for unsupervised clustering. The model uses a two level hierarchy of Dirichlet process priors to handle clusters which may be multi-modal, skewed and/or heavy tailed. The authors present a collapsed Gibbs sampler for inference which exploits the conjugacy of the model. The authors do an excellent job of motivating the model by explaining the deficiencies of the standard infinite mixture of Gaussians.

First provide a summary of the paper, and then address the following criteria: Quality, clarity, originality and significance. In this paper, authors analyze sparsity of the posterior parameters in LDA using a variational Bayesian algorithm. They derive an expression for the VB free energy which shows its asymptotic behaviour with respect to number of words (N), number of documents (M), vocabulary size (L) etc. Their results suggest that, for certain settings of L,M,N, the sparsity behaviour changes drastically at a particular hyper-parameter setting. These changes differ from those of MAP and partial-Bayes algorithms. The problem discussed in this paper is original, interesting, and is perhaps useful too.