This paper considers the problem of developing flexible choice models that are not constrained to satisfy traditional, restrictive choice axioms (such as Luce's axiom of independence of irrelevant attributes, IIA), but that can be tractably inferred from data. A (discrete) choice model over n items specifies probabilities of the form p(i,S) Prob( i chosen from S) for each subset of items S \subseteq [n] and each item i \in S. One of the most widely used models of discrete choice is the multinomial logit (MNL) choice model, which can be inferred efficiently from data but which is constrained to satisfy IIA and other restrictive assumptions. The paper proposes a new Markov chain based model of discrete choice that is parametrized by a (n x n) pairwise selection probability matrix. The model avoids several of the earlier restrictive assumptions, but is shown to satisfy an interesting property termed contractibility, which in turn also implies a reasonable property of uniform expansion. Parameter estimation in the model is done by maximum likelihood (the log-likelihood function is non-concave in general, but the experiments suggest that good parameters are learned).

The paper is well-written and clear. The proposed idea is interesting. I have the following comments/questions: 1) Does the Liptschiz assumption hold here with a probability or is it assumed to always hold? 2) Figure 1: should it be \bar{s}_2 instead of s_2 in the caption? The use of bar for non-sets is confusing. I do not see the need for the last intersection in Equation 4. 4) When you repeatedly apply Equation 4, the number of states that satisfy the safety constraint shrinks because you use Liptschiz in the worst scenario sense.

The paper is globally sound and makes a new contribution to an important topic. However some technicalities need to be addressed and a revised version should be encouraged. Major remarks: - There is a confusion on whether the Markov chains under consideration are supposed to be stationary or not. Indeed, the concept of t-trail either requires that the Markov chains under study are stationary or one should specify that all the trails start with same initial distribution, i.e. these trails are observations of (X_1,X_2,X_3) and not (X_s, X_{s 1},X_{s 2}) for some s. I first understood that you adopt the second approach (as you count the parameters of initial distributions as free parameters) but in the real data experiments, you take many (3001)-trails and break them into 3000 overlapping 3-trails (by the way, you need a 3002-trail to obtain these).

Technically the paper is very strong. The results presented by the authors are, to the best of my knowledge, novel and significant. However my main criticism of the paper is that the presentation is very esoteric. The is clear already in the introduction where the authors fail to explain some of the basic notation that is central to the remaining of the paper, see (1)-(3) below. This continues throughout the paper making it hard to read for non-experts in the field, see e.g.

The proposed model is novel and practical, as seen from the experimental result. It is rare to see a Bayesian nonparametric model being applied to large data as it is generally not very scalable. It is a feat to see this model applied to data with high dimensions (9000 dimensions with millions of events). I am interested to know how much time is spent for training? It would be good to also present the computational time (say in the supplementary material).

Technical quality: This paper tackles the problem of inference and learning of factored HMMs on large sequences and with large latent dimensionality. The primary contribution of the paper is integrating several existing approaches together to enable large-scale learning of FHMMs without a loss in modeling performance. The technical details of the components of the approach (the bivariate Gaussian copula variation posterior, the recognition network, the SVI learning approach) appear to be technically correct. The experimentation touches on the correct points including the accuracy of the learned models and the scalability of the proposed approach. The accuracy of the learned models is assessed using log likelihood on held-out test data. The experiments show that the model performs similarly to the SMF approach on both simulated and real (the Bach Corals) data.

The paper refers to [2] and says that those authors proved statistical consistency. However, I am then surprised to see in section 4.3 that non-zero shrinkage is obtained (including for gamma 0) for the very simple case of modelling a N(0,I) distribution with N(0, sigma 2 I). What is going on here?? A failure of consistency would be a serious flaw in the formulation of a statistical learning criterion. Also in sec 3 (Stability and KL regularization) the authors say that at least for learning based on samples (\hat{p}_{theta}) that some regularization wrt the KL divergence is required. This clearly weakens the "purity" of the smoothed Wasserstein objective fn.

This is a well-done paper. It attacks a problem that is worthwhile: how to construct and train a sequence-to-sequence model that can operate on-line instead of waiting for an entire input to be received. It clearly describes an architecture for solving the problem, and walks the reader through the issues in the design of each component in the architecture: next-step prediction, the attention mechanism, and modeling the ends of blocks. It clearly explains the challenges that need to be overcome train the model and perform inference with it, and proposes reasonable approximate algorithms for training and inference. The speech recognition experiments used to demonstrate the utility of the transducer model and to explore design issues such as maintenance of recurrent state across block boundaries, block size, design of the attention mechanism, and depth of the model are reasonable.

Contextual MDPs are a specific type of POMDPs with the restriction that the optimal q-function depends only on the most recent observation (instead of the belief state). The authors show that Contextual MDPs are not poly PAC learneable even when either memoryless policies are considered or value function approximation is used. However, when both memoryless policies and value function approximation is used and the transitions are deterministic, then the model is PAC learnable in a polynomial number of episodes (and the complexity is independent of the number of observations). The paper is well written overall. The proofs are quite clear and quite thorough. I am not quite sure that the 16 pages of technical proofs in the appendix are suitable for a conference; the paper may better fit a journal format.

Autonomous agents operating in real-world scenarios frequently encounter uncertainty and make decisions based on incomplete information. Planning under uncertainty can be mathematically formalized using partially observable Markov decision processes (POMDPs). However, finding an optimal plan for POMDPs can be computationally expensive and is feasible only for small tasks. In recent years, approximate algorithms, such as tree search and sample-based methodologies, have emerged as state-of-the-art POMDP solvers for larger problems. Despite their effectiveness, these algorithms offer only probabilistic and often asymptotic guarantees toward the optimal solution due to their dependence on sampling.