Update: I have read the author response and am satisfied with the commitment to elaborate on \beta and \pi and to simplify the Stan PE code with a "pseudo-extended" function. This paper presents a new MCMC sampling method called pseudo-extended MCMC that uses an instrumental distribution to projects the data into a higher-dimensional space where the modes are connected, making it easier for the sampler to mix. A default instrumental distribution based on tempering is provided. The method is compared to existing baselines showing efficacy on three benchmark datasets. The paper is well-placed within the existing literature.

Reviewers reached consensus that the paper makes a valuable contribution for MCMC. There are specific suggestions for improving the experiments that we ask the authors to seriously consider.

While stochastic gradient descent (SGD) is still the de facto algorithm in deep learning, adaptive methods like Clipped SGD/Adam have been observed to outperform SGD across important tasks, such as attention models. The settings under which SGD performs poorly in comparison to adaptive methods are not well understood yet. In this paper, we provide empirical and theoretical evidence that a heavy-tailed distribution of the noise in stochastic gradients is one cause of SGD's poor performance. We provide the first tight upper and lower convergence bounds for adaptive gradient methods under heavy-tailed noise. Further, we demonstrate how gradient clipping plays a key role in addressing heavy-tailed gradient noise.

Strengths: Soundness: The model formulation appears to be mathematically sound. As with several previous works, the authors utilize linear-Gaussian distributions for dynamics, which have the benefit of permitting exact computation of expectations, e.g. The authors propose two main improvements over related models: 1) the use of recurrent switch transitions through Gaussian switch variables, and 2) non-linear emission models through the use of an additional (auxiliary) latent variable, z. They train this model with a sequential Monte Carlo objective utilized in previous works. This paper builds off of many of the theoretical developments of previous works, adding a couple of useful techniques.

This paper proposes an extension to state-switching Gaussian linear systems, which extends the ideas of traditional Kalman filtering to the setting of switching states.

This paper studies how to test the stationarity of stochastic gradient with momentum using some advanced testing statistics that take the time correlations into accounts. Extensive experiments are run to demonstrate the advantage of the proposed method over existing approaches. Originality: The paper is based on extending a recent paper by Yaida. It does not seem that original to me but the authors do combine the condition by Yaida with some more advanced testing statistics in a new way. Overall I think the extension is quite natural, so the conceptual novelty is not that high.

The paper proposes to automate the tuning of learning rate schedules in stochastic gradient methods, which is an important problem. In this regards, the authors propose a statistical test to determine when to decay the learning rate. The statistical test build upon a prior work with simple albeit useful extensions. Resulting statistical test is simple and can be deployed easily. There are some concerns regarding mismatch between theoretical assumptions made and the setup in practice. Nevertheless, empirically the learning rate schedule followed by decaying when the test is true seems to be almost competitive with hand-tuned methods.

Additional Feedback: Overall, I think that this paper fits the NeurIPS standard and will be interesting to the community, so I recommend accepting it. However, I have a few questions I wish the authors could answer: • The issue with {\hat y} is puzzling. Almost all prior work on strategic classification assumes that the agents can only misreport their feature vectors while the labels are the agents' inherent property. In this paper, as a byproduct of defining h*, agents can hypothetically change their label too. However, the semi-formal discussion in lines 96-100 forbids it. Formally, this is equivalent to restricting the agent (the optimization problem in line 92) to pick z that with h*(z) y_t.

The author response addressed all major concerns of the reviewers. The paper makes an interesting and important contribution to strategic classification, and we are happy to recommend acceptance.

Relation to Prior Work: The paper is missing references to and comparisons with important related works on adversarial examples for nearest neighbors and other non-parametric methods. For example, [1] provides a direct convergence rate for robustness of nearest neighbors to adversarial examples; it would be good to discuss how the bounds in this paper are different. Similarly, the convex program proposed by this paper feels similar to [5] as well as [2]; it would be good to discuss the relationship between these works.