Word similarity has many applications to social science and cultural analytics tasks like measuring meaning change over time and making sense of contested terms. Yet traditional similarity methods based on cosine similarity between word embeddings cannot capture the context-dependent, asymmetrical, polysemous nature of semantic similarity. We propose a new measure of similarity, Word Confusion, that reframes semantic similarity in terms of feature-based classification confusion. Word Confusion is inspired by Tversky's suggestion that similarity features be chosen dynamically. Here we train a classifier to map contextual embeddings to word identities and use the classifier confusion (the probability of choosing a confounding word c instead of the correct target word t) as a measure of the similarity of c and t. The set of potential confounding words acts as the chosen features. Our method is comparable to cosine similarity in matching human similarity judgments across several datasets (MEN, WirdSim353, and SimLex), and can measure similarity using predetermined features of interest. We demonstrate our model's ability to make use of dynamic features by applying it to test a hypothesis about changes in the 18th C. meaning of the French word "revolution" from popular to state action during the French Revolution. We hope this reimagining of semantic similarity will inspire the development of new tools that better capture the multi-faceted and dynamic nature of language, advancing the fields of computational social science and cultural analytics and beyond.

Even these ideas are not so novel. For example, the local reparametrization trick is something that we use all the time when we do Variational Bayes (VB) (say in a logistic regression model) and transform high-dimensional integrals into one-dimensional integrals under a Gaussian approximate posterior. For example, if you have a likelihood of the form \prod_{i 1} n \sigma(w T x_i) and apply VB with q(w mu,Sigma), then you end up with a sum of expectations of the form \sum_{i 1} n q(w mu,Sigma) \log \sigma(w T x_i) d w and then the local reparametrization trick is applied to transform each separate (initially high-dimensional integral over the vector w) into a 1-D integral over the univariate standard normal. The authors essentially use this separately for each activation unit and apply stochastic approximation instead of integration. Having said that, I must admit that as far as the stochastic variational inference algorithms are concerned and the related research community (born a couple of years ago!) the use of this local reparametrization trick, as far as I know, is novel and people should know about it because it is useful.

Additional Feedback: * One philosophical question that comes to mind when reading the three observations in the Introduction and while going over the example in Figure 1 is the following: Could it be that the exact reason that the representation methods learn interesting patterns is the fact that they are allowed to twist and curve the space as required, in order to bring nodes that are far apart in terms of graph distance close together in terms of Euclidean distance? It is not obvious to me that constraining the optimization algorithm of this ability can only have positive outcome. There's a parallel to be drawn here with the kernel trick in Support Vector Machines, where we are allowed to embed the data in a higher dimension, where the classes become linearly separable. This way, the sum could be defined over (q', q'') \in \Gamma_{i, j} * The sectional curvature could have been more thoroughly introduced. Unfortunately, for a paper that is heavily based on geometric notions, there's a clear shortage of pictures.

The paper uses an online approximation to MCMC to draw parameters for a Bayesian neural network. The predictive distribution under these samples is then fitted using stochastic approximation. The comparisons are to recent work on approximate Bayesian inference applied to the same models and example problems. The paper does not yet present demonstrate that these methods will push forward any particular application. The paper is a fairly natural extension of existing work.

Additional Feedback: It would be interesting to see a discussion of how this work lies in comparison to classes of knowledge bases that enable tractable abductive reasoning [1]. For example, is this result a special case of some known class/language? I just wanted to address the author's request for specific references "that might cast doubt on the novelty of our work". Sorry for not being more concrete, but here are some specific references. David Eppstein The polynomial time enumeration algorithm proposed for Eq 16 is basically subset sum where we enumerate all subsets that sum less than some threshold.

What is the lower-bound the paper refer to? - Currently the way that the paper is written does not highlight the novelties of the work. It seems the work combines existing methods for solving min-max with exiting variance reduction modules inside. Please clarify the novelties in the analysis further (the explanation in section 5 does not highlight the difficulties and challenges).

The reviewers all agree that the improved complexity is new for the class of stochastic nonconvex-strongly-concave minimax problems, and the claim of optimal complexity is clarified in the rebuttal. The weakness pointed out by the reviewers is the seemingly lack of novelty by combining existing techniques of SGDA and SARAH gradient estimators, but the author rebuttal is convincing that the combination in the minimax setting requires innovation. Another weakness is that the SREDA algorithm is rather complex, having multi-level loops and can be hard to tune in practice. Overall I recommend acceptance based on the value of the theoretical improvement. The authors should carefully address the remaining concerns of the reviewers in the revision, especially to clarify the claim of optimal complexity.

This paper considers the online trend estimation in the non-stationary stochastic optimization framework, where the comparator sequence satisfy certain variational constraints. The main contribution is a polynomial time policy extending Vovk-Azoury-Warmuth forecaster the achieves the minimax optimal rate for dynamic regret. All reviewers liked the paper, appreciating connecting the batch non-parametric regression to online stochastic optimization, techniques from wavelet computation, a model based on variational constraints which nicely captures sparsity and intensity of changes, and the (asymptotically) optimal algorithm.

Weaknesses: The dictionary used in reconstructing HR images is hand-crafted. Why can the filters in the dictionary not be learned as kernels in neural network and enjoy the benefit of end-to-end learning as many pure deep learning-based SISR method? In the experiment, when comparing with SOTA SISA methods, only x2 and x4 results are shown while x3 results are missing. The authors are recommended to provide x3 results as well. In addition, FALSR-C and FALSR-A in Table 2 used only DIV2K as the training set, while the training set of the proposed method are both DIV2K and Flickr2K, and thus the comparison here is not fair.

This submission proposes to do single image super-resolution using a network which produces coefficients for a fixed bank of Gaussian/DoG filters. The super-resolution results produce nearly SotA super-resolution PSNR while the proposed approach is 1-2 orders of magnitude more efficient than SotA. Reviewers liked the idea of incorporating a filter bank dictionary. While all of the reviewers felt that these weaknesses put the submission below the acceptance threshold, metareviewers felt that the authors' response adequately addressed each of these concerns. Please add comparisons with the SotA approaches (EDSR, RCAN, ESRGAN, ProSR) in terms of PSNR, efficiency (MultAdds), and parameter count.