Goto

Collaborating Authors

 Search


Reviews: Learning Hierarchical Priors in VAEs

Neural Information Processing Systems

This paper discussed how to enhance the existing methods in which designed prior could over regularize the posteriori, so it will try to find a way to learn a complex prior which can learn the latent pattern of data manifold more efficiently. To learn such prior, paper adopted and modified one dual optimization technique and introduced an efficient algorithm on how to update the hierarchical prior and posteriori parameters. The combination of complex priori with the introduced algorithm have learned a posterior which has more informative latent representation and avoids posteriori collapse. In addition, paper introduced a graph search method to interpolate the states and showed how effective algorithm can discover a meaningful posteriori over the experiment section. So we can summarize the contribution of this paper as following - Introduce a hierarchical prior which can avoid over regularization of the posterior while learning latent variables manifold - Adopting and expanding an optimization technique and an algorithm to learn hierarchical prior and hierarchical posterior parameters.


Reviews: Minimax Optimal Estimation of Approximate Differential Privacy on Neighboring Databases

Neural Information Processing Systems

The authors study the rates of estimating approximate differential privacy (aDP). They do so by reformulating it as a property estimation problem. I find this reduction fairly novel and ties DP to a large body of work on polynomial estimation. In property estimation, it is known that carefully trading off the bias and variance via polynomial approximation, particularly in regions of low probability, allows for obtaining the optimal min max rates. The authors follow the same recipe and show that the min max error scales as Se \epsilon / n \log n.


Reviews: Minimax Optimal Estimation of Approximate Differential Privacy on Neighboring Databases

Neural Information Processing Systems

The reviews are broadly positive, but the authors should take the following into account for the camera-ready version: - Change the title as the paper does not currently deliver what the title promises. In particular, the end-to-end problem of detecting DP-violation is not completely solved by this paper. The authors should of course also follow other reviewer comments to improve the write-up of the paper.


Reviews: SPoC: Search-based Pseudocode to Code

Neural Information Processing Systems

Originality Improving the translation procedure by leveraging error detection on the target programming language is not a new idea, even though the authors do not seem to be aware of this. In semantic parsing, this has been used quite a bit over the last year, under the name of "execution-guided decoding" (see for example the WikiSQL leaderboard, where the top 4 entries currently use this). I believe this first appeared in "Execution-Guided Neural Program Decoding" (Wang et al, 2018); the idea is closest to the prefix-based pruning idea in this submission. This leaves the new "multiclass classification" error localization technique as original contribution, as well as the new dataset. Clarity The core contribution of the paper is a new algorithm, which is only described in prose; I would have preferred pseudo-code here, which would have been more concise.


Reviews: SPoC: Search-based Pseudocode to Code

Neural Information Processing Systems

Main contributions: * New dataset of line-by-line, human-generated pseudocode for learning to map from descriptions to source code. The first stage generates a set of candidate translations from pseudocode to code for each line. The second stage enumerates over combinations of candidates, tries compiling them, and then learns to use the error messages to prioritize which combinations to explore next. There are three well-qualified reviewers who did a great job with their reviews and were active in the discussions. The discussions centered around the following points: * Is this dataset a step forward compared to NAPS?



Review for NeurIPS paper: Heuristic Domain Adaptation

Neural Information Processing Systems

Summary and Contributions: The paper presents a method inspired by heuristic search algorithms (such as A*) for the problem of Domain Adaptation (DA). The authors claim that to achieve domain invariant representations, one must explicitly model domain-specific characteristics. The ideal representation is considered as the goal and the intermediate domain-specific representations are regarded as the distance from the current representation to the ideal one. When the heuristic representations are near zero the terminal state is reached. To achieve this goal the authors propose a domain adaptation network made of a fundament network F and a heuristic network H (that is possibly divided into several subnetworks).


Reviews: Efficient Forward Architecture Search

Neural Information Processing Systems

This paper proposes novel neural architecture search method dubbed Petridish which is based on gradient boosting of "weak learners" (i.e. Originality: The main contribution of the paper is applying basic ideas from gradient-boosting of weak learners to the task of neural architecture search. This is an original idea, which allows a more guided exploration of the space of neural architectures compared to the random steps done, e.g. in evolutionary algorithms. Most related work is adequately discussed. The connection/differences to NAS methods combining network morphisms with evolutionary algorithms should be discussed in more detail as these explore the search space based on similar steps (modifying a model by small incremental additions) but select steps randomly and not based on gradient boosting.


Reviews: Fast AutoAugment

Neural Information Processing Systems

While I feel that the new random baselines significantly strengthen the paper's results on CIFAR-100, random baselines are not provided for CIFAR-10, SVHN, or ImageNet. I've updated my score from a 6 to an 7, based on the random baselines for CIFAR-100 and the authors' promise to clarify their evaluation measure in the final submission. However, Cubuk et al.'s original algorithm is extremely resource-intensive. The main contribution of this paper is an algorithm that can operate on the same search space and come up with data augmentation schemes orders of magnitude more efficiently. The most closely related work I'm aware of is Population Based Augmentation (ICML 2019), which tries to solve the same problem in a different way.


Reviews: Fast AutoAugment

Neural Information Processing Systems

This paper is concerned with automating the search for data augmentation transformations for image classification with DNN models. It does so in a way that avoids having to re-train (or fine-tune) the model for every transformation scored. This leads to a method which, compared to previous SotA (AutoAugment), is very much faster, but is shown to provide results of similar quality. While both this work and AutoAugment use a carefully choosen search space, for which neither is strongly outperforming random search over this space, the dramatic reduction in resource need over AutoAugment justifies its publication. However, the authors are asked provide further results in the final version, in particular a more thorough comparison against random search baselines with the same advanced search space, also including random repetitions, in order to convince readers their method improves enough over random search in order to justify its added complexity.