Inductive Learning
Review for NeurIPS paper: LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration
Weaknesses: This is not a weakness per-say, but a suggestion to make the paper stronger. In juxtaposition to the existing work the authors present the argument several times that using a UV parameterization is inherently inferior to 3D representations, as it requires seam-cuts and results in distortion of highly curved regions, etc. While this is conceptually correct and true, it would have made the paper stronger if the authors had somehow demonstrated this to be true empirically as well for their problem. For example, perhaps via a simpler problem -- maybe for the fully-supervised case or for the case when the entire pipeline is not necessarily end-to-end differentiable, but a combination of a landmarks/correspondence estimation a traditional optimization approach. It would be interesting to see if the signed distance representation to predict correspondences with a CNN along with its Lagrangian loss formulation to encourage points to lie on the surface improves the accuracy of correspond prediction by itself and if so by how much versus an approach that learns to map scan points to the UV space instead.
Review for NeurIPS paper: LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration
The rebuttal addressed the main criticisms raised by the reviewers: the assumption on warm start, the robustness to noise, and the clarification of the model. The answers of the authors contributed to the discussion and the proper evaluation of this work. The terminological issue doesn't affect the final decision.
Review for NeurIPS paper: Uncertainty Aware Semi-Supervised Learning on Graph Data
Clarity: Overall the paper is very clear. The authors did an excellent job. Equation 5 - I am confused on a few things. The notation P(y x; theta) is confusing because the semicolon implies that theta is a vector and not a random vector, however, the conditional distribution of theta is given P(theta G). So what is the point of the semicolon? Also, there is a typo in Equation 5 I think because the entropy term is not defined correctly.
Review for NeurIPS paper: Uncertainty Aware Semi-Supervised Learning on Graph Data
R#2 and R#3 generally liked the paper. R#1 has a brief review that raised concern on novelty of the method. The rebuttal well addressed the concerns and made all reviewers increase their score. We have collected comments from an additional reviewer, who pointed out more issues on writing and the theoretical results (see blew). We advise the authors to take efforts to address these issues in the revision.
Reviews: Machine Teaching of Active Sequential Learners
This paper considers the problem of teaching an active sequential *machine learner* (e.g., an active learning algorithm), with a teacher which can "fake" the labels/outcomes of training examples with the goal of steering the learner faster to the goal state. The authors refer to such teacher a "planning teacher", as opposed to a "naive teacher" which is often considered in the classical machine teaching problems. This setting differs from conventional machine teaching settings, in that In classical machine teaching setting, the teacher can only choose among a given set of training examples that are consistent with the target concept, and is often not allowed to provide inconsistent examples. The majority of the existing work in machine teaching considers teaching a "passive learner", with a few exceptions (see additional reference in the comments below). The assumption that the teacher can choose the data-generation distribution makes it a very powerful teacher with a much richer action set than conventional teaching.
Review for NeurIPS paper: wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Weaknesses: A weakness of this work is that from this work alone it's not clear why the proposed changes should work well for the problem domain. Moreover, why the interaction of the two proposed changes is so beneficial. While this is a problem in the body of work that effectively searches through the neural-network-architecture space, it would be very beneficial to try and focus on justifying more rigorously the design choices made. An example of how this could be done is designing a toy problem that exemplifies that pre-existing work cannot handle this case, and such, the proposed changes should be accepted. As a result of this, it's not clear how significant this work is/will be.
Review for NeurIPS paper: wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
This paper proposes an end-to-end self-supervised learning approach for speech representations. It can serve as the unsupervised pre-training for fast and robust deployment of automatic speech recognition systems, especially for those with low resource or limited amounts of labeled data. The authors reported compelling performance of the proposed technique on Librispeech and TIMIT. This is a strong paper and all reviewers are supportive for acceptance. Large-scale unsupervised pre-training has made great impacts in vision and NLP, the work reported here is analogous in the speech community in that effort.
Review for NeurIPS paper: Fairness constraints can help exact inference in structured prediction
My biggest concern is with the way that \epsilon_1 behaves depending on n. Firstly, it seems the choice of the value -n for \rho is arbitrary (with the choice being repercuted in the definition of \epsilon), and this should be discussed more clearly in the text. Next, it is not clear to me why the choice \rho -n is the best. Does it optimize \epsilon_1 in some way? Furthermore, as n tends to \infty, it seems that \epsilon_1 does NOT tend to infinity.
Review for NeurIPS paper: Fairness constraints can help exact inference in structured prediction
This paper is about structures output predictions analysis under'fairness' constraint. This paper shows that constraints relative to fairness can help to increases accuracies. Fairness is one of the notion whose importance is rising in our community, and this paper give interesting insights about it. One of the main issue raised by one of the reviewer that pleads for non acceptance is the "vagueness" of the definition of fairness here. I personally think that this issue should not be taken to much into account here, there is still in our community some "vagueness" according to what the good definition should be.
Reviews: Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
The authors present a way of self-supervised auxiliary learning in which the images in the training set are rotated with 4 different rotations, and the neural network has to predict the type of rotation. The authors show with various experiments that this type of SSL increases the robustness against all kinds of perturbations, ranging from adversarial attacks to motion blur and fog. In addition, the outputs indicating the rotation can be used for detecting outliers. The article makes a good case for both contributions. One main remark is that the title of the article talks about uncertainty estimation, while the experiments focus on outlier detection. These two tasks are related but not identical.