Inductive Learning
Reviews: Graph-Based Semi-Supervised Learning with Non-ignorable Non-response
I think this claim should be motivated well enough, at least to me it was not entirely clear why this is important. If authors can provide some scenarios which can help understand the claim, it will be beneficial for the readers. However, the experimental analysis can be improved. One of the baselines the authors consider is "SM", but do not mention the paper in which it is proposed. They should produce results on multiple real world datasets. However, authors do not compare the proposed model with state of the art graph based SSL methods like GAT (Velickovic et al., ICML 2018) etc. [Velickovic et al., ICML 2018] Graph Attention Networks 4. Minor points: -- "vertexes" - "vertices" -- Not sure if using gradient descent qualifies as a contribution.
Reviews: Graph-Based Semi-Supervised Learning with Non-ignorable Non-response
This paper considers graph-based semi-supervised learning when missing labels are not missing at random (NMAR), in other words, the absence of a label is'nonignorable'. It introduces a graphical neural network that models the relationship between the presence or absence of a label and the labels of its neighbors. The paper also proves that this model is identifiable. The reviewers agree that studying NMAR labels in the context of neural graph embeddings is a novel and important topic. They make several suggestions for improvement, including comparing with recent work such as Velickovic et al., ICML 2018.
Meta-Reinforced Synthetic Data for One-Shot Fine-Grained Visual Recognition
This paper studies the task of one-shot fine-grained recognition, which suffers from the problem of data scarcity of novel fine-grained classes. To alleviate this problem, a off-the-shelf image generator can be applied to synthesize additional images to help one-shot learning. However, such synthesized images may not be helpful in one-shot fine-grained recognition, due to a large domain discrepancy between synthesized and original images. To this end, this paper proposes a meta-learning framework to reinforce the generated images by original images so that these images can facilitate one-shot learning. Specifically, the generic image generator is updated by few training instances of novel classes; and a Meta Image Reinforcing Network (MetaIRNet) is proposed to conduct one-shot fine-grained recognition as well as image reinforcement.
RELexED: Retrieval-Enhanced Legal Summarization with Exemplar Diversity
Santosh, T. Y. S. S., Jia, Chen, Goroncy, Patrick, Grabmair, Matthias
This paper addresses the task of legal summarization, which involves distilling complex legal documents into concise, coherent summaries. Current approaches often struggle with content theme deviation and inconsistent writing styles due to their reliance solely on source documents. We propose RELexED, a retrieval-augmented framework that utilizes exemplar summaries along with the source document to guide the model. RELexED employs a two-stage exemplar selection strategy, leveraging a determinantal point process to balance the trade-off between similarity of exemplars to the query and diversity among exemplars, with scores computed via influence functions. Experimental results on two legal summarization datasets demonstrate that RELexED significantly outperforms models that do not utilize exemplars and those that rely solely on similarity-based exemplar selection.
Reviews: Unlabeled Data Improves Adversarial Robustness
This paper presents a theoretical analysis on using unlabeled data (under a self-training scheme of a Gaussian model) to improve the robustness against adversarial noise, followed by a semi-supervised learning method to learn deep networks. The empirical results are state-of-the-art. However, this paper heavily overlaps with another paper "Are Labels Required for Improving Adversarial Robustness?". As a condition to accepting and including the paper in the proceedings, put the following disclaimer in the footnote on the first page: "The authors declare that the present paper is independent of "Are Labels Required for Improving Adversarial Robustness?"."
Reviews: Exact inference in structured prediction
Overview: - This paper studies the conditions for exact recovery of ground-truth labels in structured prediction under some data generation assumptions. In particular, the analysis generalizes the one in Globerson et al. (2015) from grid graphs to general connected graphs, providing high-probability guarantees for exact label recovery which depend on structural properties of the graph. On the other hand, the assumed generative process (lines 89-101, proposed in Globerson et al., 2015) is somewhat toyish which might make the results less interesting. Therefore, I am inclined towards acceptance but not strongly. Comments: - I feel like the presentation can be greatly improved by including an overview of the main result at the beginning of Section 3. In particular, you can state the main result, which is actually given in Remark 2 (!), and then provide some high-level intuition on the path to prove it.
Reviews: Exact inference in structured prediction
The paper gives a theoretical analysis of Markov random fields. The authors answer the question of when exact inference can be done exactly in a polynomial time. This is a generalization of a result of in Globerson et al. (2015) from grid graphs to general connected graphs, which is on my opinion, a non-trivial generalization. The paper is self contained and readable for the Machine Learning community, although quite technical. Indeed, I consider that it is a theoretical paper that has all the quality for a NeurIPS acceptance.
Review for NeurIPS paper: Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
While it is interesting that self-supervised methods are more invariant to occlusion, it is unclear why they wouldn't also be more invariant to the other augmentations used during training. For example, supervised learning appears more invariant to "Illumination Color" (Top-25 category) despite self-supervised learning methods using aggressive color augmentation techniques. This discrepancy is not discussed and we are left wondering what it means. Next, while the analysis of transfer performance as a function of cropped vs. original training and test datasets is interesting, it is unclear whether the results really support the authors' interpretation. They find that training and testing on the same type of images (i.e. This is to be expected, as this minimizes the domain gap between training and testing.
Review for NeurIPS paper: Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
The topic of the paper is very relevant to the NeurIPS community, given the increased interest in understanding self-supervised learning. Reviewers have appreciated the direction the paper takes for this, ie, to study invariances learned by self-supervised learning methods, comparing them with supervised representations. There were some concerns about the interpretations of the emprical results which have been addressed in the author response. This paper takes the first and important step towards understanding the invariances in self-supervised representations and their implications on downstream tasks, and would be of interest to the NeurIPS community.
Review for NeurIPS paper: Adversarial Self-Supervised Contrastive Learning
Weaknesses: I have several major concerns on the presentations of this paper: (1) The proposed transformation smoothed inference may cause gradient obfuscation, therefore Expectation of Transformation [1] should be used to properly attack this model. Also, the details of transformation smoothed inference are missing, e.g., what transformations are used? Nonetheless, I am pretty confused about the discussion there. First of all, I am pretty confused about what comparisons are conducted there? The only useful information I found is "Compared to the semi-supervised learning methods, RoCL takes about 1/4 times faster with the same computation resources", but how about comparisons on other metrics, e.g., robustness, accuracy?