This paper makes a contribution toward the theory of semi-supervised learning for graph classification, as well as an efficient algorithm for computing the proposed classifier. This is an interesting problem and the reviewers agree the contribution is at least incremental. I suggest the authors carefully revise the paper to address reviewer concerns to get the maximum impact.

It seems trivial to extend the Triplet Mutual Information [1] and its code [2]. The contribution of the proposed method is not clear. Please explain the difference between your work and [1] about Triplet Mutual Information. For the comparison, how were the parameters of other methods tuned? 4. Deformable template matching is an existing technology. Please explain the difference between your work and [3, 4] separately.

The reviewers found the usage of the deformable templates here interesting and this work will be useful to a large part of the NeurIPS community.

Weaknesses: - I expected to see the linear evaluation performance on ImageNet can be impressive. However, it's a pity to see this transfer learning's performance is poor with only TC-S: 20.9% at best. This seriously limits the impact of this work. If the model can only perform well on some easy datasets that are close to the SAYCam, we cannot get too much benefits from learning on such datasets, especially with access to so many big datasets. Maybe the authors can change the SAYCam to other standard videos (Charades) and see if they can have good transfer learning performances.

This is an interesting paper combining machine learning and psychology, and brings interesting insights about what can be learned from naturalistic, egocentric, real-world datasets, and how the learned representations can be used on downstream tasks. It's well-written and clearly presented, and likely of interest to the general NeurIPS audience. Reviewers 3 and 4 initially had concerns about the model's performance, and suggestions about using other datasets. After discussion and the rebuttal, they were able to be convinced that these run counter to the main motivation of the study. They subsequently raised their scores and all reviewers - and myself - are in agreement to accept.

Weaknesses: - Despite the extensive empirical evaluations, the three multimodal variants as proposed by the paper are direct extensions of the DeepCluster algorithm [4]. The main contributions appear to be (1) a working pipeline which demonstrates that variants of DeepCluster works with video and audio encoders; (2) scaling up the training to extremely large datasets. While both contributions are interesting, they appear to me to be less relevant to the audience of NeurIPS. It would also be great if such conjectures are accompanied with empirical evaluations on more diverse tasks than the three classification datasets. That would help the audience understand when to apply the XDC variant of DeepCluster (e.g. is it specific to audio and visual in videos, or is it more general?),

The reviewers generally agree this paper has great execution, a great idea, and great results. The reviewers noted the impact that self-supervised learning on video can have, which has been less explored than the image counterpart. The reviewers also praised the strong empirical results, which will be of high interest to the community.

Post-feedback update: Thanks for your update. Your additional explanations and results will help improve the paper, and I definitely think this work is strong and should be accepted. The framework itself is new, and the authors make it very clear how prior work fits into the framework as special cases. At the same time, a good case is made for why this framework is useful to have and how it can be better to use than prior losses. Quality: this paper makes a compelling case for the framework it introduces.

All reviewers agreed that this paper make a nice contribution to NeurIPS by providing a novel general framework for generating calibrated surrogate loss functions for structured prediction problems. On the other hand, in discussion, they also stressed that including some baselines (e.g., SSVM/CRF approximation/SPEN) in the experiments and reporting runtimes could make this paper much stronger. The authors should implement their promised changes in the camera-ready version.

The paper presents three GNN architectural guidelines for combating this, which can lead to improved predictions, particularly on networks exhibiting heterophilous structure (i.e., non-homophilous labels). The design choices are motivated theoretically and intuitively and then combined into a single model that can provide better predictions on networks with heterophilous structure, as demonstrated by synthetic and real-world data experiments. The paper provides a number of interesting insights into why certain GNN architectural choices can help predictions in the case of low network homophily. Although not mentioned in their paper, a similar idea to higher-order neighborhoods (Section 3.1.2) I believe that these ideas provide further motivation for the design choices appearing in this paper and including them will strengthen some of the intuition.