This paper explores useful modifications of the recent development in contrastive learning via novel probabilistic modeling. We derive a particular form of contrastive loss named Joint Contrastive Learning (JCL). JCL implicitly involves the simultaneous learning of an infinite number of query-key pairs, which poses tighter constraints when searching for invariant features. We derive an upper bound on this formulation that allows analytical solutions in an end-to-end training manner. While JCL is practically effective in numerous computer vision applications, we also theoretically unveil the certain mechanisms that govern the behavior of JCL. We demonstrate that the proposed formulation harbors an innate agency that strongly favors similarity within each instance-specific class, and therefore remains advantageous when searching for discriminative features among distinct instances. We evaluate these proposals on multiple benchmarks, demonstrating considerable improvements over existing algorithms.

W e use notation Eq.(M.xx) to refer to the equation Eq.(xx) presented in the main paper, and use Eq.(S.xx) Similarly, we use Fig./Section/T able (M.xx) JCL more efficiently exploit these samples. Qi Cai and Y u Wang contributed equally to this work. This work was performed at JD AI Research. Eq.(M.8), we update the queue The ablation experiments are conducted on a subset of ImageNet1K (i.e., ImageNet100) following For the experiments that visualize the distributions of similarities and variances in Section (M.4.5),

We derive a particular form of contrastive loss named Joint Contrastive Learning (JCL).

Additional Feedback: I think it is too strong to claim that "we also theoretically unveil the certain important mechanisms that govern the behavior of JCL." The main theoretical tool in the proposed method is an application of Jensen's inequality. There is also a section (3.3) that discusses some very basic properties of the the objective. To claim any of this as a significant "theoretical contribution" is too strong in my view. To me, the most interesting aspect of Fig2 is part (b).

This paper achieved a high accept consensus. The paper puts forward a simple core idea, shows it being helpful, and gives analysis/insights that justify how it works. The method beats SOTA in various tasks. However, a bad quality of language and some experimental details missing were reported and I encourage the authors to fix these following the reviewer's recommendations for the final version of the manuscript.

This paper explores useful modifications of the recent development in contrastive learning via novel probabilistic modeling. We derive a particular form of contrastive loss named Joint Contrastive Learning (JCL). JCL implicitly involves the simultaneous learning of an infinite number of query-key pairs, which poses tighter constraints when searching for invariant features. We derive an upper bound on this formulation that allows analytical solutions in an end-to-end training manner. While JCL is practically effective in numerous computer vision applications, we also theoretically unveil the certain mechanisms that govern the behavior of JCL.

Few-shot named entity recognition (NER) aims at identifying named entities based on only few labeled instances. Most existing prototype-based sequence labeling models tend to memorize entity mentions which would be easily confused by close prototypes. In this paper, we proposed a Prototypical Semantic Decoupling method via joint Contrastive learning (PSDC) for few-shot NER. Specifically, we decouple class-specific prototypes and contextual semantic prototypes by two masking strategies to lead the model to focus on two different semantic information for inference. Besides, we further introduce joint contrastive learning objectives to better integrate two kinds of decoupling information and prevent semantic collapse. Experimental results on two few-shot NER benchmarks demonstrate that PSDC consistently outperforms the previous SOTA methods in terms of overall performance. Extensive analysis further validates the effectiveness and generalization of PSDC.