Feature Extraction Framework based on Contrastive Learning with Adaptive Positive and Negative Samples

Currently, high-dimensional data is widely used in pattern recognition and data mining, which leads to high storage overhead, heavy computation, and excessive time consumption apart from causing the problem known as "curse of dimensionality". A significant way to address these issues is feature extraction, which transforms the original highdimensional spatial data into a low-dimensional subspace by a projection matrix. Although, the effect of feature extraction is often worse than it in deep learning, it has always been a research hotspot because of its strong interpretability and particularly well on any type of hardware (CPU, GPU, DSP). Therefore, it is an urgent need in traditional feature extraction to better extract discriminative features for downstream tasks. In the field of deep learning, contrastive learning has attracted extensive scholarly attention as the primary method of self-supervised learning. Contrastive learning uses information of data to supervise itself by constructing positive and negative samples, which strives to learn more discriminative features. InfoNCE loss based on contrastive learning is proposed in contrastive predictive coding (CPC)van den Oord et al. [2018]. CPC proves that minimizing the InfoNCE loss maximizes a lower bound on mutual information, which provides theoretical support for its advantages in extracting more discriminative features. Consequently, a large number of studies based on contrastive learning are proposed.