ClusterNet : Semi-Supervised Clustering using Neural Networks

arXiv.org Machine Learning

Clustering using neural networks has recently demon- strated promising performance in machine learning and computer vision applications. However, the performance of current approaches is limited either by unsupervised learn- ing or their dependence on large set of labeled data sam- ples. In this paper, we propose ClusterNet that uses pair- wise semantic constraints from very few labeled data sam- ples (< 5% of total data) and exploits the abundant un- labeled data to drive the clustering approach. We define a new loss function that uses pairwise semantic similarity between objects combined with constrained k-means clus- tering to efficiently utilize both labeled and unlabeled data in the same framework. The proposed network uses con- volution autoencoder to learn a latent representation that groups data into k specified clusters, while also learning the cluster centers simultaneously. We evaluate and com- pare the performance of ClusterNet on several datasets and state of the art deep clustering approaches.


Multi-class Classification without Multi-class Labels

arXiv.org Machine Learning

This work presents a new strategy for multi-class classification that requires no class-specific labels, but instead leverages pairwise similarity between examples, which is a weaker form of annotation. The proposed method, meta classification learning, optimizes a binary classifier for pairwise similarity prediction and through this process learns a multi-class classifier as a submodule. We formulate this approach, present a probabilistic graphical model for it, and derive a surprisingly simple loss function that can be used to learn neural network-based models. We then demonstrate that this same framework generalizes to the supervised, unsupervised cross-task, and semi-supervised settings. Our method is evaluated against state of the art in all three learning paradigms and shows a superior or comparable accuracy, providing evidence that learning multi-class classification without multi-class labels is a viable learning option.


A probabilistic constrained clustering for transfer learning and image category discovery

arXiv.org Artificial Intelligence

Neural network-based clustering has recently gained popularity, and in particular a constrained clustering formulation has been proposed to perform transfer learning and image category discovery using deep learning. The core idea is to formulate a clustering objective with pairwise constraints that can be used to train a deep clustering network; therefore the cluster assignments and their underlying feature representations are jointly optimized end-to-end. In this work, we provide a novel clustering formulation to address scalability issues of previous work in terms of optimizing deeper networks and larger amounts of categories. The proposed objective directly minimizes the negative log-likelihood of cluster assignment with respect to the pairwise constraints, has no hyper-parameters, and demonstrates improved scalability and performance on both supervised learning and unsupervised transfer learning.


Learning Neural Models for End-to-End Clustering

arXiv.org Machine Learning

We propose a novel end-to-end neural network architecture that, once trained, directly outputs a probabilistic clustering of a batch of input examples in one pass. It estimates a distribution over the number of clusters $k$, and for each $1 \leq k \leq k_\mathrm{max}$, a distribution over the individual cluster assignment for each data point. The network is trained in advance in a supervised fashion on separate data to learn grouping by any perceptual similarity criterion based on pairwise labels (same/different group). It can then be applied to different data containing different groups. We demonstrate promising performance on high-dimensional data like images (COIL-100) and speech (TIMIT). We call this ``learning to cluster'' and show its conceptual difference to deep metric learning, semi-supervise clustering and other related approaches while having the advantage of performing learnable clustering fully end-to-end.


Co-Representation Learning For Classification and Novel Class Detection via Deep Networks

arXiv.org Machine Learning

Deep Neural Network (DNN) has been largely demonstrated to be effective for closed-world classification problems where the total number of classes are known in advance. However, when the total number of classes that may occur during test time is unknown, DNNs notorious fail, i.e., DNN will make incorrect label prediction on instances from novel or unseen classes. This severely limits its utility in many real-world web applications, particularly when data occurs as a continuous stream. In this paper, we focus on addressing this key challenge by developing a two-channel DNN based co-representation learning framework that not only predicts instances from known classes, but also detects and adapts to the occurrence of novel class instances over time. Concretely, we propose a metric learning method using pairwise-constraint loss (PCL) function to learn a feature representation where intra-class compactness and inter-class separation is achieved. Moreover, we apply the temperature scaling scheme on the softmax function to replace traditional softmax output and design an open-world classifier. Our extensive empirical evaluation on benchmark datasets demonstrates the effectiveness of our framework compared to other competing techniques.