An approach to improve network interpretability is via clusterability, i.e., splitting a model into disjoint clusters that can be studied independently. We find pretrained models to be highly unclusterable and thus train models to be more modular using an ``enmeshment loss'' function that encourages the formation of non-interacting clusters. Using automated interpretability measures, we show that our method finds clusters that learn different, disjoint, and smaller circuits for CIFAR-10 labels. Our approach provides a promising direction for making neural networks easier to interpret.

Although numerous algorithms have been proposed to solve the categorical data clustering problem, how to access the statistical significance of a set of categorical clusters remains unaddressed. To fulfill this void, we employ the likelihood ratio test to derive a test statistic that can serve as a significance-based objective function in categorical data clustering. Consequently, a new clustering algorithm is proposed in which the significance-based objective function is optimized via a Monte Carlo search procedure. As a by-product, we can further calculate an empirical $p$-value to assess the statistical significance of a set of clusters and develop an improved gap statistic for estimating the cluster number. Extensive experimental studies suggest that our method is able to achieve comparable performance to state-of-the-art categorical data clustering algorithms. Moreover, the effectiveness of such a significance-based formulation on statistical cluster validation and cluster number estimation is demonstrated through comprehensive empirical results.

Exclusive clustering: As the name suggests, exclusive clustering specifies that a data point or object can exist only in one cluster. Hierarchical clustering: Hierarchical tries to create a hierarchy of clusters. There are two types of hierarchical clustering: agglomerative and divisive. Agglomerative follows the bottom-up approach, initially treats each data point as an individual cluster, and the pairs of clusters are merged as they move up the hierarchy. Divisive is the very opposite of agglomerative.

In machine learning, unsupervised learning is a machine learning model that infers the data pattern without any guidance or label. Many models are included in the unsupervised learning family, but one of my favorite models is Agglomerative Clustering. Agglomerative Clustering or bottom-up clustering essentially started from an individual cluster (each data point is considered as an individual cluster, also called leaf), then every cluster calculates their distance with each other. The two clusters with the shortest distance with each other would merge creating what we called node. Newly formed clusters once again calculating the member of their cluster distance with another cluster outside of their cluster.

For example, in unsupervised machine learning, to evaluate theperformance of a clustering method, researchers typically assess agreement between a reference standard partition that purports to represent the true cluster structure of the objects (golden standard), and a trial partition produced by the method that is being evaluated (Wallace 1983; Halkidi, Batiskis and Vazirgiannis 2002; Jain 2010). High agreement between the two partitions may indicate good recovery of the true cluster structure. Agreement between partitions can be assessed with so-called external validity indices (Albatineh, Niewiadomska-Bugaj and Mihalko 2006; Brun et al. 2007; Warrens 2008a,2008b; Pfitzner et al. 2009). External validity indices can be roughly categorized into three approaches, namely 1) counting object pairs, 2) information theory (Vinh, Epps and Bailey 2010; Lei et al. 2016), and 3) matching sets (Rezaei and Fränti 2016). Most external validity indices are of the pair-counting approach, which is based on counting pairs of objects placed in identical and different clusters.