Using competency questions to select optimal clustering structures for residential energy consumption patterns

During cluster analysis domain experts and visual analysis are frequently relied on to identify the optimal clustering structure. This process tends to be adhoc, subjective and difficult to reproduce. This work shows how competency questions can be used to formalise expert knowledge and application requirements for context specific evaluation of a clustering application in the residential energy consumption sector. While cluster analysis is an established unsupervised machine learning technique, identifying the optimal set of clusters for a specific application requires extensive experimentation and domain knowledge. Cluster compactness and distinctness are two important attributes that characterise a good cluster set (Sarle et al., 1990) and different metrics, such as the Mean Index Adequacy (MIA), Davies-Bouldin Index (DBI) and the Silhouette Index have been proposed to measure cluster compactness and distinctness.