A Review, Framework and R toolkit for Exploring, Evaluating, and Comparing Visualizations

High dimensional data can contain a large amount of noise and importantly for visualization, the human brain can only comprehend three dimensions. Thus, there is a need to reduce data into an interpretable format by converting high dimensional data into two or three dimensions, which can subsequently be visualized using a two or three dimensional scatterplot. To meet the need for dimensionality reduction methods, a plethora of algorithms and associated fitting methods have been developed. A researcher wishing to perform dimensionality reduction for visualization will be presented with a choice of hundreds of algorithms. Which algorithm should be used? This paper describes a visualization framework called QVisVis and associated software tools implemented in R to help choose dimensionality reduction methods, tune these methods, and visually evaluate the quality of dimensionality reduction solutions. The major contributions of these paper are to review and synthesize previous work on evaluating and "visualizing" performance metrics, create an overall visualization framework for "visualizing" visualization quality, and implement the framework in an R toolkit.