Does visualization help AI understand data?

Victoria R. Li * Harvard University Johnathan L. Sun * Harvard University Martin Wattenberg Harvard University, Google ResearchFigure 1: We test two prominent vision-language models, GPT 4.1 and Claude 3.5, on three settings motivated by common data analysis tasks. In each setting, we ask models to describe datasets under five conditions, providing: (1) numerical data alone, (2,3) numerical data with incorrect or blank visuals as baselines, (4) numerical data with correct visuals, and (5) correct visuals alone. After 12,000 trials, we find consistent performance gains when models are provided with accurate visualizations. A BSTRACT Charts and graphs help people analyze data, but can they also be useful to AI systems? To investigate this question, we perform a series of experiments with two commercial vision-language models: GPT 4.1 and Claude 3.5. Across three representative analysis tasks, the two systems describe synthetic datasets more precisely and accurately when raw data is accompanied by a scatterplot, especially as datasets grow in complexity. Comparison with two baselines-- providing a blank chart and a chart with mismatched data--shows that the improved performance is due to the content of the charts.