IEEE VIS 2024 Content: Enhancing Data Literacy On-demand: LLMs as Guides for Novices in Chart Interpretation

Enhancing Data Literacy On-demand: LLMs as Guides for Novices in Chart Interpretation

Kiroong Choe -

Chaerin Lee -

Soohyun Lee -

Jiwon Song -

Aeri Cho -

Nam Wook Kim -

Jinwook Seo -

Screen-reader Accessible PDF

Room: Bayshore I + II + III

2024-10-18T12:30:00Z GMT-0600 Change your timezone on the schedule page
2024-10-18T12:30:00Z
Exemplar figure, described by caption below
Our system allows users to interact with charts using both text and visual inputs. Users can ask questions or share visualizations, and the system will provide the current chart annotations to the LLM agent. The agent can then propose new annotations and suggest follow-up questions for deeper analysis.
Fast forward
Keywords

Visualization literacy, Large language model, Visual communication

Abstract

With the growing complexity and volume of data, visualizations have become more intricate, often requiring advanced techniques to convey insights. These complex charts are prevalent in everyday life, and individuals who lack knowledge in data visualization may find them challenging to understand. This paper investigates using Large Language Models (LLMs) to help users with low data literacy understand complex visualizations. While previous studies focus on text interactions with users, we noticed that visual cues are also critical for interpreting charts. We introduce an LLM application that supports both text and visual interaction for guiding chart interpretation. Our study with 26 participants revealed that the in-situ support effectively assisted users in interpreting charts and enhanced learning by addressing specific chart-related questions and encouraging further exploration. Visual communication allowed participants to convey their interests straightforwardly, eliminating the need for textual descriptions. However, the LLM assistance led users to engage less with the system, resulting in fewer insights from the visualizations. This suggests that users, particularly those with lower data literacy and motivation, may have over-relied on the LLM agent. We discuss opportunities for deploying LLMs to enhance visualization literacy while emphasizing the need for a balanced approach.