IEEE VIS 2024 Content: Testing the Test: Observations When Assessing Visualization Literacy of Domain Experts

Testing the Test: Observations When Assessing Visualization Literacy of Domain Experts

Seyda Öney - University of Stuttgart, Stuttgart, Germany

Moataz Abdelaal - University of Stuttgart, Stuttgart, Germany

Kuno Kurzhals - University of Stuttgart, Stuttgart, Germany

Paul Betz - University of Stuttgart, Stuttgart, Germany

Cordula Kropp - University of Stuttgart, Stuttgart, Germany

Daniel Weiskopf - University of Stuttgart, Stuttgart, Germany

Download preprint PDF

Room: Bayshore I

2024-10-14T16:00:00ZGMT-0600Change your timezone on the schedule page
2024-10-14T16:00:00Z
Exemplar figure, described by caption below
Domain experts may be asked to take the Mini-VLAT test to assess their visualization skills. However, factors such as the time limit on each question could cause stress, potentially affecting their performance.
Full Video
Abstract

Various standardized tests exist that assess individuals' visualization literacy. Their use can help to draw conclusions from studies. However, it is not taken into account that the test itself can create a pressure situation where participants might fear being exposed and assessed negatively. This is especially problematic when testing domain experts in design studies. We conducted interviews with experts from different domains performing the Mini-VLAT test for visualization literacy to identify potential problems. Our participants reported that the time limit per question, ambiguities in the questions and visualizations, and missing steps in the test procedure mainly had an impact on their performance and content. We discuss possible changes to the test design to address these issues and how such assessment methods could be integrated into existing evaluation procedures.