Beyond the Post Hoc User Study: Modeling Visual Decision-Making with Active Inference
Authors
Harrison J Goldwyn (National Laboratory of the Rockies), Graham Johnson (National Laboratory of the Rockies), Christopher Ibarra (National Laboratory of the Rockies), Lace M. Padilla (Northeastern University), Kenny Gruchalla (National Laboratory of the Rockies)
Presentation
- Session
- Figuring out how to do good research
- Time
- Tuesday, Nov 10, 15:48 – 16:00 (US/Eastern) · session 15:00 – 16:30
- Location
- Hall America center
Links
Sign in to access the preprint PDF.
Sign in- Download Supplemental Material
Keywords
Perception and Cognition, Visualization Design and Evaluation, Cognitive Modeling, Active Inference, Decision Making
Abstract
The evaluation of visual encodings is grounded in empirical user studies, including controlled comparisons of alternative designs. These studies are essential for measuring whether a visualization supports accurate human judgment, and many can provide evidence about underlying perceptual and cognitive mechanisms. However, empirical evidence alone does not enable causal prediction of interpretation errors. In this sense, evaluations are often post hoc: they assess visualization efficacy after a design has been specified, rather than predicting how human attention, uncertainty, memory, and bias may lead a viewer to accurate or erroneous interpretation. This mechanistic gap restricts the field’s ability to accumulate predictive design knowledge. Even with evidence-supported cognitive frameworks for visual decision-making, we lack general means to simulate user behavior and predict errors in silico. To bridge this gap, we demonstrate a translation of a cognitive theory of visualization interpretation into executable simulation using Active Inference: a probabilistic process theory of living systems often applied to human perception, learning, and action-taking. Active Inference agents iteratively minimize the probability of surprising observations by updating internal belief states and choosing informative actions. Simulating human interpretation of data visualization in this context, we frame chart reading as a dynamic visual search, minimizing both uncertainty and cognitive effort to reach task completion. As a foundational proof-of-concept, we engineer Active Inference agents that perform a bar-chart average-estimation task under a dual-process theory of decision-making. Crucially, our architecture aims to replicate human perceptual vulnerabilities by presenting a Fast, heuristic (Type 1) agent prone to tick salience bias and a Slow, analytic (Type 2) agent more prone to working-memory decay. Both agents yield inspectable cognitive traces, including the evolution of belief uncertainty and the chosen visual fixation sequences. By distilling these hypothesized failure mechanisms into interpretable parameters, we present this architecture as a framework for formalizing hypotheses about mechanisms of visualization interpretation. These models highlight a new role for empirical studies in visualization design: to parameterize, test, refine, or falsify decision-making simulations. Fitting these models to user data could support a shift from primarily post hoc testing toward predictive in silico validation, allowing researchers to anticipate efficacy earlier in the design process.
For Practitioners
We expect our work to be of interest to data visualization scientists at the intersection of visualization efficacy and mechanistic study of human cognition.