Harnessing LLMs Without Surrendering Control: Delegation Boundaries in Visual Data Storytelling Authoring
Authors
Zhuojun Jiang (Arizona State University), Yuki Ueno (Arizona State University), Chris Bryan (Arizona State University)
Presentation
- Session
- Me, Myself, and AI
- Time
- Wednesday, Nov 11, 13:18 – 13:27 (US/Eastern) · session 13:00 – 14:30
- Location
- Hall America center
Keywords
Visual data storytelling, LLMs, qualitative study
Abstract
Despite the emergence of large language models (LLMs) for visual data storytelling workflows, there are open questions about how authors decide what activities or tasks to entrust to them and what should be "protected" or maintained under human control. To investigate this, we interviewed a cohort of 12 expert visual data storytellers. Our analysis shows that participants rarely treated LLMs as autonomous storytellers. Instead, they tend to selectively delegate execution-oriented tasks to LLMs while retaining control over activities that shape narrative intent and story meaning. Our findings show that LLM assistance is most productive after human seeding and constraint-setting, and that it shifts labor from production to verification. We discuss design implications for boundary-aware authoring tools, data-grounded generation, low-fidelity ideation, and reporting practices for LLM-based visualization research. Supplemental materials for this paper are available at https://osf.io/hcnp6.
For Practitioners
This paper will be of interest to visual data storytellers, data journalists, visualization designers, researchers, and developers of AI-assisted authoring tools. Practitioners can use the findings to decide which storytelling tasks to delegate to LLMs and which to keep under human control, especially around narrative intent, data claims, visual judgment, and final accountability. The paper also suggests practical strategies for using LLMs through clear constraints, verification, and curation rather than full automation.