Skeleton: Visual Authoring of Non-visual Data Experiences

Authors

Frank Elavsky (Carnegie Mellon University), Chieri J Nnadozie (Carnegie Mellon University), Lucas Nadolskis (University of California, Santa Barbara), Patrick Carrington (Carnegie Mellon University), Dominik Moritz (Carnegie Mellon University)

Presentation

Session

How can I do this myself?

Time

Thursday, Nov 12, 08:24 – 08:36 (US/Eastern) · session 08:00 – 09:30

Location

Hall America center

Keywords

visualization, accessibility, non-visual design

Abstract

When sighted practitioners author accessible data visualizations, they build navigation structures (the nodes, edges, and input bindings that govern how assistive technologies traverse an interface) entirely in code, with no visual representation. This invisibility makes navigation structures difficult to inspect, debug, and iterate on. To sighted practitioners, every other aspect of a visualization is iterated on because it is visible; navigation structure ships as a first draft, if at all, because it is not. Without a representation to react to, practitioners cannot develop judgment about what makes navigation good or bad, and the quality ceiling of non-visual experiences is set by the absence of a feedback loop. We address this problem through longitudinal co-design with practitioners across cartography, design systems, and open-source visualization, and make three contributions. First, we introduce technical advancements for making the properties of accessible navigation structure visible and directly manipulable during authoring, grounded in two foundational pieces of infrastructure produced by our co-design work: an Inspector that renders navigation graphs as interactive node-link diagrams, and a Dimensions API that expresses navigation in terms of data dimensions rather than explicit graph construction. Second, building on these, we present Skeleton, a direct-manipulation authoring environment in which the properties of an accessible navigation structure are translated into visual representations authors can observe and manipulate. Key techniques include a dual-view editor that simultaneously shows the system’s navigation model and the end user’s spatial experience, a scaffolding engine that automates spatial node placement by repurposing a visualization rendering pipeline, a live label-template editor with real-time screen-reader-output preview, and a testing mode that makes traversal sequence visually trackable. Third, we evaluate Skeleton through an in-situ study with 8 practitioners across visualization design, engineering, and research. Making navigation structure visible changed how practitioners engaged with accessible design: they reconsidered the architecture of their own visualizations, attended to a broader range of input modalities, and shifted from treating accessibility as a compliance task to treating it as a design problem.

For Practitioners

Interested practitioners: Accessibility practitioners (outside of the data community and within it): especially those who build tools, develop, or design. Any of these visualization practitioners: those interested in accessibility, library-builders, application builders, product teams, software engineers, practitioners in a federal or governmental context, practitioners who are required to make their work accessible, graphic and interaction designers, data journalists, scientific communicators, or anyone whose visualizations or visualization tools are intended to be used by the general public or people with disabilities. How can practitioners apply what they learn from this paper? Well, they could download our tooling (data navigator and our dimensions API, the inspector, our bokeh wrapper, and more) and use it directly, just like our Adobe, Bokeh, and University of Wisconsin GIS collaborations did. Practitioners could also directly use our flagship tool Skeleton to develop new interaction and navigation patterns for individual visualizations or libraries of chart types or grammars. Additionally, practitioners could even generalize our approach and build visualization tools for other domains of accessibility work: such as visualizing screen reader navigation in PDF or web development tooling, or in video games or other complex interaction contexts. We would love to see our approach, where we visualized non-visual experiences, take hold in many other areas of interaction design and development.