SIGMA: A Spatial Interaction Grammar for Multi-Display Visual Analytics

Authors

Yogesh Rai (UAH), Haeyong Chung (University of Alabama in Huntsville)

Presentation

Session

Places and spaces

Time

Thursday, Nov 12, 14:12 – 14:24 (US/Eastern) · session 13:00 – 14:30

Location

Hall America center

Keywords

Spatial interaction, cross-device interaction, declarative grammar, multi-display environments, visual analytics

Abstract

In this paper, we present SIGMA, a declarative spatial interaction grammar for multiple displays and devices. SIGMA specifies (1) display entities in a display ecology, (2) spatial predicates that capture relevant spatial conditions among displays and devices, and (3) mapping rules that translate satisfied predicates into interaction semantics and corresponding visualization updates. By providing a small set of reusable primitives that can be logically composed into richer spatial conditions, SIGMA enables concise specifications over a wide range of spatial relationships and cross-display interactions. The grammar is informed by a literature review of 34 multi-display papers that met our inclusion criteria, from which we extracted 96 spatial interaction techniques to iteratively refine it. To enable declarative SIGMA specifications to work in a real multi-display system, we developed a runtime engine that interprets them by continuously processing 6DoF pose data, evaluating predicates, and triggering the corresponding visualization updates across displays. We further present three use cases showing how SIGMA supports diverse display configurations and categories of spatial interaction in real-world multi-display visual analytics scenarios.

For Practitioners

Designers and developers of multi-display user interfaces and spatial computing systems.