VACP: Visual Analytics Context Protocol
Authors
Tobias Stähle (ETH Zürich), Péter Ferenc Gyarmati (ETH Zurich), Thilo Spinner (ETH Zurich), Rita Sevastjanova (ETH Zurich), Dominik Moritz (Carnegie Mellon University), Mennatallah El-Assady (ETH Zürich)
Presentation
- Session
- Let's dig into the data (from France)
- Time
- Wednesday, Nov 11, 09:00 – 09:12 (US/Eastern) · session 08:00 – 09:30
- Location
- Hall America north
Links
Sign in to access the preprint PDF.
Sign in- Download Supplemental Material
Keywords
Visual Analytics, Intelligent Agents, AI Agents, Context Protocol, Agent-Ready Interfaces
Abstract
The rise of AI agents introduces a fundamental shift in Visual Analytics (VA), in which agents act as a new user group. Current agentic approaches—based on computer vision and raw DOM access—fail to perform VA tasks accurately and efficiently. This paper introduces the Visual Analytics Context Protocol (VACP), a framework designed to make VA applications "agent-ready" that extends generic protocols by explicitly exposing application state, available interactions, and mechanisms for direct execution. To support our context protocol, we contribute a formal specification of AI agent requirements and knowledge representations in VA interfaces. We instantiate VACP as a library compatible with major visualization grammars and web frameworks, enabling augmentation of existing systems and the development of new ones. Our evaluation across a compelling set of VA tasks demonstrates that VACP-enabled agents achieve higher success rates in interface interpretation and execution compared to current agentic approaches, while reducing token consumption and completion time. VACP closes the gap between human-centric VA interfaces and machine perceivability, ensuring agents can reliably act as collaborative users in VA systems. More information under https://vacp.dev.
For Practitioners
This paper is primarily of interest to Visual Analytics (VA) developers, AI/ML engineers, data scientists, and accessibility specialists. Practitioners can apply the Visual Analytics Context Protocol (VACP) framework and its open-source library to make both new and legacy VA applications accessible to autonomous AI agents. Specifically, developers can use VACP to expose structural logic and interaction options semantically, enabling automated analytical testing and the creation of mixed-initiative collaborative workflows between humans and specialized AI agents. Furthermore, AI engineers can leverage this structured approach to bypass error-prone visual parsing, thereby drastically reducing token consumption, minimizing execution time, and improving the success rate of AI agents operating within complex, data-driven environments.