PaintEcho: Bringing Paintings to Photos through Cross-domain Visual Exploration
Authors
Yihan Gao (Zhejiang University), Tan Tang (Zhejiang University), Jianing Yin (Zhejiang University), Haobo Zheng (Zhejiang University), Tianyi Chen (Laboratory of Art and Archaeology Image, Zhejiang University), Lu Ying (Zhejiang University), Yanhong Wu (Hithink RoyalFlush Information Network Co.), Yingcai Wu (Zhejiang University)
Presentation
- Session
- Data really is everywhere
- Time
- Tuesday, Nov 10, 15:12 – 15:24 (US/Eastern) · session 15:00 – 16:30
- Location
- Hall Essex north
Keywords
Visual analytics, cross-domain image association, intent-driven analytics, digital humanities, art history
Abstract
Exploring associations between paintings and photographs has become a vital component of art history research, as it helps reveal the evolution of visual patterns and drive interdisciplinary discovery. However, establishing these cross-domain associations remains difficult due to the stylistic gap between artistic expression and photographic realism, coupled with the multifaceted nature of visual attributes. Existing automated approaches often lack accuracy and interpretability, while their rigid outputs fail to accommodate the flexible, multi-dimensional exploration required by domain experts. Addressing these limitations poses three primary challenges: (a) the mismatch between unified representations and diverse analytical needs, (b) the difficulty of identifying relevant species from large cross-domain candidates, and (c) the synthesis of large-scale artistic insights from fragmented matches. To address these challenges, we first introduce a cross-domain visual association model that disentangles multifaceted attributes, enabling intent-driven feature steering. Building on this, we develop PaintEcho, a visual analytics system that enables multi-level exploration to facilitate the systematic discovery and interpretive analysis of cross-domain associations. Specifically, it employs a query-centric radial layout for local taxonomic organization and a "mountain-reflection" metaphor for global pattern discovery. We validate the effectiveness and usability of our approach through an algorithmic performance analysis, two use cases, and a task-based user study. Notably, PaintEcho enables experts to identify elusive species and uncover systematic stylistic shifts across historical masterpieces.
For Practitioners
Art historians and digital humanities practitioners would be the primary audience for this work. They could use the proposed approach to associate objects in paintings with real-world photographs, identify plausible species or visual references, and trace how recurring motifs and visual styles evolve across periods and artists. Visual analytics and computer vision practitioners may also benefit from the cross-domain visual association model and multi-level visual exploration designs, and adapt these techniques to other cross-domain image collections where interpretable comparison and pattern discovery are needed.