IEEE VIS 2024 Content: Investigating the Apple Vision Pro Spatial Computing Platform for GPU-Based Volume Visualization

Investigating the Apple Vision Pro Spatial Computing Platform for GPU-Based Volume Visualization



Camilla Hrycak - University of Duisburg-Essen, Duisburg, Germany

 David Lewakis - University of Duisburg-Essen, Duisburg, Germany

 Jens Harald Krueger - University of Duisburg-Essen, Duisburg, Germany

 Room: Bayshore VI

2024-10-16T16:18:00ZGMT-0600Change your timezone on the schedule page
2024-10-16T16:18:00Z

Exemplar figure, described by caption below — Screenshots of our testbed direct volume rendering application on the Apple Vision Pro. From Top: Slice-based volume rendering in a shared space with video see-through, f Bottom: Rendering the dataset in a fully immersive space. Notice varying image quality across the figures due to active foveation.

Fast forward

Full Video

Keywords

Apple Vision Pro, Volume Rendering, Virtual Reality, Augmented Reality

Abstract

In this paper, we analyze the Apple Vision Pro hardware and the visionOS software platform, assessing their capabilities for volume rendering of structured grids---a prevalent technique across various applications. The Apple Vision Pro supports multiple display modes, from classical augmented reality (AR) using video see-through technology to immersive virtual reality (VR) environments that exclusively render virtual objects. These modes utilize different APIs and exhibit distinct capabilities. Our focus is on direct volume rendering, selected for its implementation challenges due to the native graphics APIs being predominantly oriented towards surface shading. Volume rendering is particularly vital in fields where AR and VR visualizations offer substantial benefits, such as in medicine and manufacturing. Despite its initial high cost, we anticipate that the Vision Pro will become more accessible and affordable over time, following Apple's track record of market expansion. As these devices become more prevalent, understanding how to effectively program and utilize them becomes increasingly important, offering significant opportunities for innovation and practical applications in various sectors.