Opening the Black Box of 3D Reconstruction Error Analysis with VECTOR
Racquel Fygenson - Northeastern University, Boston, United States
Kazi Jawad - Weta FX, Auckland, New Zealand
Zongzhan Li - Art Center, Pasadena, United States
Francois Ayoub - California Institute of Technology, Pasadena, United States
Robert G Deen - California Institute of Technology, Pasadena, United States
Scott Davidoff - California Institute of Technology, Pasadena, United States
Dominik Moritz - Carnegie Mellon University, Pittsburgh, United States
Mauricio Hess-Flores - NASA-JPL, Pasadena, United States
Download preprint PDF
Download camera-ready PDF
Download Supplemental Material
Room: Bayshore VI
2024-10-17T15:18:00Z GMT-0600 Change your timezone on the schedule page
2024-10-17T15:18:00Z
Fast forward
Keywords
Computer vision, stereo image processing, optimization, error analysis, uncertainty, SLAM, SfM, robotics
Abstract
Reconstruction of 3D scenes from 2D images is a technical challenge that impacts domains from Earth and planetary sciences and space exploration to augmented and virtual reality. Typically, reconstruction algorithms first identify common features across images and then minimize reconstruction errors after estimating the shape of the terrain. This bundle adjustment (BA) step optimizes around a single, simplifying scalar value that obfuscates many possible causes of reconstruction errors (e.g., initial estimate of the position and orientation of the camera, lighting conditions, ease of feature detection in the terrain). Reconstruction errors can lead to inaccurate scientific inferences or endanger a spacecraft exploring a remote environment. To address this challenge, we present VECTOR, a visual analysis tool that improves error inspection for stereo reconstruction BA. VECTOR provides analysts with previously unavailable visibility into feature locations, camera pose, and computed 3D points. VECTOR was developed in partnership with the Perseverance Mars Rover and Ingenuity Mars Helicopter terrain reconstruction team at the NASA Jet Propulsion Laboratory. We report on how this tool was used to debug and improve terrain reconstruction for the Mars 2020 mission.