13 - 18 OCTOBER 2013, ATLANTA, GEORGIA, USA

Evaluation: How Much Evaluation is Enough?

Organizers
Organizer: 
Robert S. Laramee
Panelists: 
Min Chen
Panelists: 
David Ebert
Panelists: 
Brian Fisher
Panelists: 
Tamara Munzner
Description

Most of us agree that evaluation is a critical aspect of any visualization research paper. There are many different aspects to the topic of evaluation including: performance-based such as evaluating computational speed and memory requirements. Other angles are human-centered like user-studies, and domain expert reviews to name a few examples. In order to demonstrate that a piece of visualization work yields a contribution, it must undergo some type of evaluation. The peer-review process itself is a type of evaluation. When we referee a research paper, we evaluate whether or not the visualization work being described has been evaluated adequately and appropriately. In an increasing number of cases papers are rejected based on what was judged, at that time, to contain an inadequate evaluation even though the technical or design contributions are acknowledged. However, there are differing opinions as to what constitutes an adequate or appropriate evaluation when it comes to visualization. In this panel, we discuss precisely this topic: What constitutes an adequate and appropriate evaluation of a piece of visualization work?