IEEE VIS Student Reviewer Guide

Overview

The IEEE VIS Student Reviewer Program introduces an optional student reviewer role to the IEEE VIS full paper peer review process. The program allows each primary reviewer to optionally invite one student reviewer per paper, complementing the existing review structure of primary, secondary, and external reviewers. This initiative aims to expand the community’s pool of skilled reviewers while providing valuable experience to early-career researchers.

Eligibility and Selection

The program is designed for Ph.D. students and researchers who are new to the visualization and visual analytics fields. While master’s students are generally not recommended as reviewers due to their typically shorter engagement with the research community, exceptions can be made for those continuing into Ph.D. programs. Postdoctoral researchers and established researchers transitioning to visualization from other fields are also welcome to participate as student reviewers.

While we recognize that not all participants in this program may, in fact, be “students”, we use the term loosely here to refer to someone who is studying a new field (i.e. reviewing for IEEE VIS). We hope that participants will accept this moniker even if they are already experienced professionals.

Unlike external reviewers, student reviewers may come from the same institution as other reviewers on the paper. In fact, primary reviewers are encouraged to recruit their own Ph.D. students or postdocs as student reviewers. The key consideration is having a clear line of communication with the student reviewer to facilitate effective mentoring throughout the review process.

Finally, the program is optional. Primaries can assign a student reviewer, but don’t have to. Primaries should do so only if they are able and willing to spend the necessary time on the student reviewer program.

Benefits for Student Reviewers

Student reviewers receive several valuable benefits from participating in the program. Most importantly, the program provides hands-on experience with the peer review process, helping students develop critical evaluation skills and understand how to write effective reviews. Furthermore, they receive formal credit for their service as an IEEE VIS reviewer, which can be included in their academic service record. Finally, through mentorship from experienced reviewers, student reviewers gain insight into the publication process that will benefit their own research and future contributions to the visualization community.

Timeline and Process

The student reviewer program follows the main conference review timeline, with specific steps at each stage:

1. Reviewer Recruitment (April)
   • Primary reviewers may invite one student reviewer per assigned paper through PCS.
   • Direct communication with the student reviewer is encouraged before sending the formal PCS invitation.
2. Discussion Period (May)
   • Student reviewers are encouraged to calibrate their reviews and ratings with other reviewers. They should also participate in the paper discussion on PCS.
   • Primary reviewers write their own review and provide feedback on student reviews.
   • Primaries verify that student reviewers have marked their expertise level as “student”.
   • Primaries assess review quality and request revisions if necessary.
   • Primaries may contact APCs/OPCs if inappropriate reviews need to be removed.
3. Decision Phase (May)
   • Primary reviewers may incorporate student reviews into their summary as they see fit and if the secondary reviewer agrees.
   • Student reviews can be considered partially or disregarded entirely (even removed if necessary).

Mentoring Responsibilities

Primary reviewers who invite student reviewers commit to mentoring them throughout the review process. This mentorship involves guiding them through the review criteria, ensuring they understand the expectations for constructive feedback, and providing feedback on their completed reviews.

Important: Primary reviewers should do their best not to influence the student reviewer’s decision or try to align their opinion with their own.

Student reviewers will be directed to the standard IEEE VIS reviewing guidelines and receive clear instructions about their role in the process. Their reviews should maintain the same professional standards as other reviews, though they are considered advisory in nature.

Writing a Student Review

As a student reviewer for IEEE VIS, you have the opportunity to contribute to the peer review process while learning from experienced researchers. Your review is considered advisory, but it can still provide valuable perspective to both the authors and other reviewers.

Review Form

When you submit your review, you must mark your expertise level as “student” in the review form. This is crucial as it signals to authors that your review is advisory in nature. Your primary reviewer will verify this designation.

Time Management

Reviewing requires significant time and attention. For your VIS review, plan to dedicate at least a full day to the process. You should plan to:

• Read the paper multiple times.
• Take detailed notes.
• Draft and iteratively revise your review.
• Consider asking your primary reviewer for feedback before the review deadline.

Writing Your Review

Focus on providing a thorough, constructive review. While several reviewing guides are available (see resources below), here are key points to remember:

• Neutrality: For non-anonymized submissions (IEEE VIS supports both single-blind and double-blind reviewing), maintain objectivity regardless of author identities. Don’t let author reputation influence your assessment positively or negatively.
• Objectivity: Take a balanced approach:
  • Avoid letting a single aspect determine your entire assessment;
  • Consider both strengths and weaknesses;
  • Keep an open mind throughout the process; and
  • Look at the work holistically.
• Calibration: New reviewers often face calibration challenges:
  • Some are overly critical, focusing too much on minor issues.
  • Others may be too lenient, missing significant flaws.
  • Both extremes are natural; awareness helps you find balance

Working with Your Primary Reviewer

Your primary reviewer serves as your mentor in this process. While you should form your own opinions about the paper, don’t hesitate to seek guidance:

• You can share your draft review for feedback before submission.
• Ask questions about the review process.
• Seek clarification about aspects of the paper you find unclear.
• Approach this as a learning opportunity.

During the Discussion Phase

Once other reviews become visible in PCS during the discussion phase:

• Read them to see what experienced reviewers emphasize.
• Notice what they consider important (or not).
• You don’t need to modify your review to match others.
• Don’t feel pressured to align your rating with the average.

Remember, this isn’t an exam where you need to catch every point others make. Instead, think of your review as one lens through which to help the authors improve their paper.

Participating in Discussions

Engage in the discussion forum, even if briefly:

• Acknowledge that you’ve read other reviews.
• Share your perspective.
• Respond to questions about your review.
• Indicate whether you agree with the primary’s direction.
• Ask questions if you’re uncertain about something

Review Resources and Guides

• The IEEE VIS Reviewing Guide
• How to Review HCI/Visualization Papers (Niklas Elmqvist)
• Mistakes Reviewers Make (Niklas Elmqvist)
• Tips for being a Good Visualization Paper Reviewer (John Stasko)

Questions & Concerns

For questions about the Student Reviewer Program, please contact the IEEE VIS Overall Papers Chairs at opc@ieeevis.org.