We get it—waiting is hard. As we move into May, the early June notification date for the outcomes of your submissions seems no closer. Also, it could be that the decision-making process at IEEE VIS 2024 can appear opaque from the submitting authors’ viewpoint. Well, wonder no more: in this blog post, the next installment in our “Road to VIS 2024” series, we will describe what actually goes into these decisions on which papers get accepted or rejected to VIS 2024. Maybe it will not make June come sooner, but we hope that it at least will make understanding the decision-making process easier.
Let’s start with recent history. All VIS submissions were due on March 31, a date that has—save for the pandemic year of 2020 when authors were given an extra month—been constant across the decades for the IEEE visualization conferences. Once the submission deadline passed, we, the OPCs, closed the submission system, discarded incomplete submissions, and made an initial assignment of program committee (PC) members to each paper. We then handed these assignments over to the APCs on April 1—the very next day. The APCs had until April 4 to fine-tune the assignments, which were released to the PC members on April 5.
If you are a reviewer for IEEE VIS 2024—and we hope that you are—you have experienced the rest of this process firsthand. From April 5 to April 11, the PC members recruited one external reviewer for the papers for which they served as secondary. During this period, we also managed numerous paper swaps necessitated by conflicts of interest that were initially overlooked. On April 11, we sent out all review invitations in a single batch. This strategy allows external reviewers to select assignments they feel most qualified for and avoids penalizing PC members who were unable to recruit reviewers immediately. After the review request batch, there followed some hilarity as PC members scrambled—with varying degrees of success—to find replacements for review requests that were declined in the first round. This year, the period of hilarity (or tragedy, as it were) lasted until nary a week before the reviewing deadline.
The reviewing deadline on May 8 is followed by a week of discussions. During this time, all reviewers (primary, secondary, and one external) can see each other’s reviews and discuss their opinions of the submissions. Reviewers are encouraged to remain open to adjusting their evaluations based on the discussion, whether it raises new positives or negatives. At the end of the discussion period, the primary reviewer synthesizes all feedback into a summary review.
At this point, the APCs take over. Their job is now to triage the papers under their care to make preliminary recommendations and to identify the borderline cases that need deeper deliberation. This step often involves reading summaries, discussions, full reviews, and, in complex situations, the papers themselves. Each area enjoys considerable autonomy in discussing and determining the outcomes of the papers they manage. The APCs are also on the lookout for inappropriate, inadequate, or insufficiently clear reviews so that they can follow up with those reviewers to improve or clarify any problematic reviews. In some cases, they may even engage an entirely new reviewer to perform a “crash” emergency review at short notice. Once finished with their pass, the APCs hand off their recommendations to the OPCs, who discuss with each area, check that the decisions—especially the borderline ones—are sufficiently documented, and ensure that accept/reject criteria are applied consistently across areas.
The final approval before notifications are dispatched is handled by IEEE TVCG. As most visualization community members know, papers accepted to the annual IEEE VIS conference are published in a special issue of the IEEE Transactions on Visualization and Computer Graphics journal. The TVCG Editor-in-Chief, Han-Wei Shen, and his editorial board have the final say on what gets accepted to appear in the journal. For the OPCs and APCs, this means that each accept and reject decision must be sufficiently documented so that the rationale is clear and founded on scientific principles. This rationale will later be used to verify that the second-round version of each conditionally accepted paper has addressed reviewer concerns—but that’s a story for another post.
On June 6, the fruits of our reviewers’ collective labor will be revealed. We wish you the best of luck for your submissions; may the odds be ever in your favor!