In a post by the VIS Executive Committee (VEC) earlier this year, we introduced our initiative to determine the extent to which VIS should assume a hybrid format, so as to broaden participation while ensuring positive experiences for both in-person and remote attendees. As part of this initiative, we partnered with members of the IEEE Visualization and Computer Graphics Technical Community Executive Committee (VGTC ExCom) to conduct an online survey to take the pulse of the community, to collect data regarding conference attendance patterns and preferences, to determine the sentiment toward potential hybrid experiences, and to collect ideas. In this post, we summarize the results of this survey.

Since January, 192 people responded to our call to participate in the survey, which we disseminated via community mailing lists and social media. 138 respondents selected VIS as their primary VGTC conference; our analysis focused on this subset of responses. We believe that the VIS community is reflected in the survey responses, which span the dimensions of world region, gender, level of education, age, affiliation, and VIS attendance history. However, some voices are either over- or under-represented when we compare the demographics of survey respondents to VIS attendance data; for instance, VIS attendees from North America are slightly over-represented, while those from Oceania are under-represented.

Reflection on Prior Conference Experiences

In the first half of the survey, we asked about recent conference experiences, including the motivations for attending either in person or virtually, as well as the barriers that prevent people from attending a conference in person. The results are shown in Figure 1. For each question, respondents could select up to three considerations, which included an ‘Other’ option that allowed them to specify their own. Highlights include:

Figure 1: Community responses to three questions from an online survey regarding IEEE VGTC conferences, filtered to the subset respondents who identified IEEE VIS as their primary VGTC conference (N = 138). For each question, respondents could select up to three considerations, which included an ‘Other’ option that allowed them to specify their own.

• Top three motivations to attend VIS in-person: (1) networking, (2) present work, (3) attend paper presentations.
• Top three barriers to attending VIS in-person: (1) cost of travel and accommodation, (2) time availability, (3) paper not accepted.
• Top three motivations to attend conferences virtually: (1) no travel required, (2) no in-person attendance option offered, (3) affordability.

The open-ended comments largely corroborate what we saw in the fixed-response questions, however we also noted a recurring desire for hybrid formats to ensure accessibility. Additionally, respondents indicated how environmental concerns also influence their attendance decisions, with some choosing not to fly or combining travel with personal time to mitigate environmental impact. Finally, several respondents mentioned significant personal safety and human rights concerns associated with potential conference locations.

Next, we asked about respondents’ preferences and behaviors when attending conferences virtually. The majority of respondents reported consuming less than 50% of the conference program content, preferred livestreaming some of the content and watching the rest asynchronously. The majority preferred interacting with speakers via a basic chat interface, Slack or Discord, or a web application like sli.do. They preferred interacting with other attendees via Slack or Discord, a basic chat interface, or asynchronously via email. In the open-ended comments, we noted a particular appreciation for the Discord instance associated with VIS 2022, although the effectiveness of such a platform depends on activity levels.

Opinions on Future Conferences

In the second half of the survey, we asked respondents for their opinions and preferences with respect to future conferences. In particular, we asked participants to either to agree or disagree with a series of statements regarding the specifics of possible hybrid conference arrangements (See Figure 2), which included statements about presentation video recorded before the conference, presenting at satellite events, and the integration of in-person and virtual presentations.

Figure 2: Percentage of survey responses (N = 138) that Disagree or Agree with the statements regarding future VIS conferences listed in the left column, along with the percentage of Neutral responses.

So what do respondents largely agree on? Most respondents (88%) want video recordings of in-person presentations and panels to be made available after the conference (7% opposed, 5% neutral). The majority (59%) agree that presentations by speakers who cannot attend in person should be broadcast at the in-person conference venue at a designated time and room (13% opposed, 28% neutral). Similarly, the majority (57%) agree that Q&A following presentations should accommodate in-person and virtual attendees to equal measure (24% opposed, 18% neutral). Finally, the majority of respondents (55%) agree that those who wish to host a future VIS conference must propose a virtual attendee experience (29% opposed, 16% neutral). The open-ended comments largely support these preferences, with calls for standardizing presentation formats across sessions and using familiar remote communication technology.

There was no clear majority opinion for other questions that we posed, though if we consider the percentage of neutral responses, we should avoid requiring pre-recorded presentations from speakers who elect to present in-person, and we should not consider changing the annual cadence of an in-person VIS conference.

Unsurprisingly, nearly half of the respondents said “it depends” when asked if they will opt to attend future VIS conferences remotely; less than 10% said they would attend virtually for most / all conferences that offered such a format, with about the same percentage saying they do not plan to attend future conferences virtually. The majority of respondents indicated that the virtual registration fee should be 25% or less than the full in-person registration fee. If attending virtually, the majority of respondents indicated that they would livestream keynote sessions but opt to watch paper talks asynchronously. This preference is also reflected in CHI 2024 organizers’ decision not to provide a synchronous conference experience for remote attendees, or as they put it: “Live is Synchronous, Remote is Asynchronous”.

Opinions on Satellite Events

Finally, relative to remote participation in general, we noted less enthusiasm or willingness to attend satellite conference events. The comments suggest that the quality of satellite events depends heavily on attendance and local organization strength, and that the attendance of a satellite event is contingent on the number and presence of local paper presenters. However, European satellite events are particularly attractive relative to others due to Europe’s developed rail travel network, particularly when the VIS conference takes place outside of Europe.

Read more »

We get it—waiting is hard. As we move into May, the early June notification date for the outcomes of your submissions seems no closer. Also, it could be that the decision-making process at IEEE VIS 2024 can appear opaque from the submitting authors’ viewpoint. Well, wonder no more: in this blog post, the next installment in our “Road to VIS 2024” series, we will describe what actually goes into these decisions on which papers get accepted or rejected to VIS 2024. Maybe it will not make June come sooner, but we hope that it at least will make understanding the decision-making process easier.

Let’s start with recent history. All VIS submissions were due on March 31, a date that has—save for the pandemic year of 2020 when authors were given an extra month—been constant across the decades for the IEEE visualization conferences. Once the submission deadline passed, we, the OPCs, closed the submission system, discarded incomplete submissions, and made an initial assignment of program committee (PC) members to each paper. We then handed these assignments over to the APCs on April 1—the very next day. The APCs had until April 4 to fine-tune the assignments, which were released to the PC members on April 5.

If you are a reviewer for IEEE VIS 2024—and we hope that you are—you have experienced the rest of this process firsthand. From April 5 to April 11, the PC members recruited one external reviewer for the papers for which they served as secondary. During this period, we also managed numerous paper swaps necessitated by conflicts of interest that were initially overlooked. On April 11, we sent out all review invitations in a single batch. This strategy allows external reviewers to select assignments they feel most qualified for and avoids penalizing PC members who were unable to recruit reviewers immediately. After the review request batch, there followed some hilarity as PC members scrambled—with varying degrees of success—to find replacements for review requests that were declined in the first round. This year, the period of hilarity (or tragedy, as it were) lasted until nary a week before the reviewing deadline.

The reviewing deadline on May 8 is followed by a week of discussions. During this time, all reviewers (primary, secondary, and one external) can see each other’s reviews and discuss their opinions of the submissions. Reviewers are encouraged to remain open to adjusting their evaluations based on the discussion, whether it raises new positives or negatives. At the end of the discussion period, the primary reviewer synthesizes all feedback into a summary review.

At this point, the APCs take over. Their job is now to triage the papers under their care to make preliminary recommendations and to identify the borderline cases that need deeper deliberation. This step often involves reading summaries, discussions, full reviews, and, in complex situations, the papers themselves. Each area enjoys considerable autonomy in discussing and determining the outcomes of the papers they manage. The APCs are also on the lookout for inappropriate, inadequate, or insufficiently clear reviews so that they can follow up with those reviewers to improve or clarify any problematic reviews. In some cases, they may even engage an entirely new reviewer to perform a “crash” emergency review at short notice. Once finished with their pass, the APCs hand off their recommendations to the OPCs, who discuss with each area, check that the decisions—especially the borderline ones—are sufficiently documented, and ensure that accept/reject criteria are applied consistently across areas.

The final approval before notifications are dispatched is handled by IEEE TVCG. As most visualization community members know, papers accepted to the annual IEEE VIS conference are published in a special issue of the IEEE Transactions on Visualization and Computer Graphics journal. The TVCG Editor-in-Chief, Han-Wei Shen, and his editorial board have the final say on what gets accepted to appear in the journal. For the OPCs and APCs, this means that each accept and reject decision must be sufficiently documented so that the rationale is clear and founded on scientific principles. This rationale will later be used to verify that the second-round version of each conditionally accepted paper has addressed reviewer concerns—but that’s a story for another post.

On June 6, the fruits of our reviewers’ collective labor will be revealed. We wish you the best of luck for your submissions; may the odds be ever in your favor!

Read more »

March 31 has come and gone, which means that the IEEE VIS 2024 full papers deadline has passed. This is when us OPCs and APCs are starting our work, and when the PC and eventually the entire reviewer community will get involved. What is the task? Why, reviewing all those papers that were submitted to the annual deadline, of course! In this brief post, we will give some numbers about the submissions and then outline the next steps in the process.

A total of 680 abstracts were submitted for the abstract deadline on March 21. Out of those, 557 full papers were eventually submitted, a conversion rate of 82%. The number of submissions is up; in 2023, the conference saw 539 submissions out of 635 abstracts (85% conversion), and in 2022, there were 460 out of 560 abstracts (82% conversion). This means submissions increased by 3.53% from last year, and 21% from 2022. This is a fairly small increase compared to the 17% increase from 2022 to 2023. However, the conversion rate of abstract to full paper is consistent.

All APCs pull a very high load and we are grateful for their efforts. Still, some areas received more submissions than others. For the specific areas, Applications (Area 2)—not surprisingly—received the most number of submissions: 154. We wish Tatiana von Landesberger (University of Cologne, Germany) and Jiawan Zhang (Tianjin University) all the best with handling all these submissions! Second came Theoretical & Empirical (Area 1) with 112 submissions. This area is spearheaded by APCs Adam Perer (Carnegie Mellon University) and Matthew Kay (Northwestern University). This is narrowly followed by Representations & Interaction (Area 4) with 110 submissions, and stable hands Daniel Keefe (University of Minnesota) and Pierre Dragicevic (Inria Bordeaux) at the helm. Analytics & Decisions (Area 6) APCs Wenwen Dou (University of North Carolina at Charlotte) and Steffen Koch (University of Stuttgart) are handling 77 submissions. Data Transformations (Area 5) has 53 submissions, and is managed by Filip Sadlo (Heidelberg University) and Ivan Viola (King Abdullah University of Science and Technology). Finally, weighing in at 52 submissions is the Systems & Rendering area (Area 3) with Chaoli Wang (University of Notre Dame) and Christoph Garth (University of Kaiserslautern-Landau) at the wheel. Thanks all to these APCs who make our jobs as OPCs possible!

Supporting these 12 APCs is a program committee of 143 hard-working PC members who can expect to handle somewhere between 6 to 8 papers each (split between the role as primary or secondary reviewer). This PC, incidentally, was recruited earlier this spring from a total of 181 invitations. If you are thinking about serving on the VIS program committee in the future, we hope you look out for our annual PC volunteering deadline sometime next fall! We are deeply grateful to these PC members as well as the external reviewers they will be inviting to review these submissions in the weeks to come.

From this point onwards, the VIS review process will proceed with the PC member assignments, which will be released before April 8. After that point, secondary reviewers will look for an external reviewer for each submission. Remember that this year, there is only one external reviewer assigned to each paper. The initial batch of external review invitations will be released on April 11. The external reviews will be due on May 8. The reviews from primary and secondary PC members are due at the same time. Then, discussion will take place between all reviewers on the paper, and the primaries have until May 15 write their summary review. First-round notifications are expected to be released on June 6.

We OPCs are quite excited about all this and are eager to tackle this hard but important work. During our first skim of the submissions, we have seen some very impressive, creative, and potentially important papers. Thanks for submitting to VIS 2024, and we wish you the very best of outcomes for your hard work!

Read more »

We’re entering March and the game is heating up! With less than three weeks to go for the VIS 2024 full papers deadline, it is time that we OPCs start turning to critical matters. One of those is conflicts of interest (COIs). In the last installment, we talked about the importance of volunteering to review papers for the conferences to which you submit papers. However, it is equally vital not only that you are reviewing papers, but that you don’t review papers for which you have a conflict.

What is a conflict of interest? COIs arise because of relationships you have with an author or their institution that could affect your judgment of the work or the community’s perception of your judgment. Note the last point. You may be a person of exceptional integrity that could objectively handle reviewing even your own sibling’s work fairly and with no special treatment, but reviewing your own sibling’s paper would look bad for everyone else. One of the pillars of scientific peer review is that it is objective, and that this objectivity can be confirmed by everyone. Conflicts arise in many ways and have varying durations; see the IEEE VGTC reviewer ethics guide for all the details. Family relationships are obviously conflicts with no expiration, but those are relatively rare and quite straightforward. Another conflict is the academic equivalent of a family relationship: your Ph.D. and postdoc advisor, or anyone who has had a close mentorship role for you, are “forever conflicts”. The same is true in the other direction, i.e. for your advisees. Your close relationship means that neither of you can be expected to (or perceived to) treat each other objectively. The same can be said about close personal friendships (or, for that matter, personal animosities, which hopefully are rare).

Most other conflicts have a time expiration, and for IEEE and VIS, this expiration is three years. In other words, once a relationship has ended (e.g. collaboration on the same paper), you can consider the conflict gone after three years.

The rule of thumb is the same for conflicts with an expiration as those with none: is there a relationship that would (or would be perceived to) affect your ability to treat a person objectively? Conflicts with an expiration include sharing an affiliation, co-authorship on published work, working on the same research project or grant, or similar. Service commitments are special: you are obviously not conflicted if you serve on the same program committee, because then basically everyone in the community would be conflicted with each other. In general, even if you work with somebody closely on a small committee over an extended period of time, conflicts do not automatically arise; however, if you become sufficiently close to somebody that it feels like a conflict, then do declare it. Use your best judgment here.

As an aside, us OPCs are selected not to be conflicted with each other because we need to be able to handle each other’s submissions and conflicted submissions.

If you have a conflict with a paper, you should not be involved in any formal publication decisions regarding it. For external reviewers, this means not reviewing papers you are in conflict with. For PC members, this also means not reviewing such papers, and informing the papers chairs immediately if you are assigned to such a paper. For APCs at VIS, this means that if both APCs are in conflict, the conflicted paper will be moved to a different area to avoid the conflict. If just one APC is in conflict, one of the OPCs will step into their place. For OPCs, there is no such option, so in these situations, the conflicted chair will have to recuse themselves from any decisions involving the conflicted paper. Practically speaking, this will mean not participating in the Zoom call when discussing it. The PCS submission system provides good support for handling conflicts, by ensuring that papers chairs cannot see information about such papers.

Here’s the final question we want to cover: how do we detect these conflicts in the first place? This is where you come in. If you are a reviewer for VIS 2024 (either as a PC member or as an external reviewer), it is your responsibility to ensure that your affiliation is correct and that your conflicts have been updated in PCS (the submission system). Because VIS allows for double-blind submissions, we need PCS to flag situations when there is a conflict even if you as a reviewer don’t see the author names. Even for single-blind submissions, where the author names are visible to you, correct affiliations and declared conflicts will minimize situations where you get assigned a paper you really shouldn’t review, and then we all have to go through the hassle of getting the paper reassigned to another reviewer.

Many early-career researchers in the community will have recently changed affiliations, for example Ph.D. students who have graduated and moved on to new institutions. Please make sure you have updated PCS with your new primary affiliation, and also keep your old institution listed as your secondary affiliation for 3 years—the three-year rule applies here as well. Conversely, once those 3 years are up, please remove that institution from your own affiliation list. If you are still actively collaborating with people there, that should be handled through the usual conflicts identification mechanisms in PCS, not through your affiliation.

Speaking of PCS and conflicts: the good news is that PCS will now help accelerate the process of declaring your conflicts by checking against all recent submissions in its database. The bad news is that it’s sometimes over-enthusiastic: it uses a 4-year window rather than a 3-year window, and includes a few tracks where we do not consider conflicts to occur (such as shared participation in a panel or tutorial). So do take a close look at the automatic inferrals, in addition to entering information about any other conflicts such as new collaborations where you have not yet submitted papers together as co-authors.

With all this information fresh in your mind, please take a moment to go into PCS and make sure your affiliation is up to date. (If you’re on the VIS PC, you will also be asked to update your conflicts after the abstracts submission deadline.) Providing all this information already now will make everybody’s life easier.

Read more »

In the previous post by the The VIS Executive Committee (VEC), we outlined our initiative to investigate the wicked problem of planning future VIS conferences, focusing on the extent to which the conference should assume a hybrid format or consider other alternative formats to broaden participation, while ensuring positive experiences for both in-person and remote attendees.

We now continue this initiative with an overview of VIS attendance patterns between 2018 and 2023. This period is particularly interesting given the variation in location / time zone and format of VIS during this time:

• 2018: Berlin (in person, GMT+1)
• 2019: Vancouver (in person, GMT-8)
• 2020: Salt Lake City (virtual, in GMT-7)
• 2021: New Orleans (virtual, in GMT-6, with several in-person satellite events around the world)
• 2022: Oklahoma City (synchronous hybrid, GMT-6)
• 2023: Melbourne (in person, GMT+11)

For each year listed above, we collected anonymized registration data aggregated by world region and country. For the synchronous hybrid conference of 2022, we also aggregated the registrations by registration format (in-person or virtual). We acknowledge that when an attendee registers for VIS, their response to the location fields in the registration form may not necessarily represent their country of origin, but rather where they are currently employed or studying. Furthermore, there is no way of verifying if the self-reported country for a person who registered to attend VIS virtually between 2020 and 2022 was the country that they were physically in during the conference. Lastly, we do not currently have any data regarding the level of engagement for virtual attendees beyond registration.

Caveats aside, we report on attendance for the benefit of those considering to make a bid to host VIS in the coming years, VIS organizing committee members planning and scheduling virtual or hybrid program content, as well as for those inclined to host satellite events, such as the eight events that coincided with VIS 2021 or the asynchronous VIS 2023 satellite event in France.

Overall VIS Registration Numbers

VIS 2018 in Berlin remains to be largest VIS conference to date in terms of in-person attendee count (over 1,200), while VIS 2020 had the highest number of virtual registrations (nearly 6,000). We also saw nearly 3,700 virtual registrations in 2021 and nearly 700 in 2022 (the latter also had over 600 in-person attendees). Most recently, VIS 2023 attracted over 800 in-person attendees.

VIS Attendance by Region

Irrespective of the means of attendance (in-person / virtual), we saw some notable differences in terms of the proportion of attendees from different world regions (◼︎ = Europe; ◼︎ = Americas; ◼︎ = Asia; ◼︎ = Oceania; ◼︎ = Africa).

When VIS was in Germany (2018), a little over half of the attendees were based in Europe and made the short trip to Berlin; about a third flew in from North America, just over 10% flew in from Asia, and 1% traveled from somewhere else. The next year (Vancouver), the relative proportions of Europeans and North Americans inverted. Notably, over the next three years, the relative proportions of attendees from North America, Europe, and Asia remained somewhat stable. While the VIS program was virtual in 2020 - 2021 and hybrid in 2022, recall that each program was scheduled in time zone that was convenient for those in North America. Finally, at the most recent VIS conference in Melbourne, which had no virtual attendance option, we saw a fairly even split in terms of who made the trip to Australia: about 30% flew in from North America, 26% came from Asia, 24% were based in Oceania, and 19% made the long trip from Europe.

Virtual VIS Attendance

A couple of questions arise from our virtual registration numbers: (1) Did VIS have a broader geographic reach during the virtual-only years (2020-21)?; and (2) Who opted to attend the hybrid VIS 2022 conference virtually?

Among the virtual registrations, the majority were from those based in world regions and countries already well-represented at VIS: North America, Western and Northern Europe, and East Asia. However, we made special note of the countries that were not represented among in-person registrations between 2018 and 2023. Countries in Southern and Southeast Asia stand out in this pack, with a total of over 100 registrations between 2020 and 2022. About half as many registrations over this period were from those based in Eastern and Southern Europe. Finally, we saw a relatively small number of registrations from Africa and the Americas outside of North America.

Looking at VIS 2022 in particular, a little over 40% of the nearly 700 virtual registrations were from those based in the Americas, while a little under 30% were based in Asia, and just over a quarter were based in Europe. Across all of these registrations, only a couple dozen (~3%) were from those based in countries that had no in-person representation across the years surveyed.

Implications

Aside from the global COVID-19 pandemic that prevented international in-person gatherings in 2020 and 2021, the registration data by itself doesn’t tell us why participants opted to attend virtually in 2022. Nor does it tell us which factors matter to prospective VIS attendees mulling over whether to attend in person or virtually; this is the focus of our the survey announced in our earlier post, and we will present the results of this survey in a future post. However, the registration data does suggest that there is not a large difference in terms of global representation between in-person and virtual attendees.

While attending a regional satellite event may not be accessible or appealing to those who decide against traveling in person to the main event, the virtual registration numbers suggest that there may be a large enough contingent of people in Europe and East Asia to merit satellite events in those locales when VIS takes place in North America. There may even be a critical mass of people in South Asia and Southeast Asia willing to attend such a regional event.

While just over 40% of the virtual registrations for VIS 2022 were from those based in the Americas, the synchronous virtual conference experience for the remaining ~60% was likely to occur at an inconvenient time of day. To accommodate virtual attendees of future VIS conferences, organizers should consider the benefits and trade-offs of asynchronous or partially-synchronous virtual attendee experiences. For example, let’s imagine a future VIS conference hosted somewhere in North America. Presentations delivered by in-person attendees could be recorded during the day and uploaded in the evening, at which point they could be viewed asynchronously by remote attendees. Early the following morning, those whose talks were recorded could join a moderated virtual Q&A session with remote attendees in Europe and Africa, and later that day, they could join a second moderated virtual Q&A session with remote attendees in Asia and Oceania.

At VIS 2024 in Florida, the general chairs have already determined that paper presentation sessions will be split between those delivered by in-person attendees and those delivered by remote virtual attendees. Should this split continue in future VIS conferences, the virtual registration demographics suggest that these virtual-only sessions be scheduled in the early morning and late afternoon / early evening, so as to respect the typical working hours of those attending from around the world.

Read more »