Extend the access of reviewers to other submissions and/or reviews

After the assignment process, our reviewers have only access to submissions allocated to them. To give our reviewers a better overview related to their priority topics and the available proposals, we would like to extend access to further information like all other submissions or the evaluations of other reviewers / their co-reviewers.

Furthermore, is it possible to allow them to communicate with other reviewers via the system?

(A German version of this article is available.)

Usually reviewers have only access to contributions that were assigned to them and when the reviewing phase is enabled. Many organizers consider the submissions of authors confidential, as they represent new, unpublished work and therefore they want to limit the access to as few people as possible. Often organizers also want to prevent reviewers from being influenced by the evaluations of other reviewers.

But if required, there are several options to extend the access of your reviewers to further information. Below you find some of the available options covering different requirements.

Using the PC online forum to give reviewers read access to all submissions

Users with the role 'Reviewer' usually don’t have access to all submissions, whereas this is easily possible for users with the role 'Members of the Program Committee (PC Member)' who can access all submission in the PC online forum. Giving reviewers this status (too) and enabling the PC online forum in "Phases and Deadlines" is usually the easiest way to give your reviewers access to all submissions.

For more detailed information on user roles please consult the documentation User Roles and Statuses in ConfTool.

Please note that the online forum can be enabled and disabled for each submission type / track separately. If the track chair module is enabled, PC members get only access to the submissions of the tracks assigned to them.

Access to Co-Reviews and the Online Forum for Review Discussion

On the page for "Main Setting for Paper Reviewing" you can give reviewers access to the reviews of their co-reviewers (reviewers evaluating the same papers). Furthermore, you can give them restricted access to the PC Online Forum for the submissions that they are reviewing themselves. (NB: The status PC member is not required in this case, but the reviewers get only access to the forum for the paper assigned to them).

Please go to:
Overview => Settings => Main Setting for Paper Reviewing
… and scroll down to the section “Review Options” and update the option
“Access to Reviews of Co-Reviewers? / Access of Reviewers to Online Forum” (see image 1).
 
If enabled, reviewers can get access to all co-reviews during the reviewing phase from the page "Enter and Edit Reviews" in their user account. If you also enable the access to the forum here by selecting the option "Access to co-review and forum AFTER own review submission", please remember to enable the PC online forum in "Phases and Deadlines".

Meta-Reviewers

"Meta-Reviewers" stands for a reviewing option that allows the corresponding reviewers / PC members to access the reviews of all their co-reviewers of the submissions for which the organizers assigned this role to them. Please note that this is not a user status in general, but a status that is set during the assignment process for each review assignment separately.

To use this role, please go to:
Overview => Settings => Main Setting for Paper Reviewing
… and scroll down to the section "Review Options" and update the option "Enable Meta-Reviews?" (image 2).

Hint: There is a second function called "Senior Reviewers" that has the same function (it just has a different name and may allow you to distinguish two different groups of "Meta-Reviewers") and it can be enabled on the same page if you activate the expert settings at the bottom of the page first.

Now you can see an extra checkbox on the pages to assign reviewers to submissions and submissions to reviewers (see image 3).

Furthermore, you can enable a messaging function that allows Meta-Reviewers to send messages to the other reviewers via the system. Please go to
Overview => Settings => Main Settings for the Review Process
... and there down to section "Messaging between Chairs, Meta-Reviewers, Senior Reviewers and Reviewers".

You can find more information here: Meta reviewers / Providing access to the reviewers of the same papers

Using the bidding function to show basic information about all submissions

The function "Bid for Contribution" is another option to give reviewers an overview of all submissions of all tracks / submission types (for which the bidding function is enabled). Usually bidding is used prior to the reviewing process and it allows reviewers to leave bids for submissions they really want or would like to evaluate and also it allows them to state conflicts of interest. But of course it can also be (left) enabled after the reviewing process has started. By using filters, the reviewers can narrow the selection down to specific submission types or topics (image 4).

Please go to:
Overview  => Settings  => Main Setting for Paper Reviewing
… scroll down to the section "Bidding: Reviewers Can Bid for Submitted Contributions", enable the bidding phase by clicking on the 'Disabled/Enabled' button (see image 5) and choose a starting and ending date. Please also make sure that the bidding function is active for each submission type / track, i.e. the option: "Bid for Contributions" must be checked.

Hint for experts: You can also rename the page as well as the possible bidding options ("much wanted", "would like" etc.). If you replace the options by empty strings you can even disable the bidding options.

Assignment of all incoming submissions automatically to reviewers based on tracks or topics

If you want to allow all reviewers to submit reviews to all incoming submissions of their areas of expertise, you can enable a function that automatically assigns all suitable contributions directly after submission to all suitable reviewers.

Please go to:
Overview  => Settings  => Main Setting for Paper Reviewing
… and turn on the Expert Settings toggle in the lower navigation bar.
The toggle is labeled "Expert Settings Disabled" until activated.
Choose one of the settings for the option "Automatic Assignment of Submissions" to enable the automatic assignment of all incoming submissions to all reviewers listed for the same tracks (if the track-chair module is enabled) or topics (image 6).

To assign topics (and/or tracks if the track-chair module is enabled) to reviewers, please go to
Overview  =>  Submissions & Reviews = > Program Committee and Reviewers
… and assign the topics (and/or tracks) to each reviewer before the submission process starts.

Please note: If the submission process is already ongoing or over, use the automatic assignment function to assign all received submissions to suitable reviewers.

Please got to:
Overview => Submissions & Reviews => Automatic Assignment of Reviews
… and select the settings that best fit your needs (e.g. choose "Number of Reviews per Submission" corresponding to your number of reviewers - see image 7) to start this process. More information on the automatic assignment process can be found here: Automatic review assignments.