5 Ways to build a database of journal peer reviewers you can rely on

Image Credit: Haniel Espinal

Building a reliable list of peer reviewers for incoming submissions is one of the most challenging parts of being a journal editor. It can be hard enough to get in front of researchers with countless demands competing for their attention, and convincing them to agree to take time out of their hectic schedules to complete review assignments can prove harder still.

Given how difficult it is to find peer reviewers, it’s vital for journal editors to consistently work to cultivate a database of willing candidates and prioritize referee retention initiatives. Think of your reviewer database like you would a sports team. You want to make sure you have peer reviewers who are skilled players in all areas of your journal’s field of research (see what we did there?) and, above all, that you have a large enough team so you can easily sub in different reviewers when one of your top picks gets too busy or needs a break.

Are you wondering how to go about cultivating a reliable reviewer database? Whether you’re just starting to develop a reviewer list or you already have reviewers on deck and want to take steps to ensure you keep them, here are 5 tips to help:

1. Always be on the lookout for qualified peer reviewers

If you want to maintain a reliable reviewer database, first and foremost, your editorial team should always be on the lookout for qualified candidates. Some ways you can look for new reviewers include:

• Using external reviewer databases to find researchers working on similar topics
• Querying indexes like Web of Science (WoS) for authors with multiple publications in relevant subject areas
• Reviewing past author data for potential candidates
• Checking the references section of manuscript submissions for potential reviewers
• Posting a call for reviewers on your journal’s website and social media channels

Want more in-depth tips for finding new peer reviewers? We cover 7 in this blog post.

Of course, the term qualified candidates is paramount here. It’s imperative to ensure potential peer reviewers don’t have any apparent conflicts of interest, such as having published recently with an author of the manuscript you’re seeking reviewers for or working with the same institution/research group as an author or a competing one. Journals should also aim to cultivate a diverse pool of peer reviewers in terms of race, gender, ethnicity, nationality, career stage, etc., to invite and represent varied perspectives.

2. Use data to guide if and when to reach out to reviewers

In the process of finding willing reviewer candidates, as the name of this blog post states, it’s imperative to develop a database with all of the information you need to quickly pull up those contacts and determine appropriate times to reach out to them.

Before sending out a new reviewer invitation, all editors should ask themselves the primary questions: Is this review assignment within this scholar’s areas of expertise? Does this scholar have any known conflicts of interest? Have any editors reached out to this reviewer recently, and if so, is it too soon to make another ask? If the reviewer has taken up assignments for the journal in the past, did they complete them on time?

Your reviewer database should consist of data points to help answer the above questions, including qualitative insights like:

• Reviewers’ areas of expertise, so you can quickly check whether a candidate has the necessary experience for the manuscript in question (using peer review software with tagging functionality like Scholastica can save you a lot of time here)
• Institutional affiliations, so you can quickly check for qualifications and potential conflicts of interest
• Notes from editors with pertinent information from past interactions with the reviewer

You should also include the following quantitative data points, which will help you determine reviewers’ reliability and whether it’s OK to send them a new invitation to review or too close to past outreach:

• Number of reviews completed for the journal
• Average days to complete a review
• Number of reviews finished on time (if a reviewer tends to be late, it may be time to look for alternate options)
• Total times invited to peer review
• Number of declined peer review assignments (if you notice a pattern where a reviewer is consistently declining invitations, you should probably give them a break)
• Number of in-progress, late, and completed reviews (obviously, you won’t want to send invitations to reviewers with in-progress assignments or reviewers who tend to be late unless absolutely necessary)

Journals using Scholastica’s Peer Review System have access to all the above insights, including reviewer tagging functionality, from a central Reviewer’s Table and individual reviewer profiles. When scholars sign up for Scholastica accounts as reviewers, they also have the opportunity to input research specialties, which editors can reference in addition to journal-assigned reviewer tags.

3. Provide clear reviewer guidelines and reminders

Key to building a database of reliable reviewers is providing a positive peer review experience. You want to ensure it’s as smooth as possible every time scholars work with your journal, so they’ll be inclined to keep coming back. That starts with providing clear reviewer guidelines upfront, so reviewers know what will be expected of them before they respond to an invitation and have all of the information to complete a review on hand if they accept. Your reviewer guidelines should include:

• An overview of your journal’s peer review process (e.g., type of peer review, the typical length of time, how decisions are made, etc.)
• Your journal’s ethical policies, with specific steps reviewers should take in the event of a potential conflict of interest
• The criteria for publication reviewers should be looking for, including any article formatting guidelines, research method requirements, citation and data standards, etc. (a great way to cover this is by compiling a reviewer checklist as explained here)

Once you’ve sent reviewers invitations with upfront guidelines for reference, don’t be afraid to follow up to ensure they see your email. And, of course, when reviewers accept assignments, be sure to send them periodic due date reminders to help them stay on track. Here again, you can use peer review software like Scholastica to save time by setting up automated reminder emails and easy-to-edit reply templates.

In response to a call for input for this blog post, Susan Altman, Managing Editor of Global Environmental Politics out of MIT Press, advised: “Give them some information about the journal and reviewing process in case they are unfamiliar with it. Be polite and use an invitational tone. Send regular automated reminders, and if they don’t work, contact the potential reviewer separately to gently ask them to respond.”

Editor-in-Chief of the Journal of Asthma and Allergy, Amrita Dosanjh, M.D., also noted, “provide gentle reminders and ease of access to the reviewer portal.”

4. Consider using your editors for peer review

In addition to searching for external peer reviewers, one option to ensure your journal always has enough available referees is to make them a part of your editorial board. A great example of a journal that does this is Sociological Science. The journal has a large group of consulting editors that the core editorial team calls upon to peer review manuscripts regularly. Each consulting editor agrees to be available to complete a certain number of peer reviews a year. This arrangement works well for the journal because it ensures reviewers are always available. And it benefits the reviewing editors because they receive a title they can include on their CV.

Sociological Science still assigns external peer reviewers to each of its submissions. But by having consulting editors handle initial manuscript review, the lead editors are able to reduce the burden placed upon external reviewers. The journal asks outside reviewers to simply state whether or not they think a manuscript is fit for publication and note any apparent errors they see in the submission without having to write up lengthy comments.

5. Thank your peer reviewers early and often

Finally, a big part of building your journal’s reviewer database is maintaining the list of peer reviewers you already have. The main thing to focus on here is appreciation. Always bear in mind scholars are doing your journal an invaluable service by voluntarily reviewing submissions, so it’s essential to let your reviewers know that their work means a lot to your journal and the field.

Some ways to thank reviewers include:

• Sending thank you emails to reviewers after they complete assignments
• Publishing a list of all of the reviewers who’ve worked with your journal at the end of the year highlighting their service
• Giving reviewers access to free resources such as peer review best practices guides (many early-career scholars will thank you for this)
• Creating online certificates or awards systems to acknowledge reviewers
• Implementing external tools to give reviewers credit, like Web of Science Reviewer Recognition Service

Susan Altman, Managing Editor of Global Environmental Politics, said it well: “Thank them often! After a decision is made, send them a letter informing them of the decision made using their review.” She added that editors can also share all reviews of a submission with each reviewer anonymously to provide them valuable insight into how others viewed the paper and compiled their comments, which can serve as a helpful training tool.

Putting it all together

Once you’ve tried the above tips, go back and do them all over again! Building a reliable reviewer database requires constant effort. Your editorial team must commit to regularly searching for new referees, keeping your reviewer data clean, and prioritizing reviewer retention initiatives.

Always be on the lookout for innovative ways to find and appeal to potential peer reviewers down the line. If you stay ahead of the game, your journal will be in good shape.

Do you have additional tips for finding peer reviewers? We’d love to hear them! Please post in the comments below.