Are you giving your journal’s peer reviewers the guidance they need?
The majority of academics find themselves diving into peer review headfirst with little to no formal training on how to write a good review, and limited insight into how the editorial process works or how publication criteria within their field may vary from journal to journal. The first time a scholar is invited to review for your journal, she will likely have many questions about your review process which, if left unanswered, can slow and potentially complicate peer review.
Rather than leaving new reviewers to try and figure out what is expected of them, your journal can help them get on track more quickly by providing reviewer resources. Here are 6 steps to help your journal give your reviewers the support they need:
1. Give your reviewers clear up-front guidelines
You can help your reviewers understand what will be expected of them when they accept a review invitation by providing them with reviewer guidelines. Your journal’s reviewer guidelines should:
- Outline reviewer best practices, including the amount of time reviews are expected to be completed in
- Explain your journal’s ethical policies, with specific steps reviewers should take in the event of a potential ethical dilemma
- Include a basic overview of your journal’s editorial process
- Explain criteria for publication including article formatting guidelines, research method and presentation requirements, citation and data standards etc. - so reviewers can quickly get a sense of what they will be expected to look for and address in their review.
- If you use a peer review system, you may also want to include information in your reviewer guidelines on where reviewers can find software FAQs and direct technical questions. Journal’s using Scholastica can find a complete Reviewer Guide in our Knowledge Base and direct reviewers to contact our customer support team members at support@scholasticahq.com if they ever have any software related questions.
You’ll want to make sure your journal’s reviewer guidelines are available to reviewers before they take on a manuscript assignment, so they are able to get a clear sense of what will be expected of them prior to committing to review. One of the easiest ways to make your reviewer guidelines readily available is to add them to a page on your journal’s website, which you can link to in your initial reviewer correspondences and which potential reviewers will be able to discover on their own.
A good example of a comprehensive set of reviewer guidelines can be found at PLOS ONE.
2. Create a checklist of what reviewers should look for
In addition to providing reviewers with guidelines to address general questions they may have, your editors can also create a checklist for reviewers to use as a sort of manuscript assessment rubric.
Reviewer checklists are often lists of questions that reviewers should ask themselves or points they should address while reviewing a manuscript and when completing their review feedback. Reviewer checklists generally include both broad inquiries (e.g. “Is this article on an original concept and does it display a high level of scholarship?”) and more specific questions (e.g. “Did the authors use a reasonable number of citations and are their citations organized to meet journal standards?”).
In some cases, a journal’s reviewer checklist can be encompassed in its reviewer decision template in the form of open answer or likert scale questions. If this is the case for your journal, be sure that your reviewers know to look at your journal’s reviewer decision template before and while reviewing their assigned article, so they can begin to think about and address review questions as they go rather than trying to answer them later on.
Here are two sample reviewer checklists:
- Journal of the Association of American Medical Colleges: Checklist of Review Criteria
- Oxford Journals: Checklist for Preparing and Evaluating Review Articles
3. Make sure your reviewer invitation emails include clear next steps
The more clearly you outline in your invitation emails the next steps that potential reviewers should take, the more confident reviewers will be in deciding whether or not they are a good fit for your article assignments. Unclear instructions on how to submit a review or on what the review should encompass can lead reviewers to decline your invitation to review – and that’s the last thing you want!
Make sure that each of your reviewer invitation email templates has all the information reviewers will need including a link to your journal’s reviewer guidelines, a place to include the abstract of the article you’d like them to review, and a review assignment deadline.
Also, be sure to have a call for alternative reviewer suggestions in your invitation email. In the event that the reviewer you are reaching out to is busy or feels they are not a good fit for the manuscript you sent them, giving them an opportunity to suggest alternative reviewers will make it easier for them to gracefully decline the invitation – and help you find someone else.
If the reviewer does accept your invitation, in addition to sending a copy of the article and your reviewer checklist to him or her, be sure to resend your journal’s reviewer guidelines. Your reviewer may forget to look back at your initial invitation email and miss out on the information in your reviewer guidelines if you only send them once.
4. Remind reviewers to be helpful and professional
The distance between reviewers and authors (not to mention the fact that reviewers are anonymous to authors) can occasionally cause reviewers to look past the fact that the author will eventually read their review and likely take it very personally. When reviewers forget this, some can end up framing their feedback in a harsh or unconstructive way. Be sure to remind your reviewers when they accept a manuscript assignment to maintain a professional tone and to focus on framing feedback in a way that will help the author improve upon their work, which should always be the ultimate goal of peer review.
5. Remember to thank your reviewers and solicit their feedback
Don’t forget to let your reviewers know that you appreciate them taking the time out of their schedules to review an article for your journal! Send reviewers a thank you email after you receive their review, and encourage them to send any feedback or questions they have for you post-review. Did they feel that they had all of the information they needed? Was any point of the editorial process confusing? Did they feel that you and your co-editors were available to them?
Gathering feedback from reviewers will help your editors continually improve upon your reviewer resources and ensure your reviewers are getting the support they need.
6. Give your reviewers feedback
In addition to showing reviewers appreciation, one of the best ways to develop a relationship with them and to make the review process more worthwhile for your reviewers over time is to provide reviewer feedback. You can either do this directly, by giving reviewers feedback on the tone, scope, and level of detail of their reviews (Does the review contain a good mix of overarching and specific feedback? Is the review too long or too short by conventional standards? etc.), or by sending your reviewers copies of the decision email you send the author for the article they worked on. Sending reviewers author decision emails will give them a chance to see how you handle author communication and to compare and contrast other reviewers’ comments with their own.
