What I do when I review

I currently wear a number of hats in terms of academic publishing. I’m an author (and co-author). I am often asked to be a peer reviewer. I am the editor-in-chief of a small, but historical, regional journal. And I am an academic editor – some journals would call a similar position an associate or subject area editor – for PeerJ.

(As an aside, it should be noted that I do not get paid for any of these roles in any way. I, and other scientists, do this type of work because we feel that it is our responsibility to contribute to the process of scientific communication. Without proper and reputable avenues of communication, science would rapidly grind to a lurching halt as we each worked in isolation in our own little realm. The scientific endeavor has always relied on communication, and communication is only accomplished if it is also facilitated.)

Over the next while, in my sporadic blogging fashion, I plan to write down some of my thoughts about each of the roles that I mentioned above. Specially, I would like to write briefly about the mechanics, the philosophy, and perhaps some of the side issues that arise in each role. Today I’m beginning with the role of peer reviewer.

Peer review is the cornerstone of scientific communication. Not all of our communication as scientists is peer reviewed (e.g. most conference presentations or posters are not peer reviewed). But prior to results and analyses being entered into the permanent scientific record in the form of a journal paper, the work is reviewed by two or more independent referees.

The process of peer review has been fairly standard over the years. The reviewers receive the paper (or just the title and abstract) from a journal editor or associate editor. The potential reviewer has a preliminary look at it to see if they are able to review it. If they can and are willing to do the work, they read the manuscript carefully and provide a report back to the editor. Often the report comes with recommendations to accept as is (rare), accept with minor or major revisions, reject with an invitation to resubmit, or to outright reject. The journal editor compares the various reviews from the referees and arbitrates a final decision.

So, what do I do when I receive a request to review a paper? And how do I go about reviewing? Here are some of my thoughts in the form of do’s and don’t’s:

• Do consider reviewing a paper when a request is sent to you. We are all busy, and sometimes we are so busy that we need to set reviewing tasks aside for awhile. But that cannot and should not be a permanent reality for a practicing scientist. As long as I am submitting papers to journals and am relying on the good graces of volunteer referees, I should be willing to do the work on the other end of the equation as well.

• Do not review a paper if you are not qualified to do so. If the paper is out of your realm of expertise, don’t pretend that it is. Your review will not be helpful. On the other hand, there might be a reason that the editor asked you to do the review. Perhaps one portion of the paper is highly specialized and in your area. While you might not be qualified to comment on all aspects of the paper, you should be able to comment on that part. The editor should have outlined this to you in their request. But feel free to ask them if you find yourself puzzled by a request.

• Do not review a paper if you perceive a conflict of interest. Not every editor knows every possible collaborative or collegial arrangement. If you know or suspect that there is a conflict of interest, let the editor know the details and then let them decide on your eligibility to review.

• Do help the editor, even if you are not going to do the review. If you cannot review a paper for the reasons listed in #2 or #3, provide the editor with some alternate names of colleagues who are qualified to do the work.

• If you accept the assignment, do be cognizant of the deadline and do your level best to meet it. Nothing is worse from an author’s point-of-view than waiting for months to get the reviews back on their paper. Nothing, that is, except for being an editor having to poke recalcitrant reviewers while also fending off increasingly irate emails from authors.

• Do follow the journal’s specific reviewing guidelines. Not all journals are the same. All reputable journals expect scientific rigor and appropriate analysis. But there is variation beyond that (e.g., some journals look for “impact” while some do not).

• Do review as you would wish to be reviewed. In other words, follow the golden rule of reviewing. No author wants to hear the bad news that there are flaws in their analysis or reasoning. But neither does any author want to publish a flawed manuscript. So tell it like it is. Use your expertise. But – and this is an important “but” – be respectful. Whether you are recommending accepting or rejecting the paper, give constructive and useful feedback. Note the positive aspects of the work. Explain what you think could be improved in clear terms. Be as extensive in your comments as you need to be; but never be blunt and brief to the point of being insulting. If you have taken the job on, then do a good job. A good job always entails more than six lines of halfhearted text. Be helpful, be kind, and be honest. In short, be professional.

• Do be willing to re-review the paper if necessary. If you do end up rejecting the paper or recommending major revisions, let the editor know that you would be willing to have another look at the paper if or when the authors resubmit it. Since you now have some of the best knowledge of the state of the paper, you are among the best placed to assess the recommended changes.

• Do record your work in your CV. You are not being paid for the work, but it is part of your contribution to the scientific community. Immediately after finishing a reviewing task, record the task (not revealing the authors’ names or other identifying information, of course) in your CV so that you don’t forget about it.

• Do maintain confidentiality as expected. Most journals still use an anonymous (and sometimes double-blind) review system. That means that you are obliged not to reveal details of your review unless the authors and you both agree at some point. There are a few caveats to this. First, if you wish to sign your review, you can reveal yourself to the authors, but even then, you cannot discuss the details of the review with others. Some journals, such as PeerJ or F1000Research, encourage open peer review. If that’s the case and you choose to abide by that system, then the entire review process will be made public. But, even then, you must maintain confidentiality until the paper and the accompanying reviews are published. Again, be professional.

• Do look forward to reading the paper in the literature. If you have done a good job of reviewing it, you should take some degree of pride in the outcome because you have had a (hopefully) positive influence on the direction of science.

Peer reviewing takes time and effort, but it is also a rewarding experience. Besides allowing scientists another avenue for participation in the scientific process, it also exposes us to new ideas and cutting-edge thinking. And, above all, it ensures rigor in the scientific record. So enjoy the work, learn from it, and take pride in doing a good job.