Tag: journal

Posted on

What I do when I review

I currently wear a number of hats in terms of academic publishing. I’m an author (and co-author). I am often asked to be a peer reviewer. I am the editor-in-chief of a small, but historical, regional journal. And I am an academic editor – some journals would call a similar position an associate or subject area editor – for PeerJ.

(As an aside, it should be noted that I do not get paid for any of these roles in any way. I, and other scientists, do this type of work because we feel that it is our responsibility to contribute to the process of scientific communication. Without proper and reputable avenues of communication, science would rapidly grind to a lurching halt as we each worked in isolation in our own little realm. The scientific endeavor has always relied on communication, and communication is only accomplished if it is also facilitated.)

Over the next while, in my sporadic blogging fashion, I plan to write down some of my thoughts about each of the roles that I mentioned above. Specially, I would like to write briefly about the mechanics, the philosophy, and perhaps some of the side issues that arise in each role. Today I’m beginning with the role of peer reviewer.

Peer review is the cornerstone of scientific communication. Not all of our communication as scientists is peer reviewed (e.g. most conference presentations or posters are not peer reviewed). But prior to results and analyses being entered into the permanent scientific record in the form of a journal paper, the work is reviewed by two or more independent referees.

The process of peer review has been fairly standard over the years. The reviewers receive the paper (or just the title and abstract) from a journal editor or associate editor. The potential reviewer has a preliminary look at it to see if they are able to review it. If they can and are willing to do the work, they read the manuscript carefully and provide a report back to the editor. Often the report comes with recommendations to accept as is (rare), accept with minor or major revisions, reject with an invitation to resubmit, or to outright reject. The journal editor compares the various reviews from the referees and arbitrates a final decision.

So, what do I do when I receive a request to review a paper? And how do I go about reviewing? Here are some of my thoughts in the form of do’s and don’t’s:

  • Do consider reviewing a paper when a request is sent to you. We are all busy, and sometimes we are so busy that we need to set reviewing tasks aside for awhile. But that cannot and should not be a permanent reality for a practicing scientist. As long as I am submitting papers to journals and am relying on the good graces of volunteer referees, I should be willing to do the work on the other end of the equation as well.
  • Do not review a paper if you are not qualified to do so. If the paper is out of your realm of expertise, don’t pretend that it is. Your review will not be helpful. On the other hand, there might be a reason that the editor asked you to do the review. Perhaps one portion of the paper is highly specialized and in your area. While you might not be qualified to comment on all aspects of the paper, you should be able to comment on that part. The editor should have outlined this to you in their request. But feel free to ask them if you find yourself puzzled by a request.
  • Do not review a paper if you perceive a conflict of interest. Not every editor knows every possible collaborative or collegial arrangement. If you know or suspect that there is a conflict of interest, let the editor know the details and then let them decide on your eligibility to review.
  • Do help the editor, even if you are not going to do the review. If you cannot review a paper for the reasons listed in #2 or #3, provide the editor with some alternate names of colleagues who are qualified to do the work.
  • If you accept the assignment, do be cognizant of the deadline and do your level best to meet it. Nothing is worse from an author’s point-of-view than waiting for months to get the reviews back on their paper. Nothing, that is, except for being an editor having to poke recalcitrant reviewers while also fending off increasingly irate emails from authors.
  • Do follow the journal’s specific reviewing guidelines. Not all journals are the same. All reputable journals expect scientific rigor and appropriate analysis. But there is variation beyond that (e.g., some journals look for “impact” while some do not).
  • Do review as you would wish to be reviewed. In other words, follow the golden rule of reviewing. No author wants to hear the bad news that there are flaws in their analysis or reasoning. But neither does any author want to publish a flawed manuscript. So tell it like it is. Use your expertise. But – and this is an important “but” – be respectful. Whether you are recommending accepting or rejecting the paper, give constructive and useful feedback. Note the positive aspects of the work. Explain what you think could be improved in clear terms. Be as extensive in your comments as you need to be; but never be blunt and brief to the point of being insulting. If you have taken the job on, then do a good job. A good job always entails more than six lines of halfhearted text. Be helpful, be kind, and be honest. In short, be professional.
  • Do be willing to re-review the paper if necessary. If you do end up rejecting the paper or recommending major revisions, let the editor know that you would be willing to have another look at the paper if or when the authors resubmit it. Since you now have some of the best knowledge of the state of the paper, you are among the best placed to assess the recommended changes.
  • Do record your work in your CV. You are not being paid for the work, but it is part of your contribution to the scientific community. Immediately after finishing a reviewing task, record the task (not revealing the authors’ names or other identifying information, of course) in your CV so that you don’t forget about it.
  • Do maintain confidentiality as expected. Most journals still use an anonymous (and sometimes double-blind) review system. That means that you are obliged not to reveal details of your review unless the authors and you both agree at some point. There are a few caveats to this. First, if you wish to sign your review, you can reveal yourself to the authors, but even then, you cannot discuss the details of the review with others. Some journals, such as PeerJ or F1000Research, encourage open peer review. If that’s the case and you choose to abide by that system, then the entire review process will be made public. But, even then, you must maintain confidentiality until the paper and the accompanying reviews are published. Again, be professional.
  • Do look forward to reading the paper in the literature. If you have done a good job of reviewing it, you should take some degree of pride in the outcome because you have had a (hopefully) positive influence on the direction of science.

Peer reviewing takes time and effort, but it is also a rewarding experience. Besides allowing scientists another avenue for participation in the scientific process, it also exposes us to new ideas and cutting-edge thinking. And, above all, it ensures rigor in the scientific record. So enjoy the work, learn from it, and take pride in doing a good job.

Posted on

It’s cold out there!

Most of us would find it pretty hard to live outside all winter anywhere in Canada, let alone in places where temperatures routinely dip below -30ÂşC. But this is exactly what the mountain pine beetle (and many other insects) does. The question is, of course, how does it pull this off? What is it about mountain pine beetle larval physiology that allows the insects to make it through long months of deep cold?

A paper by Tiffany Bonnett and others, that recently came out of our lab, probes this process in pine beetles in a way that has not been done before. The publication is entitled “Global and comparative proteomic profiling of overwintering and developing mountain pine beetle, Dendroctonus ponderosae (Coleoptera: Curculionidae), larvae” and is available as an open access publication. We have also published the raw genomics data online at figshare. You can find those data here, here, and here.

 

What did we do?

Larval mountain pine beetles were collected from trees near to Valemount, BC during the early autumn and late autumn, and then again during the early spring and late spring. The larval beetles were prepared in the lab so that we could use a process called iTRAQ to assess all of the proteins present in the larvae at each of the different collection time points. Essentially we took four snapshots – two in the autumn and two in the spring – an then compared them to each other see what was changing. This gave us a huge amount of data to work with and we used statistics to tell us which proteins increased or decreased in prevalence across either the autumn or the spring.

 

What did we find?

Among other things:

  • Larvae expend a fair amount of energy on detoxification of host resin compounds, both in preparation for the winter, and then during feeding after winter is over.
  • Stress physiology plays a large role in this entire process, particularly in the autumn as the larvae are dealing with host tree resin toxins and readying themselves for the upcoming onset of winter.
  • We saw evidence for the involvement of several compounds that may play an antifreeze role.
  • There is an evident shift between emphasizing overwintering preparations (in the autumn) and emphasizing completing development (in the spring), consistent with expected shifting priorities at different points in the life cycle.

 

Why is this novel?

The overwintering larvae of the mountain pine beetle remain nestled under the protective bark of their host tree. This makes them quite difficult to work with, and until now not very much information had been generated on this life stage, particularly in the context of winter survival. This work, which has harnessed the power of some very useful genomics databases, has cracked the door (or the bark?) open to allow us to see in broad sweeping terms what is going on in this insect during this vital time in its life cycle. We have seen aspects of larval mountain pine beetle physiology that have never been seen before, and that provides the power to ask new questions and to investigate key genes and pathways in a much more directed manner.

 

Why is this important?

Up until now, the main known winter survival mechanism for larval mountain pine beetles was the accumulation of glycerol in the autumn. Glycerol acts as a natural antifreeze and is part of the overwintering survival tool kit of many insects. But in most known cases, glycerol is not the only part of the equation, and we didn’t think that it was the sole story in mountain pine beetle either. And it turns out that we were correct with that guess – there are a lot of other things going on as well.

In a larger sense, this means that we now have targets to focus on as we work to understand how deep winter cold can impact populations. Overwintering mortality is one of the major factors contributing to control of bark beetle populations. Now that the mountain pine beetle is moving from the cold interior of British Columbia into even-colder central Alberta, a major research question relates to the climate in its expanding geographical range and how that is going to affect the insect’s potential spread to other regions. Overlay that question with the impacts of climate change, and it should be apparent that understanding mountain pine beetle overwintering physiology is becoming more and more vital.

 

Where do we go from here?

We now have numerous potential gene targets to look at, any of which is a project unto itself. Because we have shown in other work that larval mountain pine beetles in the late summer are feeding on potentially very toxic food, we are interested in finding out how larval ability to detoxify and digest their food in the autumn can make or break their chances for winter survival. We suspect that certain larvae are better adapted than others at dealing with the nutritional challenges that they face, and thus better able to produce antifreeze compounds and the other components that allow overwintering success.

In other words, we suspect that there is variation in the mountain pine beetle population that results in some larvae surviving the winter while others don’t. We, along with collaborators, hope to determine which genes are important in this process and how selection pressure in their historical and expanding ranges are changing mountain pine beetle populations.

Some of our key questions are:

  • How do specific proteins function in protecting larvae from the cold?
  • What happens if we “knock out” some of those proteins?
  • What characteristics of tree defense and nutrition make some host trees more or less likely to allow the resident larvae to survive a winter?
  • Do adult beetle parents choose trees based in any way on how their young may fare?
  • Where in the genome should we expect to see natural selection as the insects move into colder and more inhospitable regions? How will these evolutionary shifts be observed in changes in behavior and physiology?
  • What are the larger implications of climate change on these processes?

As you can see – and as is the case with science in general – this paper not only provides some answers, but also provides fertile ground for more questions. This work, and other related work in our larger mountain pine beetle system genomics project, has given us the means to chase down some of the answers. We are looking forward to the interesting work ahead. Since this publication and its associated data are all open access, we also look forward to seeing what other people might find to do with our data.

ResearchBlogging.org
Tiffany R. Bonnett, Jeanne A. Robert, Caitlin Pitt, Jordie D. Fraser, Christopher I. Keeling, Jörg Bohlmann, Dezene P.W. Huber (2012). Global and comparative proteomic profiling of overwintering and developing mountain pine beetle, Dendroctonus ponderosae (Coleoptera: Curculionidae), larvae Insect Biochemistry and Molecular Biology DOI: 10.1016/j.ibmb.2012.08.003

Posted on

Whither peer review?

If you’ve been working in science long enough to have published at least one or two papers, you are already well-acquainted with certain aspects of the process:

  • Our current system of anonymous peer review has been a resounding success in terms of furthering the scientific endeavor.
  • Anonymous peer review has been around for a long time now and has carved itself a firm niche within academic culture.
  • A good reviewer is worth their weight in gold (or ink?). Their suggestions, even when graciously rejecting your article, can be used to strengthen the work for eventual publication.
  • Thankfully, most reviewers are good reviewers. Most take the time to carefully and thoughtfully train their lens of critical expertise on the submissions that they receive. In most cases, the eventual published products benefit from the (usually mainly unrewarded) referee’s effort.
  • A poor reviewer, on the other hand, is one of the most aggravating people that you will ever encounter. Poor reviewers take many forms. There are the ones that seem to have not read your paper in the first place and ask questions about things that are explicitly mentioned in your submission. There are those who seem to have an agenda, either scientific or otherwise, and who wear that agenda on their lab coat sleeve. And there are those who obviously don’t have the time or inclination to give a proper review and so either cursorily reject (usually) or accept your paper but who offer no helpful advice in their five-sentence paragraph to the editor. There is no real recourse for response; no real opportunity for dialogue. The review is the review is the review. Good, bad, ugly, or very ugly.
  • The system can be slow, not necessarily because of careful consideration by reviewers, but simply because a manuscript can sit for weeks or months on someone’s desk before they get reminded the seventeenth and final time by the journal editor to complete the review.
  • No one has ever received tenure or promotion on the basis of their careful and fair reviews of others’ articles. Conducting reviews is vital to the ongoing work of science,  but is a generally thankless job.

There are any number of peer review horror stories out there. Some of them are real. Some of them stem from the fact that nobody likes to get their work rejected. So it’s tempting to ascribe villainous motives to the anonymous reviewer(s) who stopped your article in its tracks. It is often hard to differentiate a legitimate beef from sour grapes.

Sir Winston Churchill is reputed to have said, “(i)t has been said that democracy is the worst form of government except all the others that have been tried.” And the same might be said for anonymous peer review. The fact of the matter is that peer review has served science well and continues to do so to this day. But that doesn’t mean that the current system is the pinnacle accomplishment of the scientific publishing process. Life evolves. Culture evolves. Technology evolves.

To stretch the evolutionary analogy, are we witnessing something akin to directional selective pressure on the anonymous peer review process? If so, where is the process being pushed? Are there better forms of reviewing that we have not yet tried because, until recently, our technology would not permit them? As technology changes, will peer review also change and become better – both for the scientists involved and for the furthering of our scientific knowledge in general?

Along with the recent discussion about more open science  and more “crowd” involvement in the process, we are also hearing some interesting ideas about changes to the review process. One such idea was recently presented by James Rosindell and Will Pearse at the PLoS Biologue blog:

Peer review is an essential part of science, but there are problems with the current system. Despite considerable effort on the part of reviewers and editors it remains difficult to obtain high quality, thoughtful and unbiased reviews, and reviewers are not sufficiently rewarded for their efforts. The process also takes a lot of time for editors, reviewers and authors.

And their solution:

We propose a new system for peer review. Submitted manuscripts are made immediately available online. Commissioned and/or voluntary reviews would appear online shortly afterwards. The agreement or disagreement of other interested scientists and reviewers are automatically tallied, so editors have a survey of general opinion, as well as full reviews, to inform their decisions.

…

In our proposed system, users would log into the system and get the opportunity to vote once for each article (or reviewers comment), thereby moving it up or down the rankings. Access could be restricted to those within the academic world or even within an appropriate discipline, so only appropriately qualified individuals could influence the rankings. The publication models of established journals would be preserved, as full publication of an article can still take place once the journal is satisfied with the scientific community’s reception of the work.

There are certainly attractive elements to this idea. First, of course, is the idea of online publication of what amounts to being a preprint. This gives the authors official priority and it gets the results out to the community as soon as possible. It also allows some semblance of “democratization” as the review process would no longer be a one-way street. And, of course, it forces reviewers to be responsible for their comments and decisions; the lack of such accountability being one of the biggest issues with the system of anonymous peer review.Referees would also receive explicit credit for their good, and not-so-good, reviews. A great reviewing track record may be the sort of thing that could actually be rewarded within the academy. There would be a real incentive to conduct good reviews.

However, I have concerns as well. Just as with “liking” on Facebook, this has the potential to become a popularity contest. And science is not about popularity. It is about truth. And truth can come from unpopular sources. There is also the likelihood that some researchers in highly competitive fields will only sign on to such a system with extreme reluctance due to the fear of being scooped.

Beyond that, would already overworked researchers really take quality time to thoughtfully comment on preprints? And, would there be ways to game the system, analogous to people trying to increase their search engine rankings? Finally, what about small and boutique journals? The authors of the new peer review proposal envision a marketplace where editors bid for articles within the ranking system. As the editor of a small, regional journal, I am worried about what would happen to journals like the one that I oversee. Would we be able to win bids for quality papers? Or would we get lost in the shuffle after over 100 years of service to the scientific community?

As with the shifts that are occurring with the move toward open access and away from impact factors, I am positive that peer review will also have to change. And it’s good to see that people are thinking about how those changes will come about. Hopefully some of the various concerns with the intended and unintended consequences of changing the system will also be thoughtfully considered. There’s nothing wrong with moving quickly as long as you apply the brakes appropriately around the corners.

A quick post script: It should be noted that the peer review process is not a monolithic edifice of utter similarity across the board. Some journals (e.g., BMJ) have been practicing open peer review for quite some time now. And some new journals (e.g. PeerJ) are also pushing into new territory on this front.