Last month, I talked about some of the reasons that you might want to accept that invitation to review a review article. Let’s say you found my argument persuasive; now what? The internet (and even this very blog) is full of suggestions of how to peer review scientific research, but advice for how to review reviews is scarce. In fact, the only specific advice for reviewing review articles I could find was this single paragraph from Springer.
Yes, review articles are peer-reviewed!
Submitting to a Trends journal requires an affirmative from the editor in some form: either the editor reaches out to a potential author with an invitation to write an article, or the potential author submits a proposal for an article, which the editor chooses to invite for submission. It’s a common enough misconception that, because an editor has already given an article (or at least its topic and outline) the green light to submit, reviews don’t need to be reviewed and are simply accepted as a matter of course. That’s not the case at Trends—we need to verify a manuscript’s scientific accuracy and interest to the broader research community, among many other aspects—and it’s not the case at many other journals that publish reviews.
But I think the vastly different style and expectations of review articles in different journals make it tough to know what any journal in particular is looking for: the encyclopedic and exhaustive Annual Reviews, for instance, surely ask their reviewers to evaluate review articles differently from how we might at Trends. While this post is deliberately Trends-centric, I hope some of the advice is universally applicable to thinking about giving constructive feedback on any sort of review article.
What should I comment on?
We structure our reviewer comment form to walk you through the aspects of a potential article that we find most important. I’ve recently clarified the Trends reviewer guidelines to make them more useful and informative, and I’d like to describe my thought process in a little more detail here. (Even though you won’t see these questions if you’ve been asked to review a short article, they still make up a helpful framework to guide your thoughts.)
Balance: For Review articles, you’ll explicitly be asked if the article provides a balanced view of the topic. Here, we’re trying to ensure that all of the current applications, ideas, and hypotheses are properly accounted for. Something we’d really like to avoid is a review that’s glorified self-promotion: one that relies extensively on self-citation and essentially reiterates the authors’ own positions from their prior publications.
You won’t see this item for Opinion articles. On one hand, Opinion articles should push some agenda and be associated with a personal hypothesis. They’re not necessarily designed to give equal platforms to all of the ideas that are out there. Rather, they may be advancing ideas that are unorthodox or controversial. But on the other hand, authors of Opinions don’t have carte blanche latitude to speculate wildly or propose unrealistic paradigms.
What we’re asking you to do for Opinions is to determine whether the author’s proposal is sensible, given the existing literature on the topic, and whether competing hypotheses and the shortcomings of the proposed idea have at least been discussed. Sometimes, we get reviewer comments that recommend rejecting an Opinion article simply because the reviewer disagrees with the opinion that the authors have articulated. I personally don’t find this comment especially useful: opinions are meant to stimulate debate, and if everyone in the field already held the same viewpoint, then the article would hardly be worth publishing.
Scientific accuracy: This one is more straightforward. We’d like to know if the authors have interpreted and presented the relevant results correctly and if there are any recent and critical references that might be missing. Trends articles are meant to be concise and to focus rather narrowly on research from the past few years, so it is often impossible for an article to cite every single article ever published on the topic.
This is where feedback from our reviewers is especially important! Although we as editors try to stay informed on all of the topics that our journals cover, you’re the subject matter expert. If you think an author has misrepresented a particular finding, defined a term incorrectly, or relied on an obsolete proposed mechanism, you know that better than we do, so please let us know. Comments on important references to add or terminology to clarify are helpful too.
We’d also like to hear your thoughts on the manuscript’s figures. Are they easy to understand? Do they complement the message of the review?
Timeliness: Most of an article’s references should be to primary research from the past 2–5 years. Of course, this doesn’t mean that any research article from 2010 or introductory review should be ruthlessly stricken from the References section. But a review that mostly refers to research conducted a decade ago might suggest a topic that’s not of enough current interest for the journal’s readers to care about. And a review that just pieces together other reviews might be too far removed from the primary literature to provide a truly unique viewpoint.
Novelty: As I discussed last month, this is probably the single most characteristic aspect of Trends reviews. The manuscript you’re reviewing should say something different and give a reader information that he couldn’t find elsewhere. What makes a manuscript sufficiently novel to publish in a Trends journal? That might be as simple as “This is a topic that people care about now and hasn’t been comprehensively reviewed yet.” It could also be “Here’s a new application for a technology, or an update to a pathway based on new evidence.” A proposed set of best practices or unifying figure of merit for how to evaluate new results is certainly interesting as well.
A review that only lists experimental results without providing any synthesis, connection, or critique doesn’t give anything useful to its readers beyond a reasonably up-to-date bibliography. Similarly, a review that concludes simply by observing “additional research is needed” is not as interesting as one that posits which experiments should be conducted or suggests a path forward to commercialization or clinical translation.
Finally, if a similar review has been published lately, let us know. There may still be room for another review on the same topic if it has a different message or intended audience.
Authoritativeness: Strongly multidisciplinary articles are wonderful, but sometimes the authors’ primary expertise is in just one discipline. Let’s say you’re reviewing a review on an immunoassay that has emerged in the past couple of years, and you notice that the discussion of the applications is critical and comprehensive, but the basic immunology underlying the assay is limited and confused. This situation would be a great time to suggest to the editor that an additional author, with a different background from that of the original authors, could strengthen the manuscript.
Accessibility: We understand that, as an expert, you’re immersed in the field, but here, we’d like you to take a step back and imagine that you haven’t published dozens of papers on the topic. Would you still be able to understand the manuscript? We want you to comment on the suitability of the structure, the clarity of the take-home message (and the justification for writing the review in the first place), and whether abbreviations and acronyms are useful and standard or just confusing.
Is there anything I don’t need to comment on?
Every once in a while, I get comments from a reviewer that are one-third critique on the content and two-thirds suggestions for usage, grammar, and punctuation. If you missed your calling as a copyeditor and love commenting on these elements, great! It makes our jobs easier.
But I want to assure you that you shouldn’t feel obligated to comment on these aspects. Every Trends article gets independent editorial comments in a process that we call pre-revision. We focus in particular on the manuscript’s clarity and suitability for the journal’s audience, and we suggest passages for our authors to re-write or sections to re-structure. Then, once a manuscript is accepted, it’s professionally copyedited and typeset, which means that every comma will be in its right place upon publication.
Similarly, feel free not to worry about formatting, number of references, or word count—though if you think a section (or entire manuscript) is too long or short, that’s useful for us to know.
Finally, sometimes the paper that the authors wrote is not the one you would have written. We ask that you evaluate an article on its independent merit rather than comparing it against the arguments that you would have made or the canon of literature you would have chosen to review. (If your perspective diverges from that of the authors, I and many of the other Trends editors would be delighted to have you respond to the article in the form of a Letter.)
Putting it all together: What should I recommend?
You’ll be asked to provide a recommendation: accept, minor revision, major revision (without additional review), major revision and re-review, or reject. But after considering all of the key aspects I’ve described above, how do you know which one to pick?
I propose a three-pronged test. Imagine an average researcher who is interested in the areas covered by the journal that you are reviewing for but is not an expert in the specific topic of the review paper.
- First, can she understand the review?
- Second, does the review give her an accurate and current representation of the scientific topic?
- And third, will she learn something unique from this review that she wouldn’t learn by reading another article?
A review manuscript that survives this scrutiny is worth publishing, albeit possibly with some revisions to clarify the language, tidy up the structure, or incorporate additional references.
For even more clarity, you might think of it this way:
- Accept means that a manuscript meets those criteria as-is, and the only improvements that should be made are to the language or formatting;
- Minor revision means that a manuscript meets those criteria but could nevertheless be improved with better figures, additional discussion, a stronger message, or technical clarifications;
- Major revision means that a manuscript currently doesn’t meet the criteria but has a good chance to once the authors have had the chance to revise it;
- Major revision with re-review means that a manuscript doesn’t currently meet the criteria, but the topic is interesting enough that you’d like to give the authors the chance to come up with something that does;
- And reject means that the manuscript is so deeply flawed that it is unlikely to meet the criteria even after extensive revision.
I hope that these comments are useful—and that they’ve inspired you to consider reviewing manuscripts for the Trends journals. And if there are any other suggestions for reviewing review articles out there, we would love to hear your tips too.[“source=crosstalk”]