Monday, September 17, 2012

A modest proposal for reviewers

Peer review is a necessary evil in the life of every scientist.  On the one hand, pretty much every meaningful paper you ever publish will go to a bunch of peer reviewers, including the infamous Reviewer #3, who always suggests more experiments.  On the other hand, a significant chunk of your professional service will be reviewing on program committees, reading some good papers and a lot of others that are painful messes where the authors clearly need to do more experiments.  On the third hand, sometimes you'll find yourself wrangling reviewers yourself, and trying to get the damned procrastinators to actually turn in their reviews so you can let the authors know whether their paper is being blessed with publication or cursed with rejection or a request for more experiments.

Journal papers are particularly bad in this regard, since there's no particular schedule on which the paper has to be accepted or rejected, and sometimes a paper can languish for more than a year in limbo, unable to be cited or even submitted elsewhere.  And what's happening during that time? Well, from my own experiences wrangling reviewers, half the time the editor is waiting to see whether the reviewers will actually do the reviews they promised or not.  See, as a reviewer you get told, "We'd like to have you review this paper, and you've got six weeks to do it," or some similarly long time.

Six weeks?  No problem!  There's got to be a time in the next six weeks when you'll be able to read this paper... and then other projects and deadlines intervene, and the time slips away, and you end up at the end of six weeks trying to find a time to actually give the paper its fair shot.  I'm pretty faithful about turning in on time, but some people definitely aren't.  So when I'm acting as an editor or program chair, I spend a lot of time cajoling or tearing my hair and trying to get somebody else to review at the last moment when a reviewer fails.  And as an author waiting for a response, I'm always wondering whether the reviewers are doing anything or not...

So here's my modest proposal for fixing peer review timing: if the reviewers are going to review at the last moment, why not bring that last moment much closer?   Why don't we give reviewers only a single week, no matter how massive a paper they're going to review.  Then the process of negotiation back and forth can start much earlier and we can toss out the reviewers who aren't going to review much faster.  Everybody wins: authors get responses quickly, editors get their reviews back faster, and it will even lower the load on reviewers, since the editor no longer needs to recruit extra reviewers in case some fail.

So, dear reader, would you be in favor of such a fast-tracked world?

No comments: