Failures of Peer Review, Part 2

I've been confronting a few failures of peer review recently, and I think it's time to revive peer reviewing posts. Like most failures of peer review, it may very well be the editors who dropped the ball and not the reviewers -- or a combination of the two.

1. I came across a 2008 paper with an incredibly low resolution figure that prevented me from reading the labels of the data points (and there were >100 labels). This wasn't a problem of my computer (the pdf looked just as bad -- even more pixelated and harder to resolve at higher magnifications -- with several viewers), didn't get better on the print out, wasn't available at a better resolution in the html version), I asked others to try their hand at reading it with no success, etc. Now, this wasn't a very important figure to read to get the conclusions of the paper, I admit... it's more that the figure had to exist to get the paper published. But it was important to me for a study I'm trying to start, and I had to spend several hours deciphering the figure. I know that the reviewers could not have been able to evaluate the data in the published version of the figure, so either the journal published a lower resolution version than they gave the reviewers or neither/none of the reviewers actually read/checked this figure.

2. Twice in the last month I've been asked by senior scientists what I think of a recent paper in our field. I happen to know the paper well, since I reviewed it and recommended rejection on a number of solid grounds. Even though I still think the study is flawed, I must have been in the minority, since the paper was accepted with only a few changes. However few people will read the paper thoroughly enough to see the newly introduced hedging statements; in this busy day and age, all most people will read or remember are the title, the abstract and the quality of the journal it was published in.

So I gave these senior scientists my detailed rationale for why one shouldn't trust the paper's conclusions, it was easy to convince them, and I doubt either one will be citing it anytime soon. But I can't shake the feeling of being a failure myself as a reviewer, since the paper did get into the literature virtually unchanged. Rereading the review, I was constructive in my criticism, but I clearly challenged the premises of the paper, and backed up my argument with references. I reiterated the cause for rejection in more frank language in the confidential comments to the editor. And maybe, with another editor, my comments would have caused the paper to be rejected... but the next time I feel strongly about a rejection I will be tempted to use more emphatic language.