Failures of Peer Review, Part I

I think a lot about the state of peer review. I spend a substantial amount of time on peer reviews (and re-reviews, and re-re-reviews... it would save time to accept more papers), but that's just a symptom of my belief that without thorough, fair and constructive peer review, science doesn't work. Especially as more journals publish more papers, and as online-early articles are distributed even though they contain errors that will be corrected in proof... we're being saturated with scientific information. While Faculty of 1000 and other TV guides to the recent scientific literature can help one find the better articles in the sea of decent ones, rigorous peer review is the bedrock assumption underlying all of the articles in the literature. When we read a peer-reviewed, published article we assume that the authors are not trying to deceive us, and we assume that they correctly drew conclusions from their data. And, despite the fact that two or more scientists evaluated each of these articles prior to publication, we often find we disagree with their conclusions, or feel the published experiments were done incorrectly. The goal of peer review is to help papers go into the literature in the least flawed form possible, and as we have less and less time to devote to reading each published article, peer review becomes more important.

Or, we need to move to a wiki commenting format on each and every article, which seems like it will increase the time required to read each "paper" since one will have to read all the comments as well...

I could go on for far too long about peer review... but I was prompted to write about it today because a friend of mine sent me the best example of a peer review failure that I'd seen for a very long time.

In the reference list of von Szentpály, L. 2008. Atom-Based Thermochemistry: Predictions of the Sublimation Enthalpies of Group 12 Chalcogenides and the Formation Enthalpies of their Polonides. The Journal of Physical Chemistry A (published online 11/17/08), we find:

Reference 27: http://en.wikipedia.org/wiki/Polonium#Toxicity. Weight-for-weight, polonium is about 10^11 times more toxic than hydrocyanic acid, HCN. The element and its compounds have been found in tobacco and, particularly, cigarettes. If inhaled, polonium is deposited in the lungs, where its nucleus emits α-particles causing severest damage to the tissue. The amount of polonium taken in by a smoker is approximately equal to that taken in from all other sources.

The peer reviewers didn't insist on a more stable and authoritative reference to support this than wikipedia. Surely one sign of the impending apocalypse... but the bigger issue is that the article appears to have incorrectly cited wikipedia, at least for the information listed on that page today, 11/18/08 (the Toxicity section had not been edited since 9/5/08). The wikipedia page purports:

By mass, polonium-210 is around 250,000 times more toxic than hydrogen cyanide (the actual LD50 for 210Po is about 1 microgram for an 80 kg person (see below) compared with about 250 milligrams for hydrogen cyanide

I have to say that if I were peer reviewing an article that had a wikipedia page as a reference I would have checked it out, and likely would have noticed the discrepancy between 250,000 and 10^11 times. But, there has to be a more appropriate reference for 99.9999% of facts than wikipedia (communication and information science researchers studying wikis being one obvious exception).