imageA reader writes:

Dan, in the introduction to your blog, you state that the chemical analysis of the radiocarbon samples is “all nicely peer-reviewed in scientific journals.”  You go on to state that the chemical and physical properties of the images are also documented, “in peer-reviewed scientific journals.”

Such touting of the two words, “peer reviewed,” in a sense analogous to some sort of seal of approval by scientists, made good sense years ago.  Now, to the minds of scientists everywhere in every field, it sounds hollow, almost comically supercilious. Without knowing that the peer review was conducted by a reputable and ethical journal, the approbation is meaningless. Paperless and on-demand publishing, virtual internet offices, pay-per-click advertising and the concept called open access have created more bad journals than good ones. Anonymous peer review, which should be the ideal, can and often means two guys with laptops in a Tiki bar.

For a more complete picture you might want to read some of the material at Nature’s Peer Review Debates.

Here is the Table of Contents from those debates. In and of itself it is very interesting to read:

Nature’s trial of open peer review

Despite enthusiasm for the concept, open peer review was not widely popular, either among authors or by scientists invited to comment. by Sarah Greaves, Joanna Scott, Maxine Clarke, Linda Miller, Timo Hannay, Annette Thomas, Philip Campbell / doi:10.1038/nature05535  Full Text

Online frontiers of the peer-reviewed literature

The Internet is allowing much more interactive science publishing by Theodora Bloom / doi:10.1038/nature05030  Full Text

Trusting data’s quality

Database publication presents unique challenges for the peer reviewer by Brenda Riley / doi:10.1038/nature04993  Full Text

Opening up the process

A hybrid system of peer review by Erik Sandewall  /  doi:10.1038/nature04994  Full Text

An open, two-stage peer-review journal

The editors of Atmospheric Chemistry and Physics explain their journal’s approach. by Thomas Koop and Ulrich Pöschl  / doi:10.1038/nature04988  Full Text

Reviving a culture of scientific debate

Can ‘open peer review’ work for biologists? Biology Direct is hopeful. by Eugene Koonin, Laura Landweber, David Lipman and Ros Dignon / doi:10.1038/nature05005  Full Text

Quality and value
The true purpose of peer review 

1) What you can’t measure, you can’t manage: the need for quantitative indicators in peer review by Charles Jennings / doi:10.1038/nature05032  Full Text

2) Models of quality control for scientific research by Tom Jefferson / doi:10.1038/nature05031  Full Text

How can we get the best out of peer review?

A recipe for good peer review by Trish Groves / doi:10.1038/nature04995  Full Text

Statistics in peer review

Researchers need reviewers to check their stats. by David Ozonoff / oi:10.1038/nature04989  Full Text

How can we research peer review?

Improving the peer-review process relies on understanding its context and culture. by Joan E. Sieber  / doi:10.1038/nature05006  Full Text

Trust and reputation on the web

Online publications have several ways to give themselves a good name. by William Arms / doi:10.1038/nature05035  Full Text

Detecting misconduct

Does a digital workflow make it easier to detect ethical breeches in peer review? by Dale Benos / doi:10.1038/nature04996  Full Text

What is it for?

Analysing the purpose of peer review. by Elizabeth Wager /doi:10.1038/nature04990  Full Text

Increasing accountability

What authors, editors and reviewers should do to improve peer review. by Kirby Lee and Lisa Bero / doi:10.1038/nature05007  Full Text

Evolving peer review for the internet

Peer review needs to adapt to the pace and volume of information published online by Richard Akerman / doi:10.1038/nature04997  Full Text

Wisdom of the crowds

Scientific publishers should let their online readers become reviewers. by Chris Anderson / doi:10.1038/nature04992  Full Text

Certification in a digital era

What functions do we take for granted in print? by Herbert Van de Sompel / doi:10.1038/nature05008  Full Text

The case for group review

Peer review would be improved by discussions across the lab. by Debomoy Lahiri / doi:10.1038/nature05033  Full Text

Peer review of interdisciplinary scientific papers

Boundary-crossing research meets border patrol by Christopher Lee / doi:10.1038/nature05034  Full Text

‘I don�t know what to believe’

Understanding peer review is key to developing informed opinions about scientific research. by Tracey Brown / doi:10.1038/nature04998  Full Text

The pros and cons of open peer review

Should authors be told who their reviewers are? by Thomas DeCoursey / doi:10.1038/nature04991  Full Text

Does peer review mean the same to the public as it does to scientists?

Even reviewed literature can be cherry-picked to support any argument. by John Moore / doi:10.1038/nature05009  Full Text