Create a free Manufacturing.net account to continue

Peer Review Is Fatally Broken

That is how science should be done - in the open with all data and methods available for ANYONE to review. You never know who will have a missing piece of data or an insight that will cancel or compliment a published paper. Let anyone who wants to look and comment.

In my quasi-random walks through the net, I came across a site that reviews the current state of peer-reviewed science. It discusses the work of scientific journal creator Vitek Tracz.

"Nobody reads journals," says science publisher Vitek Tracz, who has made a fortune from journals. "People read papers." Tracz sees a grim future for what has been the mainstay of scientific communication, the peer-reviewed print journal. Within the next 10 years, he says, it will cease to exist.
This prophecy ought to carry weight. Over the past 3 decades, Tracz, chairman of a conglomerate called the Science Navigation Group, has helped transform the world of science publishing. His most notable creation to date may be BioMed Central, the first for-profit open-access publisher. The pioneering site, founded in 2000 in London, has grown into an empire with more than 250 biology and medicine journals in its stable.
BioMed Central earned Tracz a reputation as a visionary. "He's one of the most important publishers of the last decade," says Michael Eisen, a biologist at the University of California, Berkeley, and co-founder of the Public Library of Science (PLOS), a nonprofit open-access publisher that launched its first journal in 2003.

Now that you know a little about the man, how about a look at his take on peer review?

In another bold strike, Tracz is taking aim at science's life force: peer review. "Peer review is sick and collapsing under its own weight," he contends. The biggest problem, he says, is the anonymity granted to reviewers, who are often competing fiercely for priority with authors they are reviewing. "What would be their reason to do it quickly?" Tracz asks. "Why would they not steal" ideas or data?
Anonymous review, Tracz notes, is the primary reason why months pass between submission and publication of findings. "Delayed publishing is criminal; it's nonsensical," he says. "It's an artifact from an irrational, almost religious belief" in the peer-review system.
As an antidote, the heretic in January launched a new venture that has dispensed altogether with anonymous peer review: F1000Research, an online outlet for immediate scholarly publishing. "As soon as we receive a paper, we publish it," after a cursory quality check. Peer review happens after publication, and in the light of day. F1000Research selects referees, who post their names and affiliations alongside their critiques. Papers become like wikis, with reviewers and authors posting comments and revisions as the need arises.

We see a lot of this sort of thing going on at many open science sites. My current favorite example is The Cloud Radiative Effect (CRE). The paper is by an internet friend of mine Willis Eschenbach. At the top of the post he notes an error in his method (noticed by a commenter) that partially or completely invalidates his argument. Let me quote:

[UPDATE: An alert commenter, Ken Gregory, has pointed out that in addition to the temperature affecting the CRE, it is also affected by the changing solar radiation. He is correct that I did not control for this. SO ... I need to go off and re-think and then re-do the entire analysis. In the meantime, in the immortal words of RMN, my analysis below is no longer operative. Bad Willis, no cookies … but that’s the nature of science. Thanks, Ken, for pointing out my error. -w.]

That is how science should be done - in the open with all data and methods available for ANYONE to review. You never know who will have a missing piece of data or an insight that will cancel or compliment a published paper. Let anyone who wants to look and comment.

You can see Willis' latest on CRE at: The Cloud Radiative Effect, Take Two and Evidence that Clouds Actively Regulate the Temperature.

More in Operations