Home > AGW, Climate Change, Data, Global Warming, Omniclimate, Policy, Science, Skepticism > About Peer-reviewed Dogmas, or ‘Meet The Peeritarians’

About Peer-reviewed Dogmas, or ‘Meet The Peeritarians’

(this in response to yet another tired thread full of “but the findings of so-and-so have not been peer-reviewed!“)

I think I understand it now…it’s like a new religion…instead of the Pastafarians, we now have the… Peeritarians!

Those people can be recognized by their preferred way to communicate with anybody they disagree with:

Have your thoughts/proposals/findings/obvious-observations-nobody-in-their-right-mind-could-deny been peer-reviewed?

Sadly, there is no way to convince them to ask or say anything else.

If anything has not been peer-reviewed, Peeritarians will deny its very possibility of existence. Worse, if anything has been peer-reviewed it is then taken as their new dogma…because Peeritarians are characterized by being impervious to critical thinking upon reading peer-reviewed material.

Only hope is, the peer-review system will eventually publish something completely contradictory, thereby convincing to good Peeritarian to change his/her mind.

———————

In order to preserve their remaining sanity, everybody is strongly encouraged not to engage Peeritarians in discussions about hurricanes and global warming, or health and global warming, areas where there are peer-reviewed articles demonstrating pretty much everything and its opposite.

Advertisements
  1. manacker
    2009/09/26 at 10:02

    Here is a “must read” for anyone interested in how the peer review and publishing process works (or doesn’t work), by Dr. Rick Trebino of Georgia Tech. It is both hilariously funny and sad.

    But best of all is the section at the end on “how to fix the broken system”.

    This includes suggestions of how to address the problems of conflicts of interest, irresponsible reviewers and editors and correct the flawed the peer review process, itself.

    At one point the author considers a lawsuit briefly, but is advised against taking legal action.

    The “cleanup” will not automatically come from the inside, just as it is impossible to enforce legal prosecution of corruption if the police force and legal authorities themselves are part of this corruption.

    The best way for the author to succeed in bringing “truth” and “honesty” back to the system is through the “truth”, itself, making certain that this “truth”, as he has exposed it in this humorous (but dead serious) exposé, gets wide circulation, so that a thorough reform of the system can be forced due to outside pressure.

    Max

    • 2009/09/26 at 10:18

      had seen it on Climate Audit. The author makes about it too much of a romancing, he’s going to publish some collection of stories soon, I wouldn’t sell my soul on the truthfulness of each and every detail.

      The problem exists, but exaggerations in either direction will never help

  2. bill
    2009/09/23 at 20:53

    And what the peerarians are asserting is that if a paper has passed peer review, that amounts to (at least) the reviewer, the editor, quite pssibly the whole scientific consensus, saying, ‘this paper is good and correct’. That assertion is wrong and, I suspect, more often than not uttered knowingly as an untruth by those who make it, in order to deceive the gullible.

  3. bill
    2009/09/23 at 20:48

    Looking back, I see the comment above mine is on the same theme. But even Wegman seems to have not quite grasped what pper review does, talking about reviewers ‘fully vetting’ a paper. While ‘fully vet’ is open to interpretation – at one extreme it might mean replicating experimetal work, and I don’t know of any reviewer who does that (though I’ve heard of statistical analysis being re-run in the reviewing of medical papers, just because the financial consequences of publishing a paper syig ‘this drug works’ can be huge, and the moral consequences huge if the paper turns out to be baloney). Generally reviewers are asking low-hurdle questions, like, has this person clearly made the case he is claiming to make; after that, its down to stuff like has the author shown familiarity with latest work which could have a bearing; technical stuff, like whats the point of Figure 4c, unclear use of Englih in Section 3, why equations 14 and 15, and so on and so on. Primarily, a peer reviewer is answering the question, ‘Is this paper of a standard to be published’, not, ‘Is this paper good and correct’.

  4. bill
    2009/09/23 at 20:32

    and the peerarians have nothing at all to say when its pointed out to them that the paper which created the hockey stick, Mann et al, despite having been published in Nature which is supposed to have Super-A peer review standards, was absolutely demolished, notably by M&M. On their arguments, how could such a crap paper have got through peer-review?
    What they are generally trying to do is misled a non-expert audience as to the proper nature of peer review. Within that, they are unable to explain how Mann’s paper got published.
    Deconstructing the rhetoric, I supppose they are saying something of the order of, “We like papers published in journals we approve of, which present arguments and conclusions we agree with on political/policy grounds, evidence/facts notwithstanding.” Well, thats a version of peer review, but not as its generally understood.

  5. manacker
    2009/09/22 at 14:55

    Peer review is no guarantee of the validity or accuracy of a scientific report.

    A big potential problem with the peer review process is that of “social networks” and possible “tight coupling” among a small, like-minded group of specialists.

    This may be “ancient history” now, but was pointed out in 2006 by the Edward Wegman committee at the independent verification of the critiques of the Mann et al. “hockeystick” (1998, 1999) [MBH98/99] by McIntyre and McKitrick and the peer review process associated with MBH 98/99.

    To quote from the report:

    “In particular, if there is a tight relationship among the authors and there
    are not a large number of individuals engaged in a particular topic area, then one may suspect that the peer review process does not fully vet papers before they are published. Indeed, a common practice among associate editors for scholarly journals is to look in the list of references for a submitted paper to see who else is writing in a given area and thus who might legitimately be called on to provide knowledgeable peer review. Of course, if a given discipline area is small and the authors in the area are tightly coupled, then this process is likely to turn up very sympathetic referees. These referees may have coauthored other papers with a given author. They may believe they know that author’s other writings well enough that errors can continue to propagate and indeed be reinforced.”

    In the conclusions, the Wegman report stated:

    “In general we found the writing of MBH98 somewhat obscure and incomplete. The fact that MBH98 issued a further clarification in the form of a corrigendum published in Nature (Mann et al. 2004) suggests that these authors made errors and incomplete disclosures in the original version of the paper. This also suggests that the refereeing process [i.e. peer review] was not as thorough as it could have been.”

    It would appear that the peer review process could be made more effective (and the review itself more robust), if the peer review panel included non-like-minded (or even openly skeptical) individuals.

    Max

  6. Luke Warmer
    2009/09/21 at 13:39

    This link is worth a read to get an inside view of one of the institutional problems with peer-reviewed papers even once they are published:

    http://www.physics.gatech.edu/frog/How%20to%20Publish%20a%20Comment%20w%20suggestions.pdf

    (I think it came via another climate blog during the recent Morabito minimum!)

    The interplay and change of mind of the reviewers rings an accord with a recent experience of mine. (in a non-climate field).

    It’s sad that peer-reviewed is seen as the analog of “proven science”. It’s partly the way we teach hard science and the scientific method.

    The Golem by Collins & Pinch gives an example for solar neutrinos where “By 1978 over 400 papers had been published proposing ‘solutions’ to the solar-neutrino problem.” – Clearly all 400 were peer-reviewed.

    The other obvious paradox is that every scientific revolution or paradigm shift, invalidates (to a degree/incommensurability to one side) all ‘dead paradigm dependent’ previously peer-reviewed papers. You might argue that observations and raw data would still be valid but don’t overlook theory-laden observation. It’s complex but it’s real science.

    We have to recognise some kind of core science in contrast with frontier science for each discipline. The IPPC has trojan-horsed/shoe-horned AGW into the core when clearly there is so much dispute/debate that it has to be considered to be frontier.

    Glad you’re back on the case.

  7. willnitschke
    2009/09/21 at 09:41

    Among working scientists peer review simply means that the paper may not be complete BS–but it could still be–or if it does turn out to be true, it may still be irrelevant. It seems only to have acquired semi-religious status among non-scientists, in particular activists and the media.

    In a web forum a while back I expressed some skepticism towards a few wilder AGW claims, for which a poster replied with the demand that I back up my skepticism with peer reviewed papers. To this I obliged. The poster than replied with a link to an analysis done by an anonymous blogger by the name of “Tamino”. I asked this fellow what happened to his demand for peer reviewed links, but I never did get a reply. Nor was the poster prepared to give up on his Tamino analysis. You see, peer review is a double-edged sword. Many individuals are prepared to demand or quote peer reviewed papers when it agrees with their preconceived ideas, but are equally rejecting of peer reviewed work when it contradicts their belief system.

  8. PKD
    2009/09/21 at 02:57

    I see.

    So what you appear to be saying (and sorry if I get the wrong end of the stick) is there is nothing wrong with peer-reviews in themselves, but that once a theory passes pee-review it shouldn’t be the end of the questioning process?

    What then of papers and theory that clearly fail peer-review? Are you happy that those are discarded or made to go back to the author for re-evaluation?

    • 2009/09/21 at 08:14

      Peer-review as a concept is…peerless 😎 but I am sure everybody will concede that there’s lots of improvement still possible in the current peer-review system (starting from the dictatorial powers of Editors…).

      In my view, any article that passes peer-review (a) has “simply” accumulated a good deal of “goodwill points” in its quest to be accepted (by me, or anybody else) as relevant to whatever discourse the article is about (could be the lifestyle of horseshoe crabs, the whereabouts of Dark Matter, or CO2 sensitivity…it doesn’t matter). But those “peer-review points” on their own can never be enough to make the article relevant.

      Articles that have not (yet?) passed peer-review (b) have no such points.

      Articles that have failed peer-review (c) obviously have “negative goodwill” (*), and it might be a good idea for the authors to re-evaluate them (most of the time, what actually happens is that authors will ‘shop around’ until they find another journal where the article will magically pass peer-review…another flaw in the system).

      Still, if one has the time and the will to read articles of type (b) and (c), there might be precious gems of Truth hidden in there, no matter what peer-review has or has not said. Likewise, believe me, there’s lots of articles of type (a) that are unadulterated rubbish.

      Therefore, only critical thinking can save us from falling prey of what is not true. Perhaps that’s my scientific “belief system”.

      (*) I am talking in general terms. The details of rejection of individual articles will have to be taken into consideration too.

  9. PKD
    2009/09/20 at 11:45

    And so your alterantive to having the validity of scientific papers examined via the perr-review process is what exactly?

    • 2009/09/20 at 22:01

      The alternative to the blind-faith mindless acceptance of anything that has been published after peer-review is to have a healthy, critical approach to everything fellow human beings have to say, according to the strength of the arguments and evidence provided, and including what has been peer-reviewed but not published, and what has been published but not peer-reviewed.

      Peer-review publishing becomes then (as a matter of course) one of the building blocks of “the strength of the arguments and evidence provided”, still neither necessary nor sufficient.

      You might consider an example from politics…the fact that an election is democratic does not guarantee that the people elected will be democratic…blind faith in elections as cure-all for decades of dictatorship, as should be clear to all now, is a recipe for disasters.

  10. Douglas Hoyt
    2009/09/18 at 15:16

    If you look at peer reviewed papers of 100 years ago, many of them are obviously wrong. A large fraction of the papers now passing peer review will eventually be recognized as incorrect. This statement is more true for theoretical papers than it is for observational papers.

    • 2009/09/21 at 08:30

      Doug,
      Hundred years ago peer review as we know it today was almost nonexistent.
      Publications were transcripts of “readings” for the scientific societies (eg Phil. Trans. R. Soc), or manuscripts were simply accepted or rejected by the journal editor (eg Annalen der Physik). Einsteins work was also not peer reviewed.

  11. JamesG
    2009/09/18 at 14:36

    And of course when some contrarian views are finally allowed past the peer-review gatekeepers we are then loftily informed on blogs that the peer review process is flawed and that we need to have a yet stricter procedure to ensure ideological purity. After which the lumpen warmers repeat the mantra of their cult leaders that this paper has been debunked at the cult blog, as opposed to the traditional methods they were formerly so fond of. Always the double standards.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: