Alex Holcombe's blog

open science, open access, meta-science, perception, neuroscience, …

Everything’s fine with peer review- if there are any flaws, they’ll be taken care of by evolution?

with 9 comments

Bradley Voytek spotted a disturbing question in an official “Responsible Conduct of Research” training program:

This defense of the status quo has no place in a “Responsible Conduct of Research” training program. It reads like the old guard self-interestedly maintaining the current system by foisting unjustified beliefs onto young researchers!

The part that bothers me the most is the sentence “It is likely that the peer review process will evolve to minimize bias and conflicts of interest”. What is the evidence for this?

Has the process been evolving to minimize bias and conflicts, or to increase them? I don’t think the answer is very clear. As counterweight to the official optimistic opinion, here are a few corrupting influences:

  • Pharmaceutical companies continue to buy influence with medical journals, by buying hundreds of copies of journal issues that run studies that support their products.
  • Pharmaceutical companies continue to ghostwrite journal articles for doctors, to plant their views in the medical literature.
  • Scientists of every stripe often fail to disclose their conflicts of interest.
  • Journals develop new revenue streams, like fast-tracking articles for a fee, that may open them to favoring the select authors who pay.
  • Many reviewers are, like most humans, biased towards their own self-interest. This can yield a bias to recommend rejection of papers by rivals. Because reviewers in most journals are anonymous, they are never held to account.
  • Journals don’t have the resources to investigate authors accused of fraud, and universities often try to avoid finding fault with the researchers they employ.

Many people have suggested partial remedies to these problems, but it’s an uphill battle to implement them, due to the slow pace of change in the journal system. We have to remember this and not be lulled into complacency by the propaganda seen in that training program. It was created by an organization of academics called CITI.

UPDATE: In the comments below, Jason Snyder pointed out an article from CITI in which CITI reports that over 6,000 researchers a month are taking this course — being subjected to this biased question. Some of us object not only to their characterization of the peer review process, but also to their suggestion that blogs are not a good place to do science. We don’t want thousands of researchers to continue to be forced to assent to the conservative opinion articulated by CITI, so we’re drafting a letter asking them to delete the question.

About these ads

Written by alexholcombe

October 10, 2011 at 8:54 pm

Posted in academia, science 2.0

9 Responses

Subscribe to comments with RSS.

  1. Thoroughly bizarre. “Correct answer: false”. No, the correct answer is: let’s discuss it.


    October 10, 2011 at 9:15 pm

  2. Exactly @mattoddchem! Alex, this is a great breakdown of how wrong this thinking is.

    Bradley Voytek

    October 11, 2011 at 6:33 am

  3. Truly strange. Have any other Pis noticed this in a training session?

    mrgunn (@mrgunn)

    October 11, 2011 at 6:42 am

  4. We should let CITI know about our concerns.


    October 11, 2011 at 6:50 am

  5. Well then, it’s ironic that CITI has published a study on how amazing they are:

    in a non-peer reviewed journal comprised solely of articles by companies:

    with the following disclaimer:

    Jason Snyder

    October 11, 2011 at 7:39 am

  6. I’d certainly sign a complaint. I support peer review, but agree with mattoddchem that this is not a question that can be answered true/false.

    Dorothy Bishop (@deevybee)

    October 12, 2011 at 5:10 pm

  7. [...] But Cox and Foreshaw are far from isolated in their opinion. A link from that article leads to an interesting story about a question in a course about “Responsible Conduct of Research”. The question was [...]

    Peer Review 2.0 « viXra log

    November 28, 2011 at 6:43 am

  8. I just came across this document that offers a much more measured approach to what young investigators should know about peer review, from Sense About Science:


    July 19, 2012 at 8:00 am

    • Thanks Todd, I guess it’s about time to follow up with CITI and see whether after assessing it, they decided to change the question.


      July 22, 2012 at 10:52 am

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 766 other followers