Can “Responsible Conduct of Research” include publishing science via blogs?

For Open Access Week 2011, which starts today, I’ve made a video, a draft pledging website, an inspirational website, am giving a talk, and co-written a group letter. This post is about the letter.

As discussed in my last post, there’s a web-based course called “Responsible Conduct of Research” that many thousands of researchers are required to complete each year.  Brad Voytek spotted this question that seems a bit hostile (although quite possibly unintentionally) to new forms of scientific communication outside traditional journals.

I suggested we should write to the organization responsible for the course, and a few people commented on my post to indicate that they agreed. A few tweets later, we had a draft letter going. It’s been really cool to see how social media was able to quickly get a bunch of like-minded scientists together to achieve a goal. This in and of itself undermines the question that we wanted to question :)  Below see our letter- we emailed it to CITI and they responded promptly to thank us for the feedback and to say they’d consider the issues we raised.
———————————————————————————-
Dear Professor Braunschweiger (CITI co-founder) and Professor Ed Prentice (CITI Executive Advisory Committee chair):

We write to challenge the answer to one of the questions in the “Responsible Conduct of Research” online course. The question reads “A good alternative to the current peer review process would be web logs (BLOGS) where papers would be posted and reviewed by those who have an interest in the work”. The answer deemed correct by your system is “False” and the explanation provided includes the assertion that “It is likely that the peer review process will evolve to minimize bias and conflicts of interest”.

We question these claims for two reasons. First, we see real examples of rigorous science happening outside of the traditional system of journal-based peer review. Second, we believe that the future path of scholarly communication is uncertain, and indicating to young researchers that such an important issue is closed is both inaccurate and unhelpful to informed debate.

As an example of science that does not fit the mold suggested by the phrase “the current peer review process”, consider the use of the arXiv preprint server in certain areas of astronomy and physics. In these areas, researchers usually begin by posting their manuscripts to the arXiv server. They then receive comments by those who have an interest in the work. Some of those manuscripts subsequently are submitted to journals and undergo traditional peer review, but many working scientists stay abreast of their field chiefly by reading manuscripts in the arXiv before they are accepted by journals.

Even in areas that are more tightly bound to traditional journals, there are recent examples where both effective peer review of science [1] and science itself [2] have occurred primarily via blogs and other online platforms. In these cases, the online activity appears to have resulted in more rapid progress than would have been possible through the traditional system. A growing body of research suggests that scholars use social media in ways that reflect and produce serious scholarship [3][4][5].

As for the future path of the current mainstream peer review model, we believe it is speculation to say that “It is likely that the peer review process will evolve to minimize bias and conflicts of interest”. The current peer review process may be under considerable strain [6] and unfortunately there is little evidence that it significantly improves the quality of manuscripts [7]. This raises the possibility that big changes are required, not just modifications to reduce bias and conflicts of interest. Furthermore, the question presupposes that the future entity into which peer review will evolve does not involve blogging. No one can see the future clearly enough to make that assumption.

We encourage discussion of this important topic, and would be interested in the inclusion in your program of material that sparks such discussion. However, we believe a true/false question on this topic to be inappropriate, as it limits rather than promotes discussion. All of us wish to see the development and optimization of rigorous systems, both new and traditional, for scientific scholarship. Requiring young researchers to adopt a particular position on this controversial, multifaceted issue may hinder open discussion and future progress.

Sincerely,

Bradley Voytek, PhD, University of California, San Francisco Department of Neurology
Jason Snyder, PhD, National Institutes of Health, USA
Alex O. Holcombe, PhD, School of Psychology, University of Sydney, Australia
William G. Gunn, PhD, Mendeley, USA/UK
Matthew Todd, PhD, School of Chemistry, University of Sydney, Australia
Daniel Mietchen, PhD, Open Knowledge Foundation Germany
Jason Priem, School of Library and Information Science, University of North Carolina at Chapel Hill
Heather Piwowar, PhD, DataONE/NESCent, Canada
Todd Vision, PhD, Department of Biology, University of North Carolina at Chapel Hill
Cameron Neylon, PhD, Science and Technology Facilities Council, UK, Editor in Chief, Open Research Computation

[1] Online experimental peer review of the “Arsenic Life” paper that recently appeared in Science: http://rrresearch.fieldofscience.com/2010/12/arsenic-associated-bacteria-nasas.html
[2] Open Science is a Research Accelerator, M. Woelfle, P. Olliaro and M. H. Todd, Nature Chemistry 2011, 3, 745-748. http://www.nature.com/nchem/journal/v3/n10/full/nchem.1149.html
[3] Groth, P., & Gurney, T. (2010). Studying Scientific Discourse on the Web using Bibliometrics: A Chemistry Blogging Case Study. Presented at the WebSci10: Extending the Frontiers of Society On-Line, Raleigh, NC: US. Retrieved from http://journal.webscience.org/308/
[4] Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the 73rd ASIS&T Annual Meeting. Presented at the American Society for Information Science & Technology Annual Meeting, Pittsburgh PA, USA. doi:10.1002/meet.14504701201
[5] Weller, K., Dröge, E., & Puschmann, C. (2011). Citation Analysis in Twitter. Approaches for Defining and Measuring Information Flows within Tweets during Scientific Conferences. Proceedings of Making Sense of Microposts Workshop (# MSM2011). Co-located with Extended Semantic Web Conference, Crete, Greece.
[6] Smith R. Classical peer review: an empty gun. Breast Cancer Research 2010, 12(Suppl 4):S13 http://dx.doi.org/10.1186/bcr2742
[7] Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database of Systematic Reviews 2007, 2:MR000016. http://dx.doi.org/10.1002/14651858.MR000016.pub3

6 thoughts on “Can “Responsible Conduct of Research” include publishing science via blogs?

  1. Excellent letter to which I’d gladly add my name (if you want to add some late signatories).
    Dorothy Bishop, DPhil, University of Oxford, Dept of Experimental Psychology

  2. [...] irresponsible to suggest blogs could play a role in science. For the full story, see Alex’s original post. The text of the letter is posted [...]

  3. I would object more to the true/false bit. Blogs cannot “replace” the peer-review system, but can AND SHOULD supplement it. (In fact, “supplement” is probably the wrong word. A 50/50 mix would probably be just fine and you can’t really call something that substantial a supplement any more.)

    Textbooks are no more “peer-reviewed” than blogs, are just as prone to error (perhaps more so, as you can comment on blogs and thus the readers can provide live feedback and instant corrections), and are actually worse for people if they ARE in error since textbooks are used in educating others – blogs, generally not so. Yet I’m not seeing a call for a ban on textbook writing.

    Should textbooks be banned? No, obviously not, though they should be scrutinized more and school boards should be prohibited from requiring errors as policy. Blogs should, by the same rule, be scrutinized as far as possible and there should be a way to “mark” particularly good blogs and bloggers as being such. (An academic “Web of Trust”, as it were. Academia essentially does this with published papers, by means of the citation index. This merely extends the idea to uncited forms not published in recognized formats.)

  4. Good letter,
    Peer-review depends on the type of paper. In many subjects the referees are not easily able to assess all aspects (and in some cases they are simply wrong or lazy or biased). In my field (chemistry) there have been several cases where the blogosphere has been concerned about the conclusion and they have actually repeated (some/all) of the work showing it to be suspect (a generous term).

    Yes, I’d sign a second letter

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s