It is often said that the peer-review process of scientific articles is broken (1). The clamour is more in the field of biology and allied sciences. Retractions of some high profile (2) and not so-high profile papers (3) make a case for introspection on the whole peer review process. It is not that every other article is fraudulent or sloppy. Even then, individuals doing review of a manuscript must take utmost care to read, analyse, and vet upon the quality, and originality of a work. That's the prerequisite of the culture of peer-reviewed scientific literature.
But we, individual scientists and readers of published articles are supposed to be post-publication critics. Careful reading and analysis of evidences and arguments of a article are essential for further development of science. In fact, peer-review of manuscript by anonymous subject-experts was not always there in scientific publishing. Several path-breaking papers, like Einstein’s paper on relativity (1905), and Watson and Crick’s paper on the structure of DNA (1953), were published without pre-publication peer-review the way we know it today (4). Such papers stood the test of time as hundreds of readers repeatedly read and critically analysed those, post-publication. Are you maintaining that tradition of critical reading? Do we carefully scrutinize published data and the inferences drawn from those? Unfortunately not always. And it seems critical reading of articles are becoming old-fashioned in our culture. The time is over when one used to wait for issues of well-established journals and browse through every page of those, in the library. Number of journals, and the number of articles published every day has increased exponentially. Online publication of journals have increased accessibility, but seer increase in publication volume has reduced our attention span for articles. Though e-mail alerts, tweets, posts in Facebook help to keep track of path-breaking articles, detailed study of a paper, in leisure, is becoming rare. Most of the time, we put a paper under microscope, only when it challenges our own ideas or results or the paper is published by our competing research group. We put efforts to teach our students experimental techniques and data analysis. However, we rarely teach them, systematically, how to read and analyze published articles. Journal clubs organized in many institutions gives opportunity to learn critical reading. However, such occasional events are not enough to systematically inculcate the culture of critical reading of science.
Lack of emphasis on critical reading is evident in different forms. Many a time, a published article is cherry picked, and referred, without thorough evaluation, just because it helps one to substantiate a claim. Such practice is common in articles related to molecules involved in diseases like cancer. Authors often refer published articles to substantiate possible involvement of a gene in various types of cancers without giving adequate thought over the quality of data and inferences in those referred papers. And that cycle goes on from one publication to another. Erroneous use of back-references happens frequently in use of statistical and mathematical tools. Often such published tools are used inappropriately without giving thought about the assumptions and premises mentioned in the original article. Even erroneous work get referred, copied and used repeatedly to make newer claims.
It seems, we have culturally accepted that anything written in a peer-reviewed journal is khoser. Not just in literature, even in meetings, scholars often avoid putting up arguments and use back-references to justify their work. Back-references are essential as we do not want to reinvent wheel every day. But the culture of surrender to anything written in black-and white goes against the basic tenet of science.
Skepticism is at the heart of science. As Michael Shermer said, " Science begins with the null hypothesis, which assumes that the claim under investigation is not true until demonstrated otherwise" (5). This applies not just to statistical analysis of our data. This should be applied even when we read a published work. One should not accept the conclusions of a work just because it's published. Rather one should read the paper with skepticism, evaluate quality of data, analyze those and dissect the arguments provided by the authors. The data and logic, presented there, should eventually be able to clear all the apprehensions of the reader. Therefore, reading has to be dialectic.
Such critical reading enhances post-publication scrutiny. Traditionally, printed journals publish critics of published articles in form of letter to editor. Commenting on published work got easy with the advent of online publication. Various online forums has emerged to for post-publication scrutiny. PubMed Commons is a new addition to that. Critical review and online discussions have helped to discover several fraudulent publication. The recent controversy over STAP cells and subsequent retraction of the paper, is a hand-on example of success of post-publication critical review.
It is true that most of the papers will not get such extensive public scrutiny. Most of us are also not Internet-savvy and will not ever comment in public about a paper. However, we can focus on critical reading of papers in our field, particularly when we use those in our own research. Being individual we can take certain initiative to stop social acceptance and spread of sloppy or fraudulent papers. We can start with some basics:
a) Let us make sure that we refer to a paper in our manuscripts if and only if we have read that paper thoroughly and critically.
b) Let us avoid referring to review articles in our manuscripts and read the original research articles and refer those. This would avoid spread of any mistake or misinterpretation made by the author of the review article. At the same time the original research article would also get scrutinized once again.
c) Let introduce courses on writing and critical reading of scientific literature for our PhD students. Such course should introduce students to topics like data analysis, logic, and methods of drawing inferences. Through out their schools and colleges, most students learn to follow text books, like blind faith. Learning to question words, written in black-and-white, is not easy for them. We need to encourage them to do so and make it a habit than exception. Let us first make them skeptics, then scientists.
References:
