Peer review: The brilliant idea that is slowing science.

Eoin Griffin.

Peer review is the method used throughout the scientific world to critic and assess other scientists’ work. It is used as the method to determine which grants should be given and which papers should be published.  At its best, it can advance scientific knowledge and understanding through reliable and objective comments. At its malevolent it can hinder or even stop the publication of a body of scientific work. Ever since the first scientific journals were published the peer review system and other systems like it have and continue to Play, a major role within the scientific community. It, as a system, has an enormous amount of power and influence. With that in mind, is the peer review system good enough? This article will discuss two areas of concern I believe are detrimental to the respectable reputation of science and scientists. I hope to offer some ideas that I think could improve the peer review system.

If a scientist were to get a paper published in a renowned scientific journal such as nature, it could male their career. So, with such a prize at stake, not only for the future of science but the scientists themselves, it is extremely important that peer review is done as fairly as possible [1]. Peer review is by no means perfect and is constantly being compared to democracy: “A system full of problems but the least worst we have” [2].  It has many problems with possibly the principle being that of the bias of the reviewer. The peer review system is based on the trust and integrity of scientists to deliver reliable, disinterested and critical feedback on a scientific article or paper.  This ability to look past bias and evaluate the work as impartially as possible is why science has been able to progress. However, no matter how fair a reviewer believes they are, it’s inevitable that some preconceived ideas and notions are considered in context. I am not suggesting that all scientist’s display bias when refereeing a report, but having a vested interest in a certain theory or way of thinking will certainly change how objective they may be, as Peer review requires a certain level of knowledge and expertise in a particular field. Consider reviewing an article that completely goes against everything you believe in. Are you truly objective and impartial and without bias when reviewing? You may have rank, position and influence. Is it not plausible in this situation, the reviewer could stop the publication of the paper, which could potentially stop the next paradigm shifting breakthrough.

As we know there are many forms of bias; such as gender, affiliation and so on, as mentioned by Peters and Ceci a scientific paper should be “judged in the merit of [his/her]] ideas, not on the basis of academic, rank, sec, place of work” [3].  Nevertheless, when considering these other forms of bias, I believe that it is content bias that dominates when refereeing an article. However, it is important to say that peer review works, but any improvement in the process would be beneficial. Another big problem with peer review is the quality, the inconsistency and time it takes to complete a review. Since the reviews are done by scientists; with other more important commitments such as their own research, lectures to give and grant proposals to write. It means time allocated to peer review is small.  Consequently, the quality of the review can be poor. There have been numerous incidents where low quality science and completely fake results have been published. Such as at Bell labs [4]. If more time was utilised scrutinising articles for peer review, then less blatantly obvious errors would not slip through to publication, undermining science and scientists in the process. Additionally, scientists do not receive formal peer review training. This contributes to the large inconsistencies between reviewers [2]. Perhaps this problem is because everyone has an opinion and is inevitable, but even the quality of the feedback from can vary from person to person [5, 6]. However, a systematic approach to peer review would easily rectify the issue.

Potentially, there are numerous ways to improve peer review. It could be done by making all peer review systems double blind (author, review names and institutions are unknown) or have more scientists reviewing the papers, instead of the usual two or three. The problem with these ideas is that it would take far too long to review a paper. Even now sometimes it can take up to a year to review and publish a paper. In my opinion a little lateral thinking is needed. This can be done by creating a role inside the research departments of universities that is completely devoted to peer review. Training courses could then be setup to help improve the critical thinking needed, to introduce a standardised systematic approach to producing high quality consistent reviews and all should be aligned with the aim of improving communications between the science communities. I believe this approach has a number of advantages; it incentivises the scientists to concentrate on peer review as they will be paid to do so, unlike peer review at the moment which I done for free. It allows scientists the time needed to referee a paper. If the new position went post-doctorate graduates, then they will be less likely to be biased for or against certain theories and practises as they will have less preconceived ideas. Papers and articles will be reviewed more efficiently, and therefore, increase the number of high quality papers available to the scientific community. Not only this, but when the post-doctorates then go on to write their own articles, they will have the skills and will be more self-critical and evaluative of their own work. Again leading better quality scientific research. Surely a good thing!

A more radical approach would be to completely change the format of peer review. Make peer review completely open access. Thus allowing more scientist to review papers, leading to more consistent and less heavily biased opinions. For example, if a website, similar to that of reddit were to be created for scientists. Then they could upload their papers and get immediate feedback from a wide range of scientist giving a broader review. Not only will the scientist be able to comment but also a rank the paper. This would get a more accurate account of how good the paper actually is. Having more scientist comment and critique a paper is analogous to a scientist recording large data sets. The main reason to collect large data sets is that it reduces the amount of error in an experiment. Applying this to peer review would reduce the inconsistencies and reduce the chance of any errors going unnoticed. Plagiarism would be reduced too as all papers and ideas published will be on this website for all to see.  Other forms of bias could be reduced as well by making it double blind. This process will naturally sort out the great papers from the weak papers which will allow the journals to concentrate only on which papers they want, instead of wasting time on their own independent reviews.

There is no doubting the fact that for the vast majority of the time, peer review works. Getting rid of peer review, would cause a large amount of damage to science and the scientific process. After a scientific piece of work has passed peer review it receives a higher stamp of approval. It shows the public and other scientists that this piece of work is of a standard expected from scientists. However, when very poor quality papers are let through or extremely biased comments are made, this has the opposite effect and will be damaging, not only peer review and scientists, but to the way in which the public and government perceive science. The consequences of a decline in trust from the public and government would be disastrous, after all it is the tax payer’s money that pays for the majority of current scientific research.

[1] http://retractionwatch.com/2014/09/10/truly-extraordinary-simply-not-credible-suspiciously-sharp-a-stap-stem-cell-peer-review-report-revealed/

[2] Smith R. Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine. 2006;99(4):178-182

[3] Peters, D.P., and S.J. Ceci. 1982. “Peer-review practices of psychological journals: The fate of published articles, submitted again.” Behavioral and Brain Sciences no. 5 (02):187-195. doi:10.1017/S0140525X00011183.

[4] http://archiv.ethlife.ethz.ch/images/scientificfraud-1.jpg

[5] https://www.peerageofscience.org/review/review-examples

[6] https://www.timeshighereducation.com/features/the-worst-piece-of-peer-review-ive-ever-received

 

Peer review: The brilliant idea that is slowing science.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s