In search of quality: The scientific peer review process

Last Updated : 01 April 2013

Table of contents

Have you ever wondered how the work of scientists is checked? When a new finding is published, how do we know it is sound?

Before a scientific assertion is made public it should be scrutinised for its credibility. Has the scientist drawn justifiable conclusions, based on the data available from sound scientific research?

The peer review process is a form of scientific quality control, where scientists open their research to the scrutiny of other experts in the field (peers).¹ By reviewing and criticising each others’ work, scientists aim to ensure that only original and sound research is published and recognised.

How does it work?

When research is submitted for publication in a peer-reviewed journal, the journal invites several (usually two or more) independent experts to assess the credibility of the research.¹ These experts consider the scientific methods, results and conclusions presented by the authors, asking themselves, if the science is technically sound, if the interpretation is consistent with the data, and if it is new, important or ground-breaking.²

Reviewers usually remain anonymous, are not paid for their assessment, and should not have any conflicts of interest in relation to the research. If a paper does not meet the requirements, based on the peer reviews, the editor can either reject it or deem it acceptable subject to adequate changes, allowing authors to react and revise their paper.

Why is it important?

The peer review process checks that a paper explains clearly how the research was carried out, so that it can be reproduced by others. It also verifies that the methodology is appropriate for the specific field and set of objectives. Another crucial part of the review process is assessing the originality of new research and the accurate referencing of related published research, particularly if these contrast with the research at hand. The review is also useful for those whose work is being scrutinised; it allows them to fine-tune their manuscript before public release.² A manuscript is seldom accepted for publication without at least a minor revision.

The review process essentially strives to separate fact from speculation and personal opinion.² Peer-reviewed research is never beyond criticism however, and any conclusions drawn must be considered in the context of other studies. Ideally, experiments should be repeated to assess whether results can be reproduced; this is how findings are truly substantiated. The real validation, therefore, comes after publication.

Non-peer-reviewed research

Unfortunately, research results often find their way into the public domain without being peer reviewed, and are spread via newspapers, magazines, the internet, television and radio. They may be unpublished findings presented at press conferences, or published findings from a journal that does not use peer review. Even journals that do use peer review contain some non-peer-reviewed content, such as editorials and letters to the editor. Both scientists and journalists should understand the meaning and importance of peer review and clarify whether or not research they discuss has been peer-reviewed. There are potentially enormous costs to both science and society from the promotion of scientifically weak or flawed research findings.

An imperfect process

The peer review process does not protect against misconduct. It can identify mistakes, but relies on honesty and, as a result, can fail to recognise deliberately fraudulent research. Various organisations have produced integrity guidelines on good research practice aiming to reduce such occurrences.³ On the other side, financial or personal concerns may bias a reviewer’s professional judgement and objectivity. It is vital to consider in advance any factors, which could lead to bias.³ According to the European Science Foundation, preventing and managing such conflicts of interest is crucial in ensuring equity and integrity.³

Sometimes concerns are raised about the influence of the funding body on the design of the study, or the interpretation or reporting of the research outcomes. The peer review process gives credence to research, because the paper has been independently checked and critically evaluated, including the correct scientific interpretation of the results on the basis of other existing evidence – no matter who funded the research.²

Inevitably, there are variations in standards between journals. A journal’s “Impact Factor” reflects how often its papers are cited in other peer-reviewed journals, and gives some indication of importance of the journal in its field – the higher the number, the greater the impact or influence.

The process and culture of checking each other’s work is ongoing in the scientific world. Once a paper has been published, further criticism can be made by the scientific community via letters to the journal editor, discussions at conferences, or direct exchange with the research team behind the study in question. Authors can justify their findings and flaws uncovered can be corrected or retracted.^1,2 This is the nature of science; all work is open to critique by other scientists.