By Christopher Cotton, Abid Alam, and David Maslove, Queen’s University

Peer review is central to scientific publishing. Whether in economics, science, or medicine, peer review ensures research is carefully checked before reaching practitioners, policymakers, and the public. Yet, the system, which relies on experts volunteering their time, often struggles to secure enough qualified reviewers. As submissions increase and experts’ time becomes increasingly stretched, journals face delays, rushed or lower-quality reviews, and inconsistencies in evaluation.

Medicine experienced heightened challenges during the COVID-19 pandemic, as high submission volumes and limited expert availability put unprecedented strain on the system. Many researchers turned to pre-prints (papers published online without formal peer review) to speed dissemination, a practice historically less common in medical publishing.

Could paying peer reviewers help alleviate the reviewer shortage? We tested this question experimentally at Critical Care Medicine, a leading medical journal (see also the coverage of our work at Nature). Our study found modest improvements, suggesting that payment alone is insufficient to fully address the peer-review bottleneck.

Testing Paid Peer Review: A Field Experiment

In our six-month experiment, we sent nearly 720 invitations to potential reviewers. Half were offered $250 for completing a review, while the other half received no payment offer. We sought answers to three key questions:

1. Do financial incentives increase reviewers’ willingness to participate?
   • Payment increased review completion rates from about 42% to 50%. This was statistically significant but indicates that even with compensation, half of the invited experts declined to review.
2. Do payments speed up reviews?
   • Reviewers offered compensation submitted their reports approximately one day sooner (11 days vs. 12 days).
3. Do payments affect review quality?
   • We observed no difference in review quality between paid and unpaid reviewers, alleviating concerns that payment might compromise review integrity.

Would Paying Reviewers Solve the Referee Shortage?

While the effects of payment were positive, they remained modest. Paying reviewers on a large scale might, therefore, be economically challenging for many journals.

For perspective, consider a journal reviewing 1,000 submissions annually. With three reviewers per paper and a $250 payment per review, costs would reach $750,000 each year. For larger journals, the costs escalate quickly and become potentially unsustainable.

Furthermore, in medicine and many scientific fields, reviewer turnaround time is typically short relative to delays caused by editorial backlogs, decision-making processes, author revisions, and multiple rounds of resubmission. Thus, paying reviewers to shave off only one day from the overall timeline may not meaningfully accelerate publication within medicine, relative to fields (like economics) where turnaround tends to be many times longer.

What Can Be Done?

This suggests that referee payments, at least at the level we used, might not be enough to solve the reviewer shortage in medicine. What else can be done to improve the review process at medical journals?

Other potential solutions could include introducing modest submission fees to deter low-quality or long-shot submissions, with funds reinvested into editorial resources. Increased editorial staffing and clearly defined limits on referee revision requests could further streamline the peer review process.

Another promising option is AI-supported editorial processes. Automated screening tools powered by artificial intelligence can quickly identify and filter out submissions that do not meet basic criteria or standards. Such technology could also highlight required improvements for authors before formal peer review, thus streamlining both administrative and editorial workloads.

AI tools could further assist in matching papers with referees, aggregating referee comments, or summarizing manuscript strengths and weaknesses for editors, reducing the administrative burden and accelerating decision-making. Although AI cannot replace rigorous human evaluation, it could effectively complement human review and significantly reduce overall processing delays.

But, Maybe Referees Should Still Be Paid

Even if financial incentives alone cannot fully resolve the referee shortage or significantly reduce publication timelines, it remains worth reconsidering the ethical implications of a highly profitable publishing industry that relies on unpaid authors and peer-review labor. Commercial publishers generate substantial revenue from journal subscriptions and publishing fees, raising legitimate questions about why essential work such as refereeing is uncompensated. Authors and referees might reasonably expect fair compensation for their intellectual contributions, independent of whether such payments markedly improve journal efficiency.

Ultimately, while our findings suggest paying referees may not decisively resolve the peer review shortage, broader ethical and fairness considerations merit continued discussion regarding reviewer compensation. Meanwhile, combining strategic editorial policies with innovative technologies appears to offer the most practical path toward sustainably improving journal efficiency.