Let's make our reviews open, starting now

Researchers spend a lot of time doing peer review, and by a lot we are talking about over 100 million hours per year (estimate for 2020 by Aczel et al.). It is a complex and time consuming process that is often presented as a pillar to science dissemination, because of its function to scrutinize research papers to check whether they contain any flaws, oversights, or they meet certain criteria for novelty or advance, before the article appears in a journal. And yet a lot of this happens in the backstage of the journal’s process, hidden from public view. This approach to peer review results in inefficiencies in the system, where the same paper can undergo multiple rounds of review by different reviewers at different journals following rejection by one journal - another estimate suggests that over 15 million hours per year are spent on these redundant reviews. Keeping the reviews hidden also makes it difficult to credit review activities, the review process may vary from one journal to another or even from one manuscript to another within the same journal, so what is it that we would be rewarding? Reviewers can claim they did a review for a journal, but beyond this statement of a report being completed, we know very little as to whether the review was useful, constructive, or it led to any changes in the manuscript.


This is a companion discussion topic for the original entry at https://upstream.force11.org/posts/leaving-the-black-box-behind-reviews-open-by-and-for-the-community