Category: Peer review

Celebrating Peer Review Week 2023

Peer Review and the Future of Publishing

In the third week of September we celebrate the labor, role and practice of peer review to indicate some degree of research and scholarship integrity. Does a peer review label truly indicate a work’s validity, accuracy, quality and originality? Is peer review conducted as a gatekeeping tool (or signal) to benefit a publisher or to improve the quality of the work? Research integrity and trustworthiness are of critical concern in a marketplace where profits and careers are made upon the publication of scholarship. When these interests undermine works that have real and direct impact on life, the stakes are high.

Peer review has different meaning in different contexts and to different stakeholders. The new NISO Standard of Peer Review Terminology goes a long way to provide structure for the many ways peer review is conducted. The Libraries have published a new Peer Review guide to provide information on standards, best practices and training generally and within specific contexts. The guide aspires to:

  • demystify peer review,
  • encourage more scholars to conduct peer review and to receive recognition for their labor, and
  • explore ways to increase equity and inclusion in research and scholarship through its practice.

The challenges to peer review are many, from insufficient numbers of qualified reviewers for the proliferation of works to uncompensated/unrecognized labor, the evolving role of AI, inequities in participation, cost, etc. Several writers contribute their views of the single most pressing issue for the future of scholarly publishing. The topic is worthy of discussion and innovation as we all grapple with truth, integrity and inclusive participation in scholarship.

Paper mill journals delisted from Web of Science, DOAJ

Retraction Watch reported that Web of Science (WoS) delisted 19 journals and 1 special issue published by Hindawi, 10 of which were also delisted from the Directory of Open Access Journals (DOAJ). WoS-owner Clarivate withdrew 50 titles in the March update of journals it indexes after applying a new review process to detect journals that do not meet its 24 quality criteria. These include peer review, content relevance, citation practices and validity. Editors engaged with paper mills, organizations that charge authors to place uncorroborated or manipulated research in journals under their names, and did not put the papers through a peer review process. In the Scholarly Kitchen “Guest post – Of Special Issues and Journal Purges,” Christos Petrou discusses the publisher practice of using a scholar or group to edit an issue of papers on a specific topic which has fueled the growth of open access publishers Frontiers, Hindawi and MDPI and also exposed them to unethical editorial practices. However, the use of guest editors has increased across all journal publishers.

Wiley, which purchased Hindawi in 2021, temporarily suspended the publication of special issues between October 2022 and January 2023 and had retracted 500 papers in September 2022. As a result of Web of Science’s delisting of the 19 journals, their papers will no longer be indexed nor receive citation counts, and the journal will not be given an impact factor. These actions have negative consequences for authors of legitimate research in the journals whose work will be less discoverable and lack impact metrics they may count on for career advancement.

The 19 delisted Hindawi journals are:

  • Advances in Materials Science and Engineering
  • Biomed Research International
  • Computational and Mathematical Methods in Medicine
  • Computational Intelligence and Neuroscience
  • Contrast Media & Molecular Imaging
  • Disease Markers
  • Education Research International
  • Evidence-Based Complementary and Alternative Medicine
  • Journal of Environmental and Public Health
  • Journal of Healthcare Engineering
  • Journal of Nanomaterials
  • Journal of Oncology
  • Mathematical Problems in Engineering
  • Mobile Information Systems
  • Oxidative Medicine and Cellular Longevity
  • Scanning
  • Scientific Programming
  • Security and Communication Networks
  • Wireless Communications & Mobile Computing

The special issue was for Advances in Materials Science and Engineering.

eLife ends accept/reject decisions, expands open peer review to Brazilian-Portuguese preprints

eLife publishing process

eLife announced that as of January 2023 it will publish public peer reviews with preprints, ceasing rejection of manuscripts on the basis of closed peer review. Papers included in the journal will be Reviewed Preprints. Authors will have the opportunity to revise their preprints on the basis of the reviews. This is a natural extension of eLife’s “publish, review, curate” model through which they remove themselves as gatekeeper and enable experts in the disciplines to publicly assess research. This improves transparency, inclusion and speed of access to research. Quality of research and integrity of researcher communities are other trademarks of the journal. In a parallel development, eLife is reducing its submission fee from $3,000 to $2,000, with fee waivers granted to authors unable to pay.

eLife has a stated commitment to a diverse, global community of researchers sharing results for the benefit of all, and it is advancing this through its partnership with ASAPbio-SciELO to conduct open reviews of infectious disease research posted on the SciELO Preprints server in Brazilian Portuguese. This is eLife’s first non-English review group.

eLife is a non-profit, open access journal publisher of medical and health sciences research that receives financial support and strategic guidance from the Howard Hughes Medical Institute, the Knut and Alice Wallenberg Foundation, the Max Planck Society and Wellcome.

PRW 2022 – Research integrity: creating and supporting trust in research

PRW2022 logo

Following a participatory selection process, Peer Review Week has announced the 2022 theme, “Research Integrity: Creating and supporting trust in research.” In an interview on Scholarly Kitchen, the committee co-chairs Danielle Padula (Head of Marketing and Community Development at Scholastica) and Jayashree Rajagopalan (Senior Manager of Global Community Engagement at CACTUS) discuss the meaning of research integrity and the role peer review has to play. Padula defines research integrity as “conducting research in a transparent, rigorous, and ethical manner that can be verified to the full extent possible, enabling others to have confidence in the methods and findings.”

Peer review is the start of a verification process which is built upon by the works of other scholars. Trust in the peer review process depends on transparent standards, policies and practices, and it relies on editors, authors and reviewers working together to disseminate credible, vetted information. The people involved need to do their parts in ethical ways, with attention to potential bias and/or conflicts of interest. The co-chairs both emphasize that peer review is not perfect or the end game in verifying research, but it is a critical piece of the process.

The peer review crisis

Problems with peer review, and efforts to address those problems, have been regularly chronicled in this space. This month Inside Higher Education has expanded on and elevated the systemic problems in “The Peer-Review Crisis.” The latest problem is the shortage of available reviewers across all fields which has become especially acute in 2022. The number of papers submitted to journals has increased in Covid-related fields, and editors are unable to find people to review them. The issue is so significant that research publication is delayed and journal editors are resigning as a result of their higher workload. What was a fragile system prior to Covid is now teetering on collapse.

The article explores potential remedies, including paying fees for reviews, institutional recognition for peer reviewers, increasing the number of tenured faculty with professional service obligations, reducing or eliminating “revise and re-submit” practices and curbing the proliferation of journals. Some academics are refusing to provide review services to for-profit publishers without compensation. A couple of surveys suggest that money is less a motivator to incentivize grant proposal and journal article review than formal recognition of the work by institutions.

Consequences of the peer review crisis include but are not limited to delays in distribution of validated research and to the careers of scholars whose advancement depends on publications. Ultimately the problems stem from labor shortages and unfair labor practices. Ryan Cordell of the University of Illinois at Urbana-Champaign notes: “…the real answers are, like they are across labor sectors in 2022: hire more people, give them fairer contracts, reduce exploitative workloads…real solutions would require labor solidarity across academic tracks & ranks because everything else is a bandaid.”

Improving research and expanding open peer review

eLife, a “publish, review, curate” open access, non-profit life sciences and medicine publisher, and PREreview, a collaborative, open peer review platform, are furthering their collaboration to engage more scholars from communities around the world in preprint review. Together they’ve offered peer review training programs for early-career researchers in a joint project with AfricArXiv, Eider Africa and TCC Africa. Their goal is to improve and strengthen research globally by engaging multiple perspectives from traditionally marginalized communities in preprint review. Both eLife and PREreview operate open access and open source platforms, and they are integrating them with each other and with ORCID, so that any researcher with an ORCID ID can request and share reviews to any preprint with a digital object identifier (DOI). These are welcomed efforts to build more robust technical and human partnerships in service of open science.

Learning about racism in scholarly review

Anti-racist scholarly reviewing practices: a heuristic for editors, reviewers and authors (2021) sets its intention to be an anti-racist professional practice guide for its primary audience, allies/accomplices and anyone involved in scholarly publishing and lays out the ways that the review process can affirm existing racist policies and practices. These include gatekeeping, reviewer anonymity, opacity, invisible and disproportionate labor burdens, and organizations unsupportive of anti-racist practices. Stories from the perspectives of authors, editors, reviewers and tenure & promotion candidates illustrate problematic practices for Black, Indigenous and people of color (BIPOC).

The reviewing heuristic is a list of ways to check one’s bias and to offer suggestions that are helpful to BIPOC and multiply marginalized and underrepresented (MMU) people. These fall within citation practices, harmful publication processes, contingency planning, transparency, valuing the labor of reviewers, and editorial inclusivity. Examples include:

  • Reviewers resist requiring the existing canon be cited and recognize that some canonical work may be purposefully uncited because of oppressive and harmful actions taken by those authors.
  • Editors recognize problematic reviewers, resisting the use of scholarly reputation and other excuses as justification for racist review comments. Editors trust BIPOC authors who identify a review as racist.
  • Editors, reviewers, and authors proactively offer flexibility and generosity in times of personal and communal crisis.
  • Editors make a detailed description of publication processes and timelines publicly available.
  • Editors offer official letters to reviewers and editorial board members acknowledging their labor.
  • Editors and editorial boards create policies for reviewing and removing board members or reviewers whose reviewing practices are not aligned with the journal’s policies on inclusion and anti-racism.

Implementing the practices provided in this guide is a way for people in positions of privilege to engage in anti-racist practices and relieve some of the burdens of publishing and reviewing from those with BIPOC and MMU identities. The guide offers ways to signal your commitment to anti-racist reviewing practices, one of which is sharing the guide itself.

Taking credit for peer review, warts and all

Peer review is designed to determine scholarship’s validity and relevance to a particular publication through the process of subjecting it to the evaluation of experts in the same field. In its ideal form, peer reviewers check methodology, outcomes, analysis and writing clarity, and they offer helpful suggestions to improve scholarship for publication. However, its problems are legion: bias, fraud, unpaid labor, insufficient capacity, duration, private retaliation, etc.. In “Owning the peer review process: If we have to do this work, we should own it,” Charlotte Roh elaborates on these issues, and she provides examples of alternative approaches. The American Geophysical Union, In the Library with the Lead Pipe and up//root are intentionally diversifying their editorial boards, providing training for early career editors and reviewers, and attending to processes and timelines to make them more inclusive and supportive for reviewers, particularly women and Black, Indigenous and People of Color (BIPOC). up//root is paying authors and reviewers through a SPARC grant, a rarity in academic publishing in contrast to the estimated $1.5 billion of unpaid labor provided by U.S. based reviewers in 2020.

Unpaid labor is at the crux of Roh’s critique of a peer review ecosystem. It is built to give legitimacy to academic publishing and platforms that are dominated by for-profit providers driven to increase profits by expanding these publishing systems beyond Western culture. They do so by using scholars representing narrow demographics and by perpetuating models to sustain and build power and profits. In conclusion, Roh advocates for claiming the review processes and scholarly communication platforms to benefit scholarship produced by a wide swath of researcher, editor and reviewer identities that represent a range of models and types of research. While Roh does not explore here how scholars can reclaim scholarly communication ecosystems generally and peer review in particular, she does remind us to examine who benefits from reviewer labor and to give one’s labor where there is mutual benefit.

UMass Amherst authors benefit from Libraries’ investments in open access book publishers

The UMass Amherst Libraries now provide financial support for a number of open access book publishing programs that do not charge author publication fees. MIT Press Direct to Open (D2o), University of Michigan Press Fund to Mission (F2M) and Open Book Publishers each rely on slightly different funding models to make selected books open access upon publication, but all rely on academic libraries as a major source of financing. Authors follow traditional practices for manuscript submission, and the publishers coordinate peer review and provide editorial, production and marketing services.

The benefits to authors are many. They retain their copyrights to their works while more people the world over can use them at no cost. Jenny Adams, Associate Professor of English, published her book Medieval Women and Their Objects with University of Michigan Press in 2016. She raved about working with Michigan Publishing, and she appreciates the broader impact of open access publication. Adams said, “Now that it’s out there, more people have read it, more people have cited it, and more people–from high school honors students to my scholarly peers–know my work.”

Janice Irvine, Professor of Sociology, is also working with University of Michigan Press on her forthcoming book, Marginal People in Deviant Places: Ethnography, Difference and the Challenge to Scientific Racism. She noted both the editorial and production support she received:

Publishing my book on a digital, Open Access platform is an exciting opportunity, and the University of Michigan Press has been enormously supportive of this process. My editor made incisive comments on multiple drafts, learned a software program to help edit my videos, and generally fixed every problem that came up. The staff have been really responsive during the production process, all of which has been new to me because of the hyperlinks and other digital features.

Janice Irvine

In addition to the open access format which allows additional features, these publishers sell print editions of the titles through traditional channels. Emily West, Associate Professor of Communication, is writing about one of those online channels in her forthcoming book, Buy Now: How Amazon Branded Convenience and Normalized Monopoly. She says “I recognize and embrace the irony that readers will be able to access a book called Buy Now for free through The MIT Press’s Direct to Open Program, and I thank the University of Massachusetts Amherst Libraries for supporting this program.”

If you’re interested in publishing your book open access and would like more information about how the Libraries can help, contact Christine Turner, Scholarly Communication Librarian.

Reviewing open access in Canada in the wake of Covid-19

In a two part series on the Scholarly Kitchen blog, Leigh-Ann Butler, Shannon Cobb and Michael Donaldson, three authors representing Canadian national funding agencies and a non-for-profit scientific publisher, reflect on the impact of the Covid-19 pandemic on open access publishing in Canada. Their stated goal is for greater collaboration “to advance science for the public good” in a scholarly publishing ecosystem dominated by non-for-profit publishers and university presses.

In Part 1, the authors review the open access policies of three federal granting agencies – the Natural Sciences and Engineering Research Council of Canada (NSERC), the Social Sciences and Humanities Research Council (SSHRC) and Canadian Institutes of Health Research (CIHR) – the Fonds de recherche du Québec (FRQ – the province of Quebec’s research funding body) and the Université de Montréal, all of which established a requirement for funded research to be deposited in an open repository within 12 months of article publication before 2020. The Université de Montréal’s policy covers articles, book chapters and conference proceedings. FRQ has since amended its policy to eliminate the 12 month embargo in 2023. These open access policies complemented new initiatives to make Covid-19 and related research and data openly available. Many publishers eliminated their paywalls to research and data to encourage rapid and wide sharing of Covid research. However, some these policy changes were limited in scope and temporary. Questions about increasing compliance with the open access requirements and expanding participation in knowledge production continue. Canada’s First Nations communities Ownership, Control, Access, and Possession (OCAP)™ Principles is recognized as a positive example of legitimizing different perspectives and needs within research processes and outcomes. Much more must be done.

In Part II, the authors address three areas – the pre-publication stage, peer review and infrastructure – that need attention to make research more permanently and widely open. Researchers shared pre-prints at a much higher rate in the past year in an effort to quickly solve the problems associated with Covid-19. Sharing research earlier in the production lifecycle raised questions of validity and trust, and the traditional publication peer review process was challenged. Certainly the need for research review was highlighted, and openly sharing methodologies, data and analysis contributed to a faster and more widely participatory review process. Making research outputs available without fees to users has been recognized as only part of open scholarly exchange; the content must also have standard metadata, persistent identifiers (ORCiD & DOIs, for example), and indexing, and it must be delivered on open-by-default platforms. Without the infrastructure, research is not discoverable or useable. Financial stability is also needed to maintain these systems, and shrinking Covid-19 budgets threaten the organizations that have been supporting open scholarship.

The authors of this series on open access in Canada address goals, rationales and challenges to open scholarship which are familiar to readers everywhere. I hope they will follow-up with proposed solutions and actions taken to achieve the goal of advancing science for the public good.