Tag: July 2023

Evolving research assessment: DORA after 10 years

The San Francisco Declaration on Research Assessment (DORA) was published in 2013 with 155 individual and 82 organizational signatories, and over the ensuing decade recognition has grown that journal impact and other metrics are insufficient proxies for research quality. “From declaration to global initiative: a decade of DORA” looks back at the conditions that bred the original declaration: the over- and inappropriate emphasis on journal prestige which devalued scholarly contributions aligned with rigor and reproducibility, local community engagement, mentorship, intellectual property and software, and recognition of non-article works from different constituencies.

Over the years, the DORA coalition grew, formalized and became more action-oriented. Publisher funding enabled the organization to create a website and hire a program director. It sponsored forums where publishers, research institutes and funders could develop alternative, more holistic evaluation guides and tools, such as Balanced, broad, responsible: A practical guide for research evaluators, a case study repository “Reimagining academic assessment: stories of innovation and change,” and the forthcoming  Tools to Advance Research Assessment (TARA), a resource to develop policies and practices for academic career assessment.

From The intersections between DORA, open scholarship, and equity

DORA has made explicit the inextricable connections between research assessment, open scholarship, and equity & inclusion. It has altered its own governance structure to be more inclusive of organizations and practices in the Global South. It recognizes the work of other organizations and initiatives working to change research assessment across systems and regions across the globe. Power dynamics that maintain old, embedded practices are strong. As it looks forward, DORA remains focused on increasing awareness of the negative consequences of restrictive research assessment, developing alternatives to reform it, supporting advocates for ethical and equitable assessment, and securing funding to achieve its mission. DORA has partnered to transform research assessment so that it is more open, inclusive, holistic and fair. When this becomes the new norm, everyone will benefit from improved research integrity, breadth and impact.

Note: The Libraries provide a Research Impact Indicators & Metrics guide which includes an “Apply Metrics Responsibly” page.

Have academic journals outlasted their utility?

A group of authors based in Europe recently argued in “Replacing Academic Journals,” an article in Royal Society Open Science, that scholars have been calling out problems with journals as the primary means for communicating science for two decades and it’s now time for radical change. The authors describe the interlinked crises of replicability, affordability and functionality that have broken journals as a viable mode of sharing research. It takes more time at greater cost to publish unverifiable research locked into platforms controlled by private interests which are monetizing user data. The alternative they propose:

“…a decentralized (i.e. federated), resilient, evolvable network, based on open standards that allow seamlessly moving from one provider to another, under the governance of the scholarly community.”

They describe a federated scholarly information network comprised of a repository infrastructure provided by academic libraries which hold outputs (data, software, methods/protocols) and narratives (research articles, multimedia, policy advice). This information network relies on open standards and the active involvement of those producing the research. Solutions are in place and the technical advantages are many. The challenges lie in the nuances of exceptions to what should be openly available, procurement processes, market failures and reward structures. However, antidotes to these challenges currently exist.

The authors conclude that the time is ripe for journal replacement. Redirecting funding to community-owned networks and content is pivotal, as is enabling research that rewards quality of the work over prestige of the journal. Their arguments are not new, but they bring the evidence and rationale together in a strong call for change, now.

New toolkit for existing and start-up open access journals

The Open Access Journals Toolkit is a new resource developed collaboratively by the Open Access Scholarly Publishing Association (OASPA) and the Directory of Open Access Journals (DOAJ) to guide existing and new open access journal publishers in a volatile global scholarly communication landscape. Articles (with references) cover issues of funding, setup, peer review and quality assurance, running a journal in a local or regional language, software and technical infrastructure, persistent identifiers, licensing, recruiting staff and building an editorial board, and much more. The Toolkit is organized into 6 searchable sections:

  • Getting started
  • Running a journal
  • Indexing
  • Staffing
  • Policies
  • Infrastructure.

Checklists, a definition of open access with a grid of different funding models, a glossary, an FAQ and an “About” section round out the Toolkit. It is openly licensed with Creative Commons CC-BY 4.0 and can be downloaded and printed as a whole or as individual articles. Versions in English and French are currently available, with other languages forthcoming. The Toolkit is intentionally designed for accessibility and use by scholarly publishers working in different contexts and regions across the world. Because of its comprehensiveness, it serves those thinking about starting an OA journal, those considering how to better establish their journals, those who are concerned with their journal’s financial sustainability and those with challenges in specific areas. The Toolkit fills a much needed gap in the nuts and bolts of open access journal publishing.

A new framework for library publishing programs

The Library Publishing Coalition, with a mission to “extend  the impact and sustainability of library publishing and open scholarship by providing a professional forum for developing best practices and shared expertise,” published “An Ethical Framework for Library Publishing, Version 2.0” in May 2023. The second version is a significant departure from the first in that it moves from topics and recommendations, with external references, to a set of frames, statements and guidance. It is a radical approach that recognizes the convergence of two separate domains – libraries and publishing – and invites library publishers to explore and define their position, and align their policies and practices, in complex and evolving scholarly communication ecosystems. The Framework centers people within library publishing and its contexts.

“Library publishing is values-based” is the fundamental ethic from which the three other frames grow. Statements and guidance under this frame offer library publishers prompts to examine the contexts within which they operate and to explicitly define their values. “Library publishing is both librarianship and publishing” is the second frame; the statements and guidance here encourage library publishers to interrogate their roles, power and practices as they emanate from both traditions. “Library publishing is community-oriented,” the third frame with its statements and guidance, challenges library publishers to grapple with their agency as it relates to communities and individuals within them. The fourth frame, “Library publishing is dynamic,” calls out the unique attributes of library publishers and their opportunities to create positive change. The frames, statements and guidance sit in relationship to each other, each unique and together a blueprint.

Note below: In 2014 UMass Amherst was one of 27 founding institutions, with Educopia, of the Library Publishing Coalition. The 2023 Library Publishing Directory includes 159 libraries across the globe, up from 125 in 2018. At this time of building momentum for open science (see UNESCO Recommendation on Open Science and U.S. Federal Agencies’ 2023 Year of Open Science), library publishers have a singular opportunity to become significant open, transparent, accessible actors in scholarly discourse for public good.

Note below – 2: I served on the task force that authored this Framework.