Parallel Session 9 – Monitoring and Assessment of Open Research I
Moderator: TBC
Location: Room 2064
9.1) Acting Locally, Thinking Globally: Developing a National Monitor for Open Research
Presenter: Susan Reilly, Maynooth University, Ireland
Shortly after the release of the UNESCO Open Science Recommendation in 2021, Ireland launched its national action plan for open research 2022-2030. The plan set out coordinated actions developed to achieve a transition to open research at national level. The plan recognised that the transition to open research is a shared responsibility across the research system requiring coordinated action at both the local and national level.
The establishment of a national monitoring service for open access and open research more broadly was one of the priority actions in the national action plan. For the national action plan monitoring is not only an action amongst many but essential for evaluating the progress of the plan and a means to ensure that the transition to open research is fair and equitable.
In 2024 the pilot National Open Access Monitor for Ireland was launched. The process behind the development of this monitor embodies many of the principles contained in the UNESCO Draft Principles of Open Science Monitoring.
Considerable effort has gone into ensuring the relevance and transparency of the monitor. Research communities were consulted to agree a common definition of OA, prioritise what would be measured, and to develop use cases. The monitor is built on open data and infrastructure and continues to evolve based on stakeholder feedback.
The development of the monitor has highlighted further areas for targeted action e.g. the adoption of persistent identifiers, development of repositories to international standards and further policy development.
The next steps for the monitor are to work with research stakeholders to agree how the monitor should evolve to support the monitoring of open research in a way that 1)aligns with national efforts to reinforce research integrity and ensures that the current unhealthy monitoring system is not replace with another, 2) that enshrines open research principles and 3) continues to align with international best practice.
The development of the National Open Access Monitor in Ireland is an example of the draft UNESCO Principles of Open Science Monitoring in action. It also highlights the importance ensuring that local and national infrastructure is aligned with international standards and that meaningful monitoring can only be achieved by investing in continuous stakeholder engagement.
9.2) Cost-benefit of Open Research Infrastructures: Portuguese Repositories Network Case Study
Presenter: Pedro Principe, University of Minho, Portugal
Open Science is redesigning research and knowledge sharing, promoting scientific progress, innovation, transparency, and collaboration. Recognized in various policy frameworks and prioritized in the EU policy agenda since 2016, OS is integrated into several programs like Horizon Europe and EOSC. There is a need for more evidence on the economic impacts of OS and prove that the investment being made by funders and institutions is worthwhile, and which are the main gains – both financial and in terms of impact – from these practices.
This presentation aims to highlight the findings from the cost-benefit analysis of the Portuguese network of open access repositories RCAAP being developed withing the PathOS project activities. PathOS is a Horizon Europe project aiming to collect evidence of Open Science effects, by studying the impact pathways (context, resources, activities, outputs, outcomes and impacts), performing an extensive literature review, reviewing the causal effects, and doing a cost-benefit analysis of Open Science practices in selected case studies.
RCAAP (Repositório Científico de Acesso Aberto de Portugal), the Portugal Open Access Science Repository, is the central system to discover, locate and retrieve scientific content (including hundreds of thousands of publications, theses, conference proceedings, journal articles, and data) hosted in Portuguese institutional repositories. Its main objectives are to (i) increase the visibility, accessibility and dissemination or Portuguese research results; (ii) facilitate access to information regarding these outputs; and (iii) integrate Portugal in the range of international initiatives for Open Access (OA). It is operated at the central level by the University of Minho and by the Foundation for National Scientific Computing (FCCN) – the digital unit of the national research funding agency of Portugal (FCT).
The following figure shows the analysis of RCAAP in terms of inputs- activities- outputs- outcomes, as well as the foreseen short and long-term impacts, by type:
The cost-benefit analysis was performed with data collected via desk research (history, gradual development, uses and performance of the RCAAP project and its individual components); an online survey targeting RCAAP users; and interviews aimed to understand the patterns of uses of RCAAP (Portal and IR) for different classes of users, including researchers and administrative staff from the network of institutions operating the repositories, allowing in-depth discussions on the value and limits of RCAAP from different perspectives.
The scope of the analysis focused on the core of the RCAAP project, i.e., its network of IR and supporting infrastructures and services (RCAAP Portal, transversal and support services such as statistics, training and helpdesk). Following the CBA approach, the analysis was performed to compare the benefits obtained from the exploitation of RCAAP with its costs by using an incremental approach. This entailed a comparison between two different scenarios: the actual, real-world scenario with RCAAP that we can directly observe, against the counterfactual scenario where this project does not exist. This approach made it possible to pinpoint the ‘net’ benefits generated, mainly linked to the mutualisation of costs and to the greater visibility of Portuguese research allowed by RCAAP.
9.3) Where is the Money Going? A Study on the Evolution of Open Access Publishing Costs at EPFL
Presenters: Lorenza Salvatori and Lorenzo Di Sopra, EPFL, Switzerland
Since the Berlin Declaration on Open Access twenty years ago, the academic community has experienced a continuous push towards Open Access publishing.
Among the various models that have emerged over the years, the Gold route has become the dominant approach. This trend has been further accelerated by the implementation of transformative agreements. However, this shift has had a significant impact on the budgets of researchers, libraries, and institutions, raising serious concerns about the long-term sustainability and fairness of this model.
In the light of these developments, in 2021 the EPFL Library began a thorough investigation of publishing costs at the institutional level and conducted a comprehensive analysis of their evolution.
In this presentation, we aim to share with the conference audience both the methods and the results of this analysis. In particular, we will describe the multi-level and multi-source approach we adopted and the involvement of different stakeholders at the institutional level.
In collaboration with our institution’s financial services, we have estimated the so-called costs in the wild incurred by researchers for publishing Open Access. We also examined the evolution of library expenditure, tracing the shift from a traditional “read-only” contract framework to the current landscape, which includes a variety of contract types such as transformative, full Gold, and Diamond models. The results of these analyses have enabled us to establish a system (launched in 2024) to more accurately track costs in the wild, while also starting a conversation with researchers and senior management about the allocation of costs centrally managed by the library to support Open Access.
Communicating the results of our project was also a fundamental part of this work. In order to reach as wide a scientific audience as possible, we have developed several communication outputs. These include a visually appealing webpage on our institutional website to make the results more accessible, structured datasets deposited on Zenodo to promote transparency and reusability, and presentation slides to facilitate the sharing of our results in different settings.
In doing so, we hope to contribute to wider efforts to monitor and assess the dynamics and costs of Open Access, recognizing that greater awareness and collaboration are essential steps towards a more transparent, sustainable, and equitable scholarly publishing landscape.
For these reasons, by presenting our work to the conference audience, we hope to provide interesting insights, encourage further dialogue and collaboration, and contribute to the broader challenge of transforming Open Access publishing.