Community perspectives on data metrics and data evaluation

https://doi.org/10.60804/DBT4-CZ71

Make Data Count is a community hub for the development of metrics that can help us understand how data is used in research and policy activities. We reached out to Make Data Count community members for their views on data metrics and data evaluation. In this post, we share their perspectives on why understanding the use and reach of open data is important, and on their interest in Make Data Count.

Thank you to Ricardo Hartley Belmar, Universidad Central de Chile & Data Observatory Foundation; Barbora Bieliková, Slovak University of Technology; Francisco Silva-Garcés, Co-fundador Fundación Openlab Ecuador, Coordinador de la Red de Investigación de Conocimiento, Software y Hardware Libre, and Chokri Ben Romdhane, CNUDST-Ministry of High Education; for sharing their perspectives.

Why do you believe that understanding the use and reach of open data is important?

Ricardo Hartley Belmar:

In the context of open science, the full benefits of sharing data are only realized when specific structural frameworks are in place. It’s not enough to simply create repositories or assign DOIs or other persistent identifiers to datasets and expect them to magically reach various scientific and social communities: there must be knowledge and a clear strategy behind those efforts. That’s why understanding the use and reach of open data is crucial to ensure it truly serves its intended purpose. Only then can we evaluate how datasets are being reused, who is using them, and in what contexts they contribute to new research or practical applications. This insight allows for better decision making, directing data investments in ways that ensure their impact across diverse areas.

Barbora Bieliková:

From my point of view, understanding the use and reach of open data is important for advocacy about open science. I believe that scientists would be more willing to share their data if they understand how open data is used by others, what the permitted uses are for the data, and the expectations for crediting the authors.

Francisco Silva-Garcés:

We generate a vast amount of data every second from every existing device and every activity in our lives, often without even noticing it. There is data of all kinds and for all purposes, ranging from data that may not have significant relevance, to data that contributes to the commercial, industrial, and business world, as well as data that contributes to public policy, new discoveries to address societal challenges such as Long Covid, and artificial intelligence, which accelerates these discoveries and helps find solutions.

In my opinion, failing to recognize the importance of data not only halts our development as a species, as humanity, but actually sets us back. For data, in particular research data, to truly have societal impact, it must be open and free from legal, technical, and commercial barriers.

In this regard, only the condition of openness for research outputs, including research data, allows for transparency in the scientific process and enables us to understand, for example, how much and for what purposes datasets are being used, and who the creators or contributors of those datasets are. Based on this, we can establish evaluation mechanisms for recognition and incentives for contributors or their organizations, measure the impact of data use and reuse, assess collaboration stemming from data generation and use, and much more.

Chokri Ben Romdhane:

The data produced by a researcher during their research activities can be reused by others to advance their own research, enabling gains in time and productivity. In addition, making data available in an open platform with a license that allows reuse will increase their visibility and consequently their reusability. It is relevant for researchers and institutions to evaluate how their data outputs are used, and assess them within regional and global data production.  

Why are you interested in Make Data Count?

Ricardo Hartley Belmar:

Given the diverse range of stakeholders involved in the scientific system (from funders and research institutions to data managers and researchers), it can often be challenging to raise awareness about both the opportunities and risks associated with data publication systems. Make Data Count plays a key role in bridging that gap, helping us understand how data is used and valued across these different layers. It offers the flexibility to engage stakeholders at various levels, whether funders, evaluators, research institutions, researchers, or data management support teams. This multi-layered approach provides a more comprehensive understanding of how data contributes to scientific progress.

The potential of open data, including its contributions to society, lies in its ability to foster innovation and collaboration across multiple sectors. But just as society is diverse, there should also be diversity in assessing and appreciating the impact of open data—always grounded in evidence. Make Data Count helps create that evidence, ensuring that open data’s contributions are recognized and measured in ways that reflect its value to different communities within and beyond academia.

Barbora Bieliková:

Many researchers are still not eager to share their data. I hope that by sharing good practice examples with them, and showing them the benefits of sharing data, for example through additional citations, more of them will be willing to share their data.

Francisco Silva-Garcés:

There is a global consensus that the models for research evaluation need to change, including how academic and research institutions are evaluated, how research faculty are assessed, how incentives are defined, etc. To this end, new metrics are needed, and to achieve this, we require data that adhere to certain fundamental principles of transparency, diversity, among others — principles that only open data can provide.

In my opinion, metrics for evaluating and rewarding how research data is used and reused, and its impact, could make a significant contribution to changing those evaluation models. Make Data Count provides important infrastructure for understanding the use of open data within the ecosystem of open research information infrastructure that has emerged also in other areas, such as OpenAlex (publications), or the CWTS Leiden Ranking (research performance of universities).

Chokri Ben Romdhane:

The Make Data Count initiative offers a unique opportunity for the community of researchers in Tunisia and in the region, to promote their data and improve research metrics by including data as a valuable contribution in the evaluation process for research outputs. The quality of the implemented infrastructure and the broader adoption of open standards will encourage the local and regional community to contribute to this initiative and start a collective reflection process about data use and reuse.