To be meaningful, data metrics must be contextualized

Evidence and publications about data usage and data evaluation

Meaningful Data Counts project

The Meaningful Data Counts project, a collaboration between the Scholcomm lab and Make Data Count, researched data usage and data citation behaviors to better understand the role that open data play in scholarly communication. You can read about their findings in the publications below:

Survey of researcher practices, preferences, and motivations for data citation across disciplines:  Gregory et al. Tracing data: A survey investigating disciplinary differences in data citation.

Interviews with researchers across disciplines on how they want to be recognized and rewarded for data sharing and reuse, exploring also the role of data citations and other data metrics: Gregory et al. Rewarding data sharing and reuse: Initial results of an interview study.

Data policies and evaluation

Analysis of review, promotion, and tenure policies in 129 universities in the United States and Canada showing an emphasis on journal articles and monographs, with less common mention of datasets and databases: Alperin et al. The value of data and other non-traditional scholarly outputs in academic review, promotion, and tenure in Canada and the United States

An interview study with funding agencies highlights the need for better mechanisms to reward data sharing and management, as well as contributor models for data: Devriendt et al. Reward systems for cohort data sharing: An interview study with funding agencies.

Data usage trends and practices

Analysis of data references and data citations for datasets in the RADAR repository, using Google Scholar, DataCite Event Data and the Data Citation Corpus as sources: Strecker et al. How are research data referenced? The use case of the research data repository RADAR.

Study of over 8,000 social science papers that use multiple datasets reports that combining datasets – especially those rarely used together – is associated with a higher number of citations and broader reach: Yu & Romero. Does the use of unusual combinations of datasets contribute to greater scientific impact?

Data Citation Explorer, a methodology to identify citations to datasets from the Joint Genome Institute  in the literature: Byers et al. Identifying genomic data use with the Data Citation Explorer

Analysis of 12,000 data citations according to discipline, and country of the data producer and the researchers who use the data: Krause et al. Who Re-Uses Data? A Bibliometric Analysis of Dataset Citations.

Content analysis of citations to data papers published in Data in Brief: Li et al. Are data papers cited as research data? Preliminary analysis on interdisciplinary data paper citations.

Data citation and reuse practices for datasets indexed in the Global Biodiversity Information Facility (GBIF): Khan et al. Measuring the impact of biodiversity datasets: Data reuse, citations and altmetrics.

Survey of researchers’ data sharing and reuse practices across academic disciplines: Khan et al. Data sharing and reuse practices: disciplinary differences and improvements needed,.

Make Data Count activities and resources

• Perspective piece with a call to action for the prioritization of data metrics: ‘Building Trust: Data Metrics as a Focal Point for Responsible Data Stewardship’.
• ‘Recognizing Our Collective Responsibility in the Prioritization of Open Data Metrics.‘ Harvard Data Science Review, 4(3). Lowenberg, D. (2022).
• White paper from a workshop on open data that includes recommendations for different stakeholders on steps to advance data sharing and reuse practices: ‘Advancing data sharing and reuse: Insights from a workshop at the Researcher to Reader conference’.
• Book that discusses the development of data metrics, and the need for open, transparent, and community-led approaches to establish these metrics as the new standard: ‘Open Data Metrics: Lighting the Fire’.
• Make Data Count infographics translated into Spanish and French
• STM, DataCite, and Crossref Joint Statement on Research Data calling for the adoption of data citation best practices

Webinars & presentations

• Webinar ‘FORAGE: the hunt for existing data citations’
• Webinar ‘EXPLORE: the need for an open classification system’
• Webinar ‘BEGIN: Metadata for meaningful data metrics’
• Webinar ‘Principios FAIR y la implementación de flujos de trabajo para la gestión de datos en la investigación’ (In Spanish)

Make Data Count as a resource for research

The information that Make Data Count makes available to the community provides a valuable resource for research on data usage. Importantly, the data usage information that we provide aligns to our goals to ensure that measures of usage should be complete, transparent, and contextualized:

• We recognize that users interact with data in different ways and that those evaluating the use of data will also have a variety of needs and dimensions relevant to their assessment. In consideration to this, we advocate for the development of a set of indicators that provide diverse information about interactions with datasets, and caution against a single aggregate indicator or index that risks concealing important nuanced information, and introducing bias and/or ill behavior.
• All usage information is openly available under a CC0 license.
• Make Data Count usage measures are shared after their aggregation following relevant standards, such as relevant metadata schemas and the COUNTER Code of Practice for Research Data.

We encourage researchers to explore and use the usage information available via DataCite Event Data (Citations, Views and downloads) and the Data Citation Corpus data file.