Analyzing the Data Citation Corpus on Redivis

DOI 10.60804/1516-e177

Blog post by Ian Mathews, CEO at Redivis, Zach Chandler, Director of Open Scholarship Strategy at Stanford University, Erin DeLaney, Head of Design at Redivis

Analyze data citations for any repository by forking this analytical workflow on Redivis!

At Redivis, we believe that data-driven research should be accessible, reproducible, and as open as possible. We also believe that there is a significant underinvestment in dataset curation across the academic enterprise. We argue that this is due to the reality that dataset curators too often go unacknowledged for their substantial efforts and contributions to our collective research endeavors. Redivis is a platform that provides researchers and universities with the tools to host, curate, and analyze data, and we are deeply committed to ensuring our community receives credit for their work.

We’ve been incredibly excited by the ongoing work at Make Data Count, and its development of the Data Citation Corpus as a central aggregate of citations to research datasets. We were enthusiastic to see the public release of the Data Citation Corpus in the spring, and the latest v2.0 release in August. This dataset is a major step forward in ensuring attribution and credit for dataset curators. So of course, the first thing we did was to load the Data Citation Corpus into Redivis so that we could explore it further!

Exploring the Data Citation Corpus

The Data Citation Corpus is distributed as 6 CSV or JSON files totalling approximately 1.5GB. Not huge by big data standards, but sufficiently large that we want to make sure to use the right tooling to explore the data. In our case, we loaded the CSVs into a single table of over 5 million records on Redivis, which automatically combined the files, inferred data types, and generated various summary statistics. 

We endeavor to make Redivis as open as possible – anyone with a browser can search for and explore public datasets, read documentation, and browse the data and summary statistics. Some actions, like running custom queries and uploading data, require a free account, available to anyone around the world. Explore the Data Citation Corpus table for yourself below!

We can see that there are 1,692 distinct repositories represented in these 5,256,114 citations, with the European Nucleotide Archive representing almost half of the citations. We also observe that most citations have occurred over the past several decades, though there are older citations in our database.

Once we’ve logged in, we can run custom queries to ask more nuanced questions of the Data Citation Corpus. We previously saw that the European Nucleotide Archive was the most cited repository – let’s explore where those citations are coming from. By running a quick query to filter our records and looking at the summary statistics for the “journal” variable, we can see that 9,998 distinct journals cited the European Nucleotide Archive’s datasets, with PLOS ONE at the top with over 100,000 citations.

The Institutional Use Case: Counting data citations at Stanford

So far, we’ve only scratched the surface of the analytical capabilities of Redivis. To dive in further, let’s use a real-world use case: Zach Chandler is the Director of Open Scholarship Strategy at Stanford, and wants to better understand how datasets from research groups at his institution are being cited in the literature. By using the Data Citation Corpus on Redivis, Zach can develop a more comprehensive picture of the research impact of Stanford’s datasets.

To do so, Zach creates a new analytical workflow, where he can reference and combine any dataset on Redivis as he builds out his analysis. These workflows are a performant, in-browser interface where researchers can transform data with SQL (or a SQL-generating GUI), and perform further analysis via Jupyter notebooks running Python, R, Stata, or SAS. Workflows are also fully reproducible and shareable, allowing researchers to contribute and build off of each others’ work.

Here, Zach starts with the Data Citation Corpus, and uses the point and click “transform” interface to filter down to citations of datasets in the OpenNeuro repository – a free and open platform for sharing neuroimaging data, born out of the Stanford Center for Reproducible Neuroscience. He then joins these citations with metadata from the DataCite public data file, a dataset that contains information about every dataset that has been issued a DOI through DataCite. This allows us to bring in metadata about the authors of the cited datasets, so that we can appropriately give them credit.

After a bit more data wrangling, Zach has created two tables: 1) a list of OpenNeuro citations, linked to the citing publication; and 2) a list of Open Neuro citations by author, which we can then use to link dataset citations to the individuals whose data curations made the final publication possible.

This all comes together in a PYTHON notebook where we can calculate some final counts and visualize our results:

Complete your own analysis of the Data Citation Corpus

The workflow that we’ve developed for this use case is public for anyone to view or copy for further exploration. The OpenNeuro DOI prefix is parameterized within the workflow – you can simply swap it out for any other DOI prefix and re-run the analysis to search for the citations of interest to you! 

Of course, counting citations is just one of many things we can do with the Data Citation Corpus – particularly when we link it to other datasets, such as the DataCite public data file. We’ve wondered how the completeness of DOI metadata might influence a dataset’s propensity to be cited, or what sort of variability there is between scientific disciplines in citing datasets. You could even use GPU-backed notebooks on Redivis to train machine learning models that assess metadata quality. 

We invite you to explore the Data Citation Corpus on Redivis yourself to better understand this dataset and the powerful information it contains. 

As a final note, we ask for everyone to remember how important persistent identifiers (PIDs) are for the research ecosystem. If ROR identifiers were more widely adopted, this kind of analysis in our example would be much easier to do in the future. Ditto for ORCID. As a community, let’s all work to use persistent identifiers, so that everyone can get credited for their important contributions that drive our collective knowledge forward.