Crossref Journal Participation Reports

Ted Habermann, Metadata Game Changers

The Crossref Participation Reports interface (https://www.crossref.org/members/prep/{member}) first selects members, typically publishers, and then allows users to drill down to journals using a journal title. The participation report data are also available using the Crossref member or journal APIs, like https://api.crossref.org/journals/{issn}. In a recent blog post we explored Crossref Participation Reports for all Crossref members and introduced a new tool for displaying those reports and extending them by exploring connectivity instead of completeness. Here we go one layer deeper into the data to explore journal participation reports.

As a start we retrieved 173,612 journals and participation reports from Crossref using the journals endpoint. A number of these could not be included because they did not have ISSNs, had duplicate ISSNs, or no DOIs (Table 1). This left 137,827 journals for analysis. These data are in JSON and include four  items that are not displayed in the Crossref interface (Affil RORs, Funder RORs, Update pol. (Crossmark), and Descriptions). We included those items and calculated an average of all 14 measures to have a single number we could compare for each journal. The dataset includes all values if you are interested in specific items.

Table 1. Dataset selection process trimmed the number of journals from 173,612 to 137,827.
StepRecords
Journal records returned by api.crossref.org/journals173,612
Dropped: no ISSN32,208
Dropped: same ISSN set already seen (title variants of one journal)258
Dropped: ISSN present but zero DOIs, so no coverage block3,319
Kept: journals with an ISSN, DOIs, and an all-time coverage block137,827

Crossref provides participation reports for three different eras: the current era includes two years prior to the present and the present year, i.e. 2024-2026; the backfile era is prior to the present (<= 2023), and the all era combines these two. Figure 1 shows violin plots of the participation report averages for journals with >= 10 DOIs in three eras and Table 2 shows the median and inter-quartile ranges for each era. It is encouraging to see that the median participation report is highest during the current era (2024-2026), i.e. the completeness of the metadata is increasing.

Figure 1. Violin plots showing the distributions of participation report averages of journals with >= 10 DOIs in three eras. Horizontal lines show medians (with labels) and vertical lines show interquartile ranges.

Table 2. Median and inter-quartile ranges for selected journals in three eras.
EraJournalsMedianIQR
current (2024–2026)94,09127%18% to 36%
backfile (≤2023)109,36920%8% to 27%
all time127,30321%10% to 29%

Another interesting feature of these plots is their “wavey” nature, i.e. they are stacks of peaks. This reflects the fact that there are fourteen participation report checks and that many of them are either all (100%) or nothing (0%). This leads to visible peaks in the distributions at 1/14, 2/14, 3/14, … 7/14. Figure 2 shows the histogram of average completeness along with example journals on these peaks (all elements = 0% or 100%) and in between (some fractional elements).

The “Checks at 100%” column shows the most common checks that are at 100% in each group. For example, the first large peak at 1/14 (7.143%) includes 3,439 journals and 1,323 of them have similarity-check set, like Endocrine Abstracts.  Similarity check is a service provided by Crossref and powered by iThenticate that is used to check manuscripts for evidence of plagiarism. In the participation reports, similarity check counts records that carry a full-text link marked for Similarity Check. Again, a publisher that enrolls adds the link to every deposit automatically, so it is typically 100%.

The second peak example, Journal of Thoracic Disease, adds update policy which is a publisher policy that defines how a publisher handles post-publication changes—such as corrections, retractions, and withdrawals—for its published content. In this case, the most common records add update policy to similarity check to have 2/14 items. This is also a publisher level flag, typically deposited on every record once a publisher has a policy, so the completeness is 100%.

Peaks 3-7 combine the publisher level items with more traditional metadata elements like abstract, resource-links, licenses, and ORCIDs in various mixtures until only a small number of journals (23) include the most common seven items.

Figure 1. Histogram of average completeness for ~100,000 journals with ten or more DOIs during the current era. The participation reports include 14 elements listed on the right. The histogram has peaks for completeness values that correspond to integer numbers of elements (1-7). The radar plots in the top row show examples of journals with values of 100% for these elements corresponding to the peaks in the distribution. Examples of journals with fractional completeness, i.e. between the peaks, are shown in the second row between the peaks.

Table 3. Journals shown as examples in Figure 2. Journals on the peaks show K/14 in the Ave column where K is the number of elements at 100%. Journals in between, i.e. with fractional elements, are shown as actual values. Journal titles are links to the Crossref Connectivity tool for the journal that shows identifier connectivity and all participation report activity. The Checks at 100% column shows participation report elements that are at 100% in the group and the number of journals with those checks.
#AveJournal / ISSNPublisherDOIsChecks at 100%
10.03SCIENCE TIME
2992-8850
Center for Advanced Studies1,866none
21/14Endocrine Abstracts
1470-3947; 1479-6848
Bioscientifica8,645similarity check
1,323 journals
30.11International Journal of Research and Innovation in Social Science
2454-6186
International Journal of Research and Innovation in Social Science20,780none
42/14Journal of Thoracic Disease
2072-1439; 2077-6624
AME Publishing Company2,870update policy, similarity check
1,264 journals
50.18JOURNAL OF CLINICAL AND DIAGNOSTIC RESEARCH
2249-782X; 0973-709X
JCDR Research and Publications5,710none
63/14Proceedings of the AAAI Conference on Artificial Intelligence
2159-5399; 2374-3468
Association for the Advancement of Artificial Intelligence (AAAI)11,273abstracts, resource links, similarity check
1,698 journals
70.25CHEST Journal
0012-3692; 1931-3543
Elsevier9,645licenses, update policy, resource links
84/14Journal of Infrastructure Policy and Development
2572-7923; 2572-7931
EnPress Publisher2,350abstracts, licenses, resource links, similarity check
2,597 journals
90.32Nature
0028-0836; 1476-4687
Springer Science and Business Media LLC12,898licenses, resource links, similarity check
105/14Philosophy and Governance
3041-248X
NGO "Eastern European Union of Scientists"241abstracts, ORCIDs, licenses, resource links, similarity check
411 journals
110.39International Journal of Biological Macromolecules
0141-8130
Elsevier25,618licenses, update policy, resource links
126/14Revista Educação Especial
1808-270X; 1984-686X
Universidade Federal de Santa Maria203abstracts, ORCIDs, references, licenses, resource links, similarity check
140 journals
130.47IEEE Internet of Things Journal
2327-4662
Institute of Electrical and Electronics Engineers11,585licenses, similarity check
147/14Journal of Modern Foreign Psychology
2304-4977
Moscow State University of Psychology and Education180abstracts, ORCIDs, references, licenses, affiliations, resource links, similarity check
23 journals

The peak examples in Table 3 demonstrate a common pathway up the Crossref participation report ladder, but the second row shows that many other metadata combinations also exist at all levels of completeness. Table 4 lists the journals with average completeness >= 75%. These current bright spots can serve as good examples and sources of lessons learned for other journals interested in improving the completeness.

Table 4. Journals with average completeness ≥75%, bright spots that provide good examples and lessons learned. Names link to tools for exploring metadata connectivity and participation reports.
AveJournal / ISSNPublisherDOIs
0.881SciPost Physics Community Reports
3051-3197
Stichting SciPost21
0.860eLife
2050-084X
eLife Sciences Publications, Ltd.7,168
0.832Gigabyte
2709-4715
GigaScience Press89
0.821ASM Food Microbiology
3070-8133
American Society for Microbiology4
0.819ASM Animal Microbiology
3068-5265
American Society for Microbiology17
0.815SciPost Physics
2542-4653
Stichting SciPost1,011
0.802SciPost Physics Lecture Notes
2590-1990
Stichting SciPost60
0.794SciPost Physics Core
2666-9366
Stichting SciPost254
0.781SciPost Physics Codebases
2949-804X
Stichting SciPost125
0.765SciPost Physics Proceedings
2666-4003
Stichting SciPost122
0.761PRX Intelligence
3070-0329
American Physical Society26
0.760Journal of Cell Biology
0021-9525; 1540-8140
The Rockefeller University Press784
0.757Journal of Bacteriology
0021-9193; 1098-5530
American Society for Microbiology670
0.754Applied and Environmental Microbiology
0099-2240; 1098-5336
American Society for Microbiology1,879
0.754mBio
2150-7511; 2161-2129
American Society for Microbiology2,211
0.752Journal of High Energy Physics
1126-6708; 1029-8479
Springer-Verlag8,062
0.752mSystems
2379-5077
American Society for Microbiology1,104
0.750Sustainability and Education
3140-197X
Institute of Integrated Science Education of Dankook University4

Conclusion

The Crossref Participation Reports provide an interface and API access to 10-14 measures of Crossref service usage and metadata completeness. Participation averages for 137,827 journals were explored to establish a baseline for understanding the scores. Comparing the current period (2024-26) to the backfile (2023 and before) shows that median Crossref Participation Report scores for journals have increased from 20 to 27%.

The Participation Reports include 14 items that can be combined in many ways to improve completeness. Several of these items tend to occur across all DOIs for a particular journal, i.e. completeness = 100%) leading to average completeness scores that cluster around values like 1/14, 2/14, … 7/14. Journals with these are like signposts along the pathway to more complete metadata. There are also many journals with fractional completeness for some items leading to a spectrum from 0% to 88% completeness.

Data Availability

The data used in this work is available at: https://doi.org/10.5281/zenodo.24431907.

Tools

Three tools are freely available for exploring Crossref metadata for any members or journals. The Crossref Tool Guide describes how these tools work together to help you explore your metadata and identify pathways to improvement.

Crossref Connectivity explores identifier occurrence in terms of, for example, the number of people that have identifiers, rather than the number of metadata records that include an identifier (completeness) and provides participation reports results in a way that makes it easier to compare across time periods.

Crossref Completeness explores metadata completeness in terms of FAIR use cases extended to Crossref from many years of work with DataCite metadata.

Crossref History explores yearly completeness using the FAIR use cases to recognize evolution of metadata completeness more generally than the backfile/current time structure of the Participation Reports.