arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.13415

A precision-driven evaluation protocol for author name disambiguation

Francesca Cappelli·Giovanni Colavizza·Silvio Peroni
View original source ↗Read source document ↗Export metadata ↓

Abstract

Author Name Disambiguation systems are commonly evaluated using pairwise and B$^3$ metrics, which summarize clustering quality through aggregate precision and recall but do not explicitly distinguish between lumping (merging publications from different authors) and splitting (fragmenting the publications of a single author). Because these errors can have asymmetric consequences in bibliographic applications, we propose a precision-driven evaluation framework that decomposes clustering quality into complementary instance-level and cluster-level signals. The framework introduces cluster-level Lumping Error and Splitting Error to separately quantify contamination and fragmentation, combines them with standard pairwise and B$^3$ metrics through a Composite Score, and uses the resulting measures for model selection. We characterize the diagnostic behavior of the framework through a controlled synthetic case study and assess the sensitivity of the Composite Score to its weighting scheme. The results show that the proposed structural measures distinguish error patterns that aggregate metrics can mask, while the sensitivity analysis identifies a stable region for the adopted weighting configuration. The framework is released as an open-source Python pipeline.

Explore related subjects

cs.DL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Francesca Cappelli, Giovanni Colavizza, Silvio Peroni. 2026-10-09. A precision-driven evaluation protocol for author name disambiguation. https://arxiv.org/abs/2610.13415

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

OC-AND: A Citation-Enriched Benchmark for Author Name Disambiguation

Author Name Disambiguation benchmarks are predominantly derived from closed infrastructures that may not reflect conditions in open bibliographic environments. We present OC-AND, an open citation-enriched benchmark for AND constructed by remapping the WhoIsWho dataset with bibliographic and citation metadata from OpenCitations. The dataset is produced through a reproducible five-stage pipeline involving DOI verification, metadata validation, OpenCitations enrichment, and author-centric transformation. OC-AND retains the WhoIsWho ground-truth files and partition structure unchanged, while its publication files are restricted to publications that could be matched to a validated DOI and an OpenCitations record. It introduces realistic characteristics of open environments: heterogeneous metadata completeness, absent abstracts and affiliations, asymmetric citation coverage, and sparse graph connectivity. The dataset contains author records with associated citations, enabling evaluation of AND methods in settings closer to real-world open scholarly infrastructure scenarios. OC-AND is released as open data with complete provenance documentation and pipeline code, supporting reproducible research and the development of citation-aware disambiguation approaches.

cs.DL↗
arXiv2026

Life after delisting: tracking the publication output and citation impact of Scopus-discontinued journals with OpenAlex

Selective databases such as Scopus periodically re-evaluate the titles they index and remove those that no longer meet their quality criteria. Once a journal is removed, its subsequent output is no longer recorded, and the consequences of this process have therefore received little attention. This study analyzes the publication output and citation impact of 514 sources delisted from Scopus between 2018 and 2024, based on 678,162 documents retrieved from OpenAlex over a window of three years before and after delisting. Differences between periods are tested with the Wilcoxon signed-rank test for paired samples, globally and by geographic region, field of knowledge and SJR quartile. Output grows until the year of delisting and falls sharply afterwards. The median annual output per journal drops from 35 to 22 documents (-37.1%; p < 0.001; r = 0.30), 61.7% of journals publish less, and 84 (16.3%) have no documents recorded in OpenAlex after delisting. Citations per document, measured over a three-year window, barely change when each journal is compared with itself (-2.4%; p = 0.079), and journals are almost evenly split between those whose impact falls and those whose impact holds steady or rises. The results indicate that delisting makes these journals less attractive as publication venues but does not penalize the citation of the work they publish to the same extent, which raises concerns from a research integrity perspective.

cs.DL↗
arXiv2026

Persistence Paradox in Dynamic Science: Evidence from the Deep Learning Revolution

Persistence is often regarded as a virtue in science. In this paper, however, we challenge this conventional view by highlighting its contextual nature, particularly how persistence can become a liability during paradigm shifts. We focus on the deep learning revolution catalyzed by AlexNet in 2012. Analyzing the 20-year career trajectories of more than 5,000 scientists active in top machine learning venues during the preceding decade, we examine how their research focus and output evolved. We first uncover a dynamic period in which leading venues increasingly prioritized cutting-edge deep learning developments, displacing traditional statistical learning methods. Scientists responded to these changes in markedly different ways: those who were previously successful or affiliated with established teams adapted more slowly. Such persistence is positively associated with productivity but, after 2012, negatively associated with scientific impact. Most researchers, and the largest share of the field's output, cluster in a band of moderate persistence, pointing to a trade-off between output and impact, as well as to institutional frictions that make larger departures costly. These conclusions are robust to alternative identification strategies and to competing explanations such as topic popularity premiums and survivorship bias. Taken together, our macro- and micro-level findings suggest that, in this case, a paradigm shift creates an opportunity structure by devaluing the very expertise that conferred incumbents' advantage in the first place.

cs.DL↗