arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.17814

The Scored Population, the Operating Point, and the Pipeline: Three Objects That Decide What an Open-Web Event System Looks Like

Shengqi Zhou·Zhigang Liu·Wenxiang Zhu·Boyong Du·Kairui Yang·Haicheng Huang·Weijian Bu
View original source ↗Read source document ↗Export metadata ↓

Abstract

Open-Web streams can reference real-world events without making them suitable for organization: a correctly linked event may serve only as background, while an event absent from the knowledge base may be central to an item. This distinction reveals an evaluation blind spot in multistage Web systems: pipelines determine not only how inputs are scored, but also which inputs receive scores at all. Using open-Web event admission as a case study, we examine three connected objects (the scored population, the selected operating point, and behavior on the complete input stream) through shared-subset comparisons, admission-loss decomposition, and controlled extraction-recovery interventions. We find that reported ranking differences depend on the scored population, scoring gaps exhibit distinct backbone-dependent failure modes, and recovering missing scores can increase false acceptance under an unchanged threshold. On Original-302, Qwen candidate validity scores 255 of 302 items while direct admission scores all 302; evaluating both on the shared subset narrows their AUC gap. For Mistral, span-safe recovery raises false acceptance from 73.1% to 92.5% with the threshold fixed. Across 602 human-annotated evaluation items within a 3,290-record resource, these findings establish a reporting framework that considers score availability, threshold selection, and complete-stream decisions together.

Explore related subjects

cs.IR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Shengqi Zhou, Zhigang Liu, Wenxiang Zhu, Boyong Du, Kairui Yang, Haicheng Huang, Weijian Bu. 2026-10-13. The Scored Population, the Operating Point, and the Pipeline: Three Objects That Decide What an Open-Web Event System Looks Like. https://arxiv.org/abs/2610.17814

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

LLMRec: Benchmarking Large Language Models on Recommendation Task

Recently, the fast development of Large Language Models (LLMs) such as ChatGPT has significantly advanced NLP tasks by enhancing the capabilities of conversational models. However, the application of LLMs in the recommendation domain has not been thoroughly investigated. To bridge this gap, we propose LLMRec, a LLM-based recommender system designed for benchmarking LLMs on various recommendation tasks. Specifically, we benchmark several popular off-the-shelf LLMs, such as ChatGPT, LLaMA, ChatGLM, on five recommendation tasks, including rating prediction, sequential recommendation, direct recommendation, explanation generation, and review summarization. Furthermore, we investigate the effectiveness of supervised finetuning to improve LLMs' instruction compliance ability. The benchmark results indicate that LLMs displayed only moderate proficiency in accuracy-based tasks such as sequential and direct recommendation. However, they demonstrated comparable performance to state-of-the-art methods in explainability-based tasks. We also conduct qualitative evaluations to further evaluate the quality of contents generated by different models, and the results show that LLMs can truly understand the provided information and generate clearer and more reasonable results. We aspire that this benchmark will serve as an inspiration for researchers to delve deeper into the potential of LLMs in enhancing recommendation performance. Our codes, processed data and benchmark results are available at https://github.com/williamliujl/LLMRec.

cs.IR↗
arXiv2026

Ontology Interoperability: A Comprehensive Framework for Industrial-Scale Applications

Different ontologies with conflicting and overlapping concepts cause havoc in the design, develop, and deploy of ontology-driven applications. In this work, we propose a comprehensive ontology interoperability framework for industrial-scale ontology-driven applications. The framework employs three state-of-the-art semantic techniques in different phases of the ontology engineering (OE) life cycle: ontology design patterns (ODPs) in the design phase, ontology matching and versioning (OM\&OV) in the develop phase, and ontology validation (OVA) in the deploy phase, to achieve better ontology interoperability.

cs.IR↗
arXiv2026

ScholarQuest: A Taxonomy-Guided Benchmark for Agentic Academic Paper Search in Open Literature Environments

Academic paper search is a core step in scientific research, and LLM-based search agents are emerging as a promising paradigm for iterative, intent-driven literature exploration. However, existing benchmarks offer limited coverage of research intents and insufficient support for scalable answer construction and reproducible multi-turn evaluation. We propose \textbf{ScholarQuest}, a taxonomy-guided benchmark constructed from over 1,000 computer science topics and organized around four research intents. To construct answer sets at scale, we combine multi-source retrieval, citation expansion, and multi-stage relevance adjudication. We further build \textit{\textbf{ScholarBase}}, a million-scale literature environment with unified retrieval and citation-expansion interfaces, together with a shared multi-turn search harness. We benchmark complete literature search systems and compare language models within this harness, evaluating retrieval coverage, search efficiency, and robustness across research intents. Results show that agentic methods outperform single-shot retrieval baselines, yet the best-performing search system achieves only 0.312 Recall@100 and 0.352 Recall@All. Comparisons under the shared harness reveal intent-dependent model strengths, with higher Planner token consumption not consistently yielding higher recall. We release the benchmark, retrieval backend, and evaluation harness for reproducible research

cs.IR↗