arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2609.29652

SmallReason-ColBERT: An Ultra-Small Late-Interaction Retriever for Reasoning Intensive Retrieval

Abdelrahman Abdallah·Mohammed Ali·Adam Jatowt
View original source ↗Read source document ↗Export metadata ↓

Abstract

Reasoning-intensive retrieval remains difficult for small models. Compact public ColBERTs are usually trained on general-purpose corpora and underperform reasoning-tuned 150M+ baselines on BRIGHT~\cite{bright} by several nDCG@10 points. However, no public reasoning-tuned ColBERT exists at edge scale. We introduce \textbf{SmallReason-ColBERT}, a 32M late-interaction retriever that closes much of this gap with three components: a varied-length contrastive warmup on ReasonIR-VL, a hard-negative contrastive polish on merged ReasonIR-HQ and BGE-Reasoner data, and a single-layer per-query-token importance head trained on top of the frozen base. The head is trained with an un-normalised weighted MaxSim score and evaluated with its length-normalised form. In a controlled re-training, replacing this training objective with the symmetric normalised score causes the loss to stall and costs $3.59$ nDCG@10. The full recipe reaches \textbf{21.41} mean nDCG@10 on BRIGHT, within $1.21$ of the 150M Reason-ModernColBERT (22.62) and above all $\le 33$M ColBERTs we evaluate. Through ablations on capacity, initialisation, and score variants, we further show that the learned head outperforms fixed IDF weighting and that simply thresholding the learned gates is harmful. https://github.com/DataScienceUIBK/SmallReason-ColBERT

Explore related subjects

cs.IR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Abdelrahman Abdallah, Mohammed Ali, Adam Jatowt. 2026-08-29. SmallReason-ColBERT: An Ultra-Small Late-Interaction Retriever for Reasoning Intensive Retrieval. https://arxiv.org/abs/2609.29652

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

WebArxiv: A Reproducible Benchmark for Evaluating Multimodal Web Agents on arXiv Tasks

Foundation models now enable autonomous agents to interact with real-world websites, but existing benchmarks emphasize general-purpose browsing, underrepresent research-oriented environments and scholarly discovery workflows, and often depend on live sites whose changing content and structure undermine reproducibility. arXiv provides a realistic, reproducible, hierarchically structured, information-centric testbed without privacy-sensitive interactions. We introduce WebArxiv, a static-snapshot benchmark comprising 510 time-invariant tasks, each with a unique deterministic ground truth. Its diverse, realistic scholarly tasks go beyond simple information lookup and rule following to emphasize multi-constraint paper retrieval, fine-grained content extraction, and cross-paper comparison. Evaluations of a range of foundation-model-based web agents show that WebArxiv remains challenging. Behavioral analysis reveals that agents over-rely on fixed interaction histories, causing incomplete or repetitive reasoning. We therefore equip agents with a lightweight dynamic-memory mechanism for adaptive retrieval and reasoning over relevant context. The benchmark and code are available at https://anonymous.4open.science/r/74E4423BVNW/README.md.

cs.IR↗
arXiv2026

RQ-Reg: A Residual-Quantization-Based Framework for Continuous Value Prediction in Recommender Systems

Predicting continuous values such as watch-time and gross merchandise value (GMV) is a core problem in industrial recommendation systems. Its inherent difficulty stems from the highly complex and long-tailed distributions of the target signals, which are hard to model accurately. Existing regression methods typically rely on fixed parametric assumptions on the target distribution: overly simple assumptions underfit real-world data, whereas more intricate ones tend to sacrifice scalability and generalization. To address these limitations, we propose a sequence modeling framework based on residual quantization (RQ), in which the target continuous value is decomposed into a sequence of quantization codes that represent progressively finer approximations. The model autoregressively predicts these codes from coarse to fine granularity, with each step refining the residual error left by the previous one. To further improve the quality of the learned representations, we introduce an ordinal-aware representation learning objective that aligns the RQ code embedding space with the ordinal structure of target values, thereby yielding continuous representations of quantization codes and more accurate predictions. We conduct comprehensive experiments on public benchmarks for watch-time and lifetime value (LTV) prediction, together with a large-scale online A/B test for GMV prediction on an industrial short-video recommendation platform. Across all settings, the proposed method shows competitive performance among existing state-of-the-art approaches and generalizes well across diverse continuous value prediction scenarios.

cs.IR↗
arXiv2026

Cross-Country Code-Mixing for Generative Recommendation

Cross-country recommendation on modern e-commerce platforms is typically deployed with disjoint user and item ID spaces across markets, removing the shared anchors that conventional cross-domain methods rely on. Generative recommendation (GR) mitigates this by mapping items into a shared token space and training a unified model, but existing approaches keep behavior sequences strictly country-specific, so knowledge transfer occurs only at the parameter level and remains absent at the data level. Inspired by code-switching corpora in multilingual natural language processing, we propose CMRec, a cross-country GR framework that injects cross-country supervision at the data level via dual-constrained, context-aware code-mixing. CMRec first learns a shared semantic codebook from multi-modal content and behavioral co-occurrence across countries. It then uses this codebook to synthesize mixed-country sequences via token-level substitutions that satisfy both static (content) and dynamic (e.g., price, audience, popularity) constraints. Finally, it introduces a context-aware loss that reweights mixed samples according to their plausibility in the current sequence. Experiments on two real-world multi-country datasets and an online A/B test show that CMRec substantially improves recommendation quality in data-sparse countries while preserving performance in data-rich countries, achieving +1.77% advertising revenue and +2.64% orders on a large-scale e-commerce platform.

cs.IR↗