RAGit: Turning Document Preprocessing into an Experimental Variable for RAG

Omar El Bachyr; Saad Ezzini; Tegawendé F. Bissyandé; Jacques Klein

Home/Archive/arXiv

arXiv · 2610.24659

RAGit: Turning Document Preprocessing into an Experimental Variable for RAG

Omar El Bachyr·Saad Ezzini·Tegawendé F. Bissyandé·Jacques Klein

View original source ↗Read source document ↗Export metadata ↓

Abstract

Document preprocessing choices can substantially affect retrieval in Retrieval-Augmented Generation (RAG) pipelines, yet they are often selected heuristically and are difficult to compare under consistent conditions. We present RAGit, an open-source experimentation workbench for the systematic and reproducible evaluation of document preprocessing choices through downstream retrieval. RAGit represents parsing, chunking, indexing, retrieval, and evaluation as configurable experimental stages, enabling controlled comparisons while the remaining downstream settings are kept fixed. It preserves provenance from retrieved chunks to source pages and maps heterogeneous retrieval units to a common page-level ranking for evaluation. The current implementation supports multiple parsers and chunking strategies, dense and late-interaction retrieval, reusable intermediate artifacts, and multi-configuration benchmarking. We demonstrate RAGit through controlled experiments comparing alternative preprocessing configurations under a fixed retrieval setup, illustrating how preprocessing can be treated as an explicit experimental variable. RAGit and its documentation are publicly available on GitHub (https://github.com/OmarElbachyr/ragit). A demonstration video is available at https://youtu.be/tsst-Bhfz4I.

Explore related subjects

cs.IR

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Omar El Bachyr, Saad Ezzini, Tegawendé F. Bissyandé, Jacques Klein. 2026-10-17. RAGit: Turning Document Preprocessing into an Experimental Variable for RAG. https://arxiv.org/abs/2610.24659

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Reproducing LightMem: When Does Memory Construction Help Over Raw-Turn Retrieval?

Long-term conversational agents require access to information from earlier interactions, such as a user's preferences, past requests, or previously mentioned facts. Repeatedly providing the full dialogue history can be expensive as conversations grow, so many memory approaches instead transform past interactions into compact entries that can be retrieved when needed. LightMem is a recent lightweight memory-management approach that reports strong effectiveness while maintaining relatively low construction cost. However, it still relies on a separate constructed memory representation and is evaluated with only one retriever, leaving unclear how sensitive its results are to retriever choice and whether memory construction provides sufficient benefits over retrieving directly from raw turns. In this study, we reproduce LightMem and compare it with Naive RAG, which retrieves directly from raw user turns. We recover LightMem's main configuration trend, but find that retriever choice substantially affects answer accuracy: changing only the retriever over a fixed LightMem store shifts answer accuracy from 58.1\% to 75.5\%. Retrieval over constructed memory also does not consistently outperform raw-turn retrieval. At matched retrieval depths, Naive RAG generally achieves higher answer accuracy, whereas LightMem has an advantage under tight answering-token budgets. Oracle-reference evaluation further suggests that memory construction may omit or alter answer-relevant information. Overall, LightMem provides a conditional context-efficiency advantage rather than a consistent answer accuracy advantage over Naive RAG. Strong retrieval over raw turns can match or outperform constructed memory as the answering-token budget increases, motivating future work on retrieval, reranking, query formulation, and their interaction with raw-turn and constructed memory representations.

cs.IR↗

arXiv2026

Is Serving-Time Language-Model Inference Necessary for Recommendation? Evidence from Generative and Agentic Settings

Language models (LMs) are increasingly used in recommender systems to generate recommendations or coordinate recommendation tools. However, does their strong performance imply that serving-time LM inference is necessary? We challenge this seemingly reasonable but unverified assumption. To investigate this question, we introduce recovery-based mechanism auditing, which tests whether independent alternatives can achieve the accuracy of LM-based recommender systems without the audited inference mechanisms under controlled information and evaluation conditions. For generative recommendation, we propose LIME-Rec, which combines sequential, collaborative, and offline semantic information without autoregressive LM decoding. On three Amazon Reviews datasets (Beauty, Toys, and Sports), LIME-Rec achieves the highest reported mean in all 12 dataset-metric combinations, with up to a 15.2% relative improvement in Recall@10 over GRAM, an LM-based generative recommender. For agentic recommendation, we propose DART-Rec, which replaces LM-based tool control with deterministic fusion and fixed expert selection. On 1,000 Beauty users, given the same candidate items and expert scores, deterministic fusion achieves a 16.7% relative improvement in Recall@10 over a fine-tuned LM controller. With at most two tool calls, fixed expert selection matches the LM controllers' Hit@10 outcomes for every user in 17 of 18 comparisons. These findings demonstrate that strong recommendation performance does not necessarily require serving-time LM inference: under the evaluated task and information conditions, alternatives without the specified inference mechanisms can match or exceed the accuracy of their LM-based counterparts. To support reproducibility, code is available at https://github.com/Double-wk/LIME-Rec.

cs.IR↗

arXiv2026

P$^3$Rec: Distilling Prior--Posterior Preference Reasoning for LLM-based Recommendation

Large language models (LLMs) exhibit strong semantic understanding and preference reasoning capabilities, offering new opportunities for user modeling in recommender systems. Existing LLM-as-Enhancer methods typically distill LLM-derived preference knowledge into lightweight recommenders to avoid costly online LLM inference. However, they often construct distillation knowledge from only one perspective. Prior preference captures users' stable and consistent interests but provides limited guidance for the current decision, whereas posterior preference reveals target-relevant fine-grained interests but may rely excessively on target clues. To address these limitations, we propose P$^3$Rec, a framework that jointly extracts and internalizes complementary prior and posterior preference reasoning knowledge. Specifically, P$^3$Rec first derives target-agnostic prior preferences and target-conditioned posterior preferences from the user side, while further extracting item-centric preference representations from item semantics and predecessor interactions. It then progressively internalizes prior and posterior knowledge into behavioral representations through prior preference absorption and posterior-guided preference distillation. Since the resulting comprehensive preference representation may not always provide an equally decisive retrieval direction, P$^3$Rec further characterizes historical interest dispersion with interest entropy and adaptively calibrates the user representation before contrastive retrieval optimization. In this way, P$^3$Rec achieves more complete preference reasoning while preserving efficient recommendation. Extensive experiments on multiple public datasets demonstrate its effectiveness.

cs.IR↗