arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2508.05676

Domain-Specific Fine-Tuning and Prompt-Based Learning: A Comparative Study for developing Natural Language-Based BIM Information Retrieval Systems

Han Gao·Timo Hartmann·Botao Zhong·Kai Lia·Hanbin Luo
View original source ↗Read source document ↗Export metadata ↓

Abstract

Building Information Modeling (BIM) is essential for managing building data across the entire lifecycle, supporting tasks from design to maintenance. Natural Language Interface (NLI) systems are increasingly explored as user-friendly tools for information retrieval in Building Information Modeling (BIM) environments. Despite their potential, accurately extracting BIM-related data through natural language queries remains a persistent challenge due to the complexity use queries and specificity of domain knowledge. This study presents a comparative analysis of two prominent approaches for developing NLI-based BIM information retrieval systems: domain-specific fine-tuning and prompt-based learning using large language models (LLMs). A two-stage framework consisting of intent recognition and table-based question answering is implemented to evaluate the effectiveness of both approaches. To support this evaluation, a BIM-specific dataset of 1,740 annotated queries of varying types across 69 models is constructed. Experimental results show that domain-specific fine-tuning delivers superior performance in intent recognition tasks, while prompt-based learning, particularly with GPT-4o, shows strength in table-based question answering. Based on these findings, this study identify a hybrid configuration that combines fine-tuning for intent recognition with prompt-based learning for question answering, achieving more balanced and robust performance across tasks. This integrated approach is further tested through case studies involving BIM models of varying complexity. This study provides a systematic analysis of the strengths and limitations of each approach and discusses the applicability of the NLI to real-world BIM scenarios. The findings offer insights for researchers and practitioners in designing intelligent, language-driven BIM systems.

Explore related subjects

cs.IRcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Han Gao, Timo Hartmann, Botao Zhong, Kai Lia, Hanbin Luo. 2025-08-05. Domain-Specific Fine-Tuning and Prompt-Based Learning: A Comparative Study for developing Natural Language-Based BIM Information Retrieval Systems. https://arxiv.org/abs/2508.05676

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation

The retrieval stage of retrieval-augmented generation (RAG) for scientific question answering depends on how documents are segmented and how chunks are represented in embedding space. This dependence is especially relevant to chemistry texts, which contain dense terminology, symbolic notation, quantitative evidence, and context associated with document structure. However, benchmark-based evidence on the interaction between chunking strategy and embedding model remains limited for chemistry-specific retrieval. Using ChemQuests, a corpus of 952 question-answer pairs from 151 ChemRxiv papers across 17 chemistry subfields, we construct chunk-level, Massive Text Embedding Benchmark (MTEB)-compatible retrieval benchmarks for controlled evaluation. We first screen 41 embedding models on the external chemistry retrieval benchmarks ChemNQRetrieval and ChemHotpotQARetrieval using a geometric-mean metric at rank 10 (Geom@10), which we validate against the full retrieval-metric profile. We then evaluate shortlisted models on ChemQuests-derived tasks across five chunking strategies, seven chunk sizes, and multiple overlap settings. Embedding choice is associated with the largest observed differences in evidence retrieval, with retrieval-tuned E5, Beijing Academy of Artificial Intelligence General Embedding (BGE), and Nomic models among the strongest overall. Within the evaluated grid, medium-to-large chunks combined with fixed-token, recursive-token, or hierarchical-section chunking provide a practical starting point for the retrieval stage of chemistry-aware RAG. Low overlap was generally favored where overlap variation was evaluated.

cs.IR↗
arXiv2026

MARS: Modality-Aligned Retrieval for Sequence Augmented CTR Prediction

Click-through rate (CTR) prediction serves as a cornerstone of recommender systems. Despite the strong performance of current CTR models based on user behavior modeling, they are still severely limited by interaction sparsity, especially in low-active user scenarios. To address this issue, data augmentation of user behavior is a promising research direction. However, existing data augmentation methods heavily rely on collaborative signals while overlooking the rich multimodal features of items, leading to insufficient modeling of low-active users. To alleviate this problem, we propose a novel framework \textbf{MARS} (\textbf{M}odality-\textbf{A}ligned \textbf{R}etrieval for \textbf{S}equence Augmented CTR Prediction). MARS utilizes a Stein kernel-based approach to align text and image features into a unified and unbiased semantic space to construct multimodal user embeddings. Subsequently, each low-active user's behavior sequence is augmented by retrieving, filtering, and concentrating the most similar behavior sequence of high-active users via multimodal user embeddings. Validated by extensive offline experiments and online A/B tests, our framework MARS consistently outperforms state-of-the-art baselines and achieves substantial growth on core business metrics within Kuaishou~\footnote{https://www.kuaishou.com/}. Consequently, MARS has been successfully deployed, serving the main traffic for hundreds of millions of users. To ensure reproducibility, we provide anonymous access to the implementation code~\footnote{https://github.com/wangshukuan/MARS}.

cs.IR↗
arXiv2026

Time-Aware Diffusion based on Preference Disentanglement for Generative Recommendation

Recently, Generative Recommenders (GRs) have emerged as a transformative recommendation paradigm by replacing traditional item IDs with semantic indices (SIDs). Owing to the exceptional generative capabilities of diffusion models, a few pioneering works explore developing GRs with diffusion architectures as the backbone. However, a fatal limitation of existing diffusion-based GRs is that the diffusion process applies uniformly to all items within the historical interactions. In contrast, the user preference is shaped by multifaceted time-evolving factors and thus exhibits a non-stationary distribution in the temporal aspect. To bridge this gap, this study proposes a novel GR framework, named TDPM, by designing the time-aware diffusion on SID tokens. Specifically, TDPM explicitly integrates the impact of time-evolving user preferences into the diffusion process. In detail, the user preference is disentangled into (i) the period preference, which remains consistent over a long time-span, and (ii) the point preference, which is triggered by recent focal events. Extensive experiments on three public real-world datasets demonstrate the significant superiority of TDPM over the state-of-the-art baselines. TDPM achieves average improvements of up to 29.21% and 25.45% in terms of HR@20 and NDCG@20, respectively. The ablation study further underscores the necessity of time-aware token diffusion in diffusion-based GRs.

cs.IR↗