arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2405.11791

LEXA: Legal Case Retrieval via Graph Contrastive Learning with Contextualised LLM Embeddings

Yanran Tang·Ruihong Qiu·Yilun Liu·Xue Li·Zi Huang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Legal case retrieval (LCR) is a specialised information retrieval task aimed at identifying relevant cases given a query case. LCR holds pivotal significance in facilitating legal practitioners to locate legal precedents. Existing LCR methods predominantly rely on traditional lexical models or language models; however, they typically overlook the domain-specific structural information embedded in legal documents. Our previous work CaseGNN successfully harnesses text-attributed graphs and graph neural networks to incorporate structural legal information. Nonetheless, three key challenges remain in enhancing the representational capacity of CaseGNN: (1) The under-utilisation of rich edge information in text-attributed case graph (TACG). (2) The insufficiency of training signals for graph contrastive learning. (3) The lack of contextualised legal information in node and edge features. In this paper, the LEXA model, an extension of CaseGNN, is proposed to overcome these limitations by jointly leveraging rich edge information, enhanced training signals, and contextualised embeddings derived from large language models (LLMs). Specifically, an edge-updated graph attention layer (EUGAT) is proposed to comprehensively update node and edge features during graph modelling, resulting in a full utilisation of structural information of legal cases. Moreover, LEXA incorporates a novel graph contrastive learning objective with graph augmentation to provide additional training signals, thereby strengthening the model's legal comprehension capabilities. What's more, LLMs are employed to generate node and edge features for TACG. Extensive experiments on two benchmark datasets demonstrate that LEXA not only significantly improves CaseGNN but also achieves supreme performance compared to state-of-the-art LCR methods.

Explore related subjects

cs.IR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yanran Tang, Ruihong Qiu, Yilun Liu, Xue Li, Zi Huang. 2026-03-05. LEXA: Legal Case Retrieval via Graph Contrastive Learning with Contextualised LLM Embeddings. https://arxiv.org/abs/2405.11791

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Revisiting Self-Attentive Sequential Recommendation Beyond the LLM Paradigm

Sequential recommendation adopted the Transformer almost as soon as it appeared: SASRec ported the decoder to next-item prediction in 2018, a year after Attention is All You Need, and the paradigm has borrowed from language modeling ever since. The two tasks look nearly identical, both consume integer-ID sequences with causal self-attention, yet they pursue opposite ends. A recommender works to bring more users into contact with more items, an entropy-increasing goal; a language model works to converge many phrasings of a question onto one answer, an entropy-decreasing one. We argue this difference, not engineering effort, is why recommendation has not reproduced the clean scaling that language models enjoy: behavioral data is locally regular yet globally heterogeneous, a casino, whereas language is locally diverse yet globally convergent, a library. Taking SASRec as an entry point, we revisit the self-attentive paradigm as a comparative study of the two domains and ask which of its inherited assumptions, implicit-only personalization, absolute positional semantics, leakage-prone single-step evaluation, and atomic tokenization, are incidental rather than intrinsic to recommendation. Our BlueSky claim is that, beyond borrowing from language models, the next findings will come from a careful comparison of the two domains that starts from the entropy structure of behavioral data. We propose no new model; we expose the gaps, outline the data- and systems-level agenda they imply, and argue that the comparison can ultimately help both domains.

cs.IR↗
arXiv2026

Attention Calibration for Position-Fair Dense Retrieval

Dense retrieval compresses a passage into a single vector, but this compression is positionally skewed: early content dominates the embedding, and retrieval degrades when the relevant span appears later. Prior work proposed an inference-time method that counteracts this skew by equalizing the pooling token's attention across passage segments. However, (i) it redistributes attention at a fixed strength, (ii) it forces the pooling token's attention to itself to a fixed basket-level mass despite substantial variation across layers and architectures, and (iii) its effect on retrieval has not been evaluated. We introduce a strength coefficient that interpolates between uncalibrated and fully equalized attention, together with an efficient implementation that reduces peak calibration memory overhead from 5-7 GiB to under 1 MiB. Across three embedding models and two pooling schemes, moderate calibration provides a better retrieval trade-off than full equalization. We introduce a variant that preserves the pooling token's self-attention mass and redistributes only the remaining mass. On a position-aware retrieval benchmark spanning 10 languages and 31 domains, a configuration selected on English FineWeb-PosQ and transferred without tuning reduces position sensitivity in all 16 evaluated length-quartile, model, and retrieval-setting combinations, by up to 43% relative, while improving nDCG@10 by up to 4.8% relative and leaving general retrieval effectiveness on NanoBEIR essentially unchanged. Calibration runs at indexing time, adding no query-time latency. We release our code at github.com/impresso/fair-sentence-transformers

cs.IR↗
arXiv2026

Pre-retrieval Query Clustering for Adaptive Top-k Document Retrieval in RAG Systems

RAG systems commonly retrieve a fixed number of documents (top-k) to ground generation, but this static approach is brittle: simple queries suffer over-retrieval (adding noise and cost) while complex queries are under-retrieved, causing recall failures that cascade into incorrect answers. Motivated by the question of how many documents must be retrieved to answer an arbitrary query reliably, we propose a practical, general framework for query-adaptive retrieval depth. Offline, we estimate per-query retrieval difficulty by measuring NDCG under the default retriever and deriving a query-specific saturation point k* from the NDCG-k curve. Because computing these signals online is expensive, we cluster a large set of queries in embedding space and summarize each cluster with a recommended retrieval depth that targets high coverage (e.g., ~95%) using a mean-plus-variance rule. At runtime, the system assigns an incoming query to a cluster and selects the corresponding top-k in constant time. Compared with post-retrieval confidence methods that rely on clustering retrieved documents, our approach is pre-retrieval and query-centric, making it robust in heterogeneous, case-like corpora and applicable across domains such as legal, healthcare, finance, and enterprise search. Finally, this framework has been tested in full-traffic queries that improved $F_1$ by over 36% while reducing token usage by 14% on low-complexity clusters without accuracy loss.

cs.IR↗