arXiv · 2611.22353
NONSENSE: Non-Semantic Strings Expose Semanticity Gaps in Dense Retrieval
Abstract
Dense retrieval is central to modern search, question answering, and retrieval-augmented generation, where high similarity is often treated as evidence of semantic relevance. We examine how far this assumption can be stressed. NONSENSE is a black-box sparse-edit probe that constructs visibly non-semantic strings while forbidding query-term copying, optimizing only the dense similarity between a query and the string. Across five BEIR-style datasets and three popular single-vector retrievers, these strings frequently outscore the retriever's own best judged query-relevant passage (QRP), and often rank first in the full corpus. The result indicates a semanticity gap in the evaluated dense retrievers: nearest-neighbor status can be separated from passage-like meaning. These findings highlight the importance of evaluating dense retrievers not only by whether they retrieve relevant documents, but also by how reliably their similarity scores reflect semantic evidence.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Ankit Gupta. 2026-11-17. NONSENSE: Non-Semantic Strings Expose Semanticity Gaps in Dense Retrieval. https://arxiv.science/abs/2611.22353
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.