arXiv ScienceSearch

arXiv subjects

Liuchang Jing

Publications and source records attributed to Liuchang Jing.

2 recordsLinked to original sources

Reverse Neighbor Sliding and Order Selection for Efficient Multi-Proximity Graph Merging

Approximate k Nearest Neighbor (AKNN) search in high-dimensional space is a foundational problem in vector databases with widespread applications. Among the numerous AKNN indexes, Proximity Graph-based indexes achieve state-of-the-art search efficiency across various benchmarks. In many real-world scenarios, datasets are maintained as multiple segment-level graph indexes to support continuous writes and segment management. However, these fragmented indexes complicate maintenance and degrade search efficiency, making fast graph index merging essential. In this paper, we focus on the efficient merging of multiple existing graph indexes into a single one. To achieve this, we propose a Reverse Neighbor Sliding Merge (RNSM) that exploits structural information to boost merging efficiency. We further propose Merge Order Selection (MOS) to minimize total merge cost across multiple indexes by eliminating redundant operations. Experiments show that our approach yields up to a 3.86x speedup over existing index merge methods and a 9.92x speedup over index reconstruction, while maintaining comparable search performance. Moreover, our method scales to merging up to 50 sub-indexes on datasets of 100 million vectors, maintaining consistent speedups.

cs.DB

Quantization Meets Projection: A Happy Marriage for Approximate k-Nearest Neighbor Search

Approximate $k$-nearest neighbor (AKNN) search is a fundamental problem with wide applications. To reduce memory and accelerate search, vector quantization is widely adopted. However, existing quantization methods either rely on codebooks -- whose query speed is limited by costly table lookups -- or adopt dimension-wise quantization, which maps each vector dimension to a small quantized code for fast search. The latter, however, suffers from a fixed compression ratio because the quantized code length is inherently tied to the original dimensionality. To overcome these limitations, we propose MRQ, a new approach that integrates projection with quantization. The key insight is that, after projection, high-dimensional vectors tend to concentrate most of their information in the leading dimensions. MRQ exploits this property by quantizing only the information-dense projected subspace -- whose size is fully user-tunable -- thereby decoupling the quantized code length from the original dimensionality. The remaining tail dimensions are captured using lightweight statistical summaries. By doing so, MRQ boosts the query efficiency of existing quantization methods while achieving arbitrary compression ratios enabled by the projection step. Extensive experiments show that MRQ substantially outperforms the state-of-the-art method, achieving up to 3x faster search with only one-third the quantization bits for comparable accuracy.

cs.DB