arXiv · 2611.22847
Similarity-Aware Client-Side Reordering for Server-Side Storage Compression in Distributed File Systems
Abstract
As growing data volumes strain the capacity of distributed file systems, server-side storage compression transparently reduces disk usage, but its effectiveness is limited when similar blocks land in separate compression records and cannot share matches. We present a client-side reordering framework that transforms the physical block layout before persistence, so that related blocks are co-located within the same compression scope without replacing the existing storage-server compressor. It operates on bounded reordering windows, periodically selects among candidate layouts via lightweight compression of sampled data, and recovers the original logical order during reads with compact window-local metadata. We implement the framework in the SkyFS distributed file system, using the Zstandard (Zstd) compressor on the OpenZFS storage backend. Results on diverse datasets show that the framework reduces physical storage by up to 42.9% relative to logical-order writes, while achieving 94-108% of baseline sequential-read throughput and 91-99% of random-read IOPS. Client-side layout construction is the primary additional write cost, showing that adaptive similarity-aware reordering substantially improves storage efficiency with acceptable overhead.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Yuanhong Huang, Zicong Chen, Yili Ma, Jinwu Yang, Yulong Chen, Guangming Tan, Dingwen Tao. 2026-11-17. Similarity-Aware Client-Side Reordering for Server-Side Storage Compression in Distributed File Systems. https://arxiv.science/abs/2611.22847
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.