arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 1707.09320

Z-checker: A Framework for Assessing Lossy Compression of Scientific Data

Dingwen Tao·Sheng Di·Hanqi Guo·Zizhong Chen·Franck Cappello
View original source ↗Read source document ↗Export metadata ↓

Abstract

Because of vast volume of data being produced by today's scientific simulations and experiments, lossy data compressor allowing user-controlled loss of accuracy during the compression is a relevant solution for significantly reducing the data size. However, lossy compressor developers and users are missing a tool to explore the features of scientific datasets and understand the data alteration after compression in a systematic and reliable way. To address this gap, we have designed and implemented a generic framework called Z-checker. On the one hand, Z-checker combines a battery of data analysis components for data compression. On the other hand, Z-checker is implemented as an open-source community tool to which users and developers can contribute and add new analysis components based on their additional analysis demands. In this paper, we present a survey of existing lossy compressors. Then we describe the design framework of Z-checker, in which we integrated evaluation metrics proposed in prior work as well as other analysis tools. Specifically, for lossy compressor developers, Z-checker can be used to characterize critical properties of any dataset to improve compression strategies. For lossy compression users, Z-checker can detect the compression quality, provide various global distortion analysis comparing the original data with the decompressed data and statistical analysis of the compression error. Z-checker can perform the analysis with either coarse granularity or fine granularity, such that the users and developers can select the best-fit, adaptive compressors for different parts of the dataset. Z-checker features a visualization interface displaying all analysis results in addition to some basic views of the datasets such as time series. To the best of our knowledge, Z-checker is the first tool designed to assess lossy compression comprehensively for scientific datasets.

Explore related subjects

cs.OHastro-ph.IMcs.CE

Keep this discovery

BibTeXRIS

Dingwen Tao, Sheng Di, Hanqi Guo, Zizhong Chen, Franck Cappello. 2017-06-12. Z-checker: A Framework for Assessing Lossy Compression of Scientific Data. https://doi.org/10.1177/1094342017737147

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Circular Economy Synergies and Trade-offs in Data Centres

This report analyses data centre (DC) sustainability and circularity, revealing existing synergies and trade-offs: The PUE is too coarse, mixing cooling and power provisioning. It wrongly attributes server fan consumption and transformation losses to IT energy. It does not measure compute but infrastructure efficiency, which is already outstanding. Compute energy, however, is exploding. Better energy metrics for DCs would thus cover i) compute efficiency, ii) transformation efficiency, and iii) cooling overhead. Trade-offs exist between cooling energy and water as well as on-site and upstream water: Consuming water on-site lowers the cooling energy, which also lowers the water consumed upstream in power generation. For 'wet' electricity, there is little competition: It is worth spending more on-site energy to save both electricity and related upstream water. For 'dry' electricity, there is a trade-off. Waste heat recovery brings energy circularity but has limited uses and is not the same energy quality, a fact not reflected by current metrics. A better metric would consider the avoided energy through heat recovery instead of the amount recovered. Material circularity can be achieved by interpreting the 9R framework in the context of DCs. Circularity-enhancing measures can be categorised into product design, process design and business models, choice of materials, and operating conditions. Together, they have effects across all circularity levels. The relation between DCs and the power grid is complex. Modern DCs present new challenges for the grid. Mitigation includes battery storage and onsite generation. These measures have, in turn, further consequences, both beneficial and detrimental. They can offer grid flexibility as well as innovations in the field of energy. But they also bring noise, pollution, and GHGs, and compete with the energy sector for resources.

cs.OH↗
arXiv2026

Digital Twin Modeling of a Highly Automated Agricultural Tractor

In efforts to increase research efficiency and availability, a digital twin of our research tractor (AMX G-trac) is created, focusing especially on the CAN communication for data reading and actuation command following the ISOBUS protocol. Mevea Simulation Software is utilized as the foundation, providing the kinematic model and visuals, while Python is used to read and write CAN messages over a Kvaser CanKing virtual CAN channel. Various performance tests involving straight line and turning behavior are performed in both the digital twin simulation and in the real world to measure similarity. Results indicate that the Mevea model behaves very comparable in its lateral dynamics, often within 5-10 percent, but requires better data to fully capture the longitudinal aspects like acceleration. The final model described in this paper sets the table for a second iteration to include more tractor functions such as hydraulics and tractor-implement dynamics.

cs.OH↗
arXiv2026

Risk-based Design for Sustainability in Cloud Systems: Insights from an Experts' Survey

Cloud Systems' Sustainability is critical in Cloud Computing, especially with the growing demand in many industries. Sustainability risks in Cloud Computing can be tricky, mostly because of system complexity and their impact on performance. Thus, this research focuses on Risk-based design (RBD) and how it can support the early identification of possible risks for Cloud System Sustainability, as well as respective mitigation strategies of each risk. In order to successfully identify risks of Cloud System Sustainability, an Expert Survey is conducted including experts with different roles from different industries, to identify possible sustainability risks of a Cloud System on all different levels. Thematic analysis of the responses resulted in a categorization of risks, as well as in the identification of mitigation strategies and factors affecting each risk. Such findings can be helpful for researchers and practitioners that utilize RBD when building sustainable Cloud systems.

cs.OH↗