arXiv ScienceSearch

arXiv · 2410.12627

Sparse flow reconstruction methods to reduce the costs of analyzing large unsteady datasets

Abstract

The cost of writing, transferring, and storing large data from unsteady simulations limits access to the entire solution, often leaving much of the flow under-sampled or unanalyzed. For example, modeling transient behavior of rare dynamic events requires 3D snapshots at high sampling rates over long periods, generating significant amounts of data and creating challenges for practical CFD workflows, especially with limited memory resources and costly GPU writing penalties. In this work, multiple sparse flow reconstruction (SFR) methods are developed to approximate a full unsteady solution using far fewer sparse measurements, thus reducing writing costs, data storage, and enabling higher sampling rates. SFR is motivated by a large-eddy simulation of rare inlet distortion events, demonstrating that down-sampling full snapshots and supplementing them with high-frequency sparse measurements can drastically cut writing time for GPU solvers and nearly eliminate this cost for CPU solvers. The simplest single-equation "snapshot" SFR method can be compressed further using Proper Orthogonal Decomposition (POD-SFR) or a more efficient double POD-SFR variant. A streaming SFR modification improves reconstruction efficiency when local memory is limited. A sensitivity study evaluates trade-offs between sparse sampling rates and reconstruction accuracy, offering best practices. To offset error of using random sparse measurements, SFR exactly preserves dynamics in key regions by prescribing sparse measurement locations, used here to capture distortion events. Distortion events are evaluated using the conditional space-time proper orthogonal decomposition (CST-POD) to pursue physical insights that characterize the upstream causality at full resolution. A validation study of CST-POD modes confirms SFR effectiveness at retaining the event dynamics with substantial computational and memory savings.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Spencer L. Stahl, Stuart I. Benton. 2024-10-16. Sparse flow reconstruction methods to reduce the costs of analyzing large unsteady datasets. https://arxiv.org/abs/2410.12627

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Comparison of Image Processing Models in Quark Gluon Jet Classification

Quark-gluon discrimination provides a useful test case for studying how different machine-learning architectures learn the spatial structure of QCD radiation. In this work, we compare convolutional neural network (CNN), Vision Transformers (ViT), and hierarchical Swin Transformers using the same three-channel jet-image representation, consisting of charged-particle momentum, neutral-particle momentum, and charged-particle multiplicity from PYTHIA 8 jets. We study their performance for different training-set sizes and fine-tuning configurations, with particular attention to the role of local and global information in the jet images. CNN and Swin models consistently perform better than ViT in the cases studied. Since both CNN and Swin retain a strong local component in their architectures, this suggests that local jet substructure plays an important role in quark-gluon discrimination. The performance of the hierarchical Swin model also suggests that combining local features over larger spatial scales is useful. Block-wise fine-tuning improves the performance of the Transformer models, although the improvement becomes smaller and the training less stable as more blocks are unfrozen. We also find that self-supervised Momentum Contrast (MoCo) pretraining improves the model initialization, particularly when the amount of labeled training data is limited. Based on these observations, we developed a smaller Swin model adopted to the jet-image representation used in this study. It achieves comparable performance with substantially fewer parameters. The results show that it is important to adapt the model architecture and training procedure to the specific input characteristics of High Energy Physics (HEP) data when applying vision models in HEP.

physics.data-an

Online local learning for generative thermodynamic computing

Generative thermodynamic computers turn thermal noise into structured data through Langevin dynamics. We train these systems with a local update at each integration step. The reverse-path Onsager-Machlup objective yields a coupling gradient that is a symmetric sum of local residual-state correlations. We apply this gradient immediately rather than accumulating it over a full trajectory. In digital simulations using MNIST prototypes, online and trajectory-batch training reach similar validation losses on fixed noising paths. Models trained online release less heat on average in all five independently seeded pairs, with both models' parameters held fixed during sampling. Auxiliary classifier and nearest-prototype measures change modestly, while pairwise diversity decreases. The response to noise depends strongly on where the errors enter: independent zero-mean errors in the formed updates produce little heat change over a finite range of noise amplitudes, whereas residual offset and temporal correlation have much larger effects. Storing trained couplings requires substantially less precision than resolving deterministic updates during training. Together, these results establish a local online training method and show how update timing, noise structure, and precision affect generative thermodynamic computing.

physics.data-an

Information-Loss Location Estimation and Curvature-Scale Analysis of Physical Measurements: A Gaussian, Cauchy, and Logistic Neutron-Lifetime Benchmark

We develop and implement a two-step location-scale analysis for repeated physical measurements when the underlying distribution is not known. For a chosen substitute distribution, stationarity of population Kullback-Leibler information loss at fixed scale motivates the location score, which we apply to the observed measurements through empirical cross-entropy. Steiner's curvature rule then defines a companion scale separately. We place Gaussian, Cauchy, and logistic substitutes in one common construction and implement them reproducibly on a physics benchmark. For the logistic substitute, we pair the established bounded location score with a scale defined by the curvature rule rather than by a fitted tuning constant or a scale-likelihood equation. For Gaussian measurements with different quoted uncertainties, an experiment-level curvature convention and a stated variance mapping give an exact algebraic connection to a standard residual-based variance estimate for a weighted fit. Applied to a published 21-measurement neutron-lifetime compilation, the inverse-variance weighted, most frequent value, and logistic locations are 878.689, 881.164, and 882.835 s, respectively. These values are benchmark results of the compared constructions and are not proposed as a new recommended neutron lifetime.

physics.data-an