arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2512.11142

libyt: an In Situ Interface Connecting Simulations with yt, Python, and Jupyter Workflows

Shin-Rong Tsai·Hsi-Yu Schive·Matthew J. Turk
View original source ↗Read source document ↗Export metadata ↓

Abstract

In the exascale computing era, handling and analyzing massive datasets have become extremely challenging. In situ analysis, which processes data during simulation runtime and bypasses costly intermediate disk input and output steps, offers a promising solution. We present libyt (https://github.com/yt-project/libyt), an open-source C library that enables astrophysical simulations to analyze and visualize data in parallel computation with yt or other Python packages. libyt can invoke Python routines automatically or provide interactive entry points via a Python prompt or a Jupyter Notebook. It requires minimal intervention in researchers' workflow, allowing users to reuse job submission scripts and Python routines. We describe libyt's architecture for parallel computing in high-performance computing environments, including its bidirectional connection between simulation codes and Python, and its integration into the Jupyter ecosystem. We detail its methods for reading patch-based adaptive mesh refinement (AMR) simulations and handling in-memory data with minimal overhead, and procedures for yielding data when requested by Python. We describe how libyt maps simulation data to yt frontends, allowing post-processing scripts to be converted into in situ analysis with just two lines of change. We document libyt's API and demonstrate its integration into two astrophysical simulation codes, GAMER and Enzo, using examples including core-collapse supernovae, isolated dwarf galaxies, fuzzy dark matter, the Sod shock tube test, Kelvin-Helmholtz instability, and the AGORA galaxy simulation. Finally, we discuss libyt's performance, limitations related to data redistribution, extensibility, architecture, and comparisons with traditional post-processing approaches.

Explore related subjects

astro-ph.IM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Shin-Rong Tsai, Hsi-Yu Schive, Matthew J. Turk. 2026-04-09. libyt: an In Situ Interface Connecting Simulations with yt, Python, and Jupyter Workflows. https://doi.org/10.3847/1538-4365%2Fae5640

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Observations of Binary Stars with the 1.3-m Devasthal Fast Optical Telescope Using Speckle Interferometry: An Attempt

We present a feasibility study of implementing optical interferometry and speckle techniques with the 1.3-m Devasthal Fast Optical Telescope (DFOT) at Aryabhatta Research Institute of Observational Sciences (ARIES), which is currently dedicated to photometric observations. Using the sCMOS camera at the DFOT backend, we perform interferometric speckle observations of six binary stars. Standard Speckle Interferometry algorithms are applied to analyze the recorded speckle images. While this study does not aim to achieve the diffraction limit of DFOT or address a full science-driven resolution case, it serves as a crucial testbed for instrumentation, data acquisition, and analysis of Speckles with DFOT. Although the individual speckle patterns exhibit significant positional variations from frame to frame, the Speckle Interferometry analysis successfully recovers the characteristic three-peak structure of the wide binaries, allowing their relative separations and position angles to be determined, demonstrating the viability of the approach. The measured angular relative separations of $γ$ Leo, $γ$ Vir, and $ζ$ Her are $4.744'' \pm 0.012''$, $3.387'' \pm 0.027''$, and $1.582'' \pm 0.026''$, respectively, with corresponding projected position angles of $126.94^{\circ} \pm 0.14^\circ$, $171.57^{\circ} \pm 0.33^\circ$, and $81.06^{\circ} \pm 0.91^\circ$. These measurements are consistent with previously reported values in the literature, which provides strong motivation for more systematic observations and future implementation of optical interferometry techniques at meter-class telescopes.

astro-ph.IM↗
arXiv2026

Time-Domain Synthesis of Gravitational-Wave Detector Glitches using Class-Conditional Derivative Generative Adversarial Networks

Gravitational-wave detectors such as LIGO, Virgo, and KAGRA are highly sensitive instruments susceptible to many noise sources. Short-duration transient noise events, known as glitches, pose a particular challenge for data analysis pipelines, as they can mimic or obscure astrophysical signals. We present GlitchGAN, a class-conditional generative model built on the Conditional Derivative GAN (cDVGAN) architecture, capable of synthesizing seven glitch types from LIGO's third observing run (O3) directly in the time domain. GlitchGAN generalizes effectively, learning to reproduce a diverse glitch space consistent with high-quality DeepExtractor reconstructions, and can generate hybrid glitch morphologies by interpolating across its class-conditioning vector. It generates 1000 glitches in under 22 seconds on a CPU, suitable for detector simulations, mock data challenges, and pipeline validation. Synthetic glitches are validated using the Gravity Spy classifier and UMAP embeddings, both showing strong agreement with real data. To probe residual distributional differences, we train a separate holdout GlitchGAN model and use a downstream CNN to distinguish held-out real glitches from synthetic ones: detectability is high in a clean representation but drops substantially once both populations are injected into realistic detector noise, the condition under which they would typically be used. Despite this, GlitchGAN-generated glitches remain practically useful: augmenting real training sets with synthetic samples matches simple duplication of real data when data is abundant, and increasingly outperforms it as real data becomes scarce. Finally, we highlight a limitation of magnitude-only spectrograms: magnitude Q-transform classifiers can confidently misclassify physically unrealistic glitches from less robust models, underscoring the need for validation methods that preserve phase information.

astro-ph.IM↗
arXiv2026

SGN: A python framework for stream-processing pipelines

We present the Stream Graph Navigator (SGN), a lightweight Python framework for building streaming data applications. In SGN, stream-processing pipelines are built by connecting computational components into directed acyclic graphs that run within an event loop. The time-series extension of the SGN library, SGN-TS, introduces signal processing methods to handle time series data. Together, SGN and SGN-TS provide the foundation for SGNL, a matched-filtering gravitational-wave search pipeline, and are being adopted by multiple projects across the low-latency gravitational-wave data analysis infrastructure as an extensible and maintainable framework for future gravitational-wave observations.

astro-ph.IM↗