arXiv subjects

Matthias Lieber

Publications and source records attributed to Matthias Lieber.

4 recordsLinked to original sources

arXiv2026

JuPyLive: Seamless Migration of Jupyter Notebook Resources from Laptop to HPC

This work introduces JuPyLive, a migration mechanism that enables seamless transition of Jupyter notebooks between local resources of user's workstation and remote resources of high-performance computing~(HPC) environments, while preserving the user experience. JuPyLive eliminates the underlying complexities of migration process, enabling users to freely choose among available local and remote resources, directly within the familiar Jupyter notebook environment via a single click. JuPyLive leverages ElasticNotebook to manage in-memory state migration, it automates resource allocation on HPC cluster and orchestrates required remote communication channels between the source and destination to enable a bidirectional migration. Furthermore, HPC status monitor of JuPyLive provides a live overview of available remote resources, allowing users to make informed decisions on choosing the relevant resources before initiating a migration process. The proposed fully automatic mechanism requires no code changes or configurations by the end user, nor does it demand users to learn a new syntax, instead the migration process can be intuitively initiated and monitored using visual elements from within the Jupyter notebook. By bridging the gap between local workspace and remote resources, JuPyLive offers a seamless experience for scaling local resource-intensive workflows with minimal user intervention, thus further democratizing the usage of HPC clusters among the interdisciplinary researchers.

cs.DC↗

arXiv2026

RASER: Resilient Agent Scheduling and Execution Runtime for HPC Clusters

The emergence of modern agents powered by large language models has created a demand for executing long-horizon, autonomous workflows in various domains that require significant computational resources. While High Performance Computing clusters provide the ideal infrastructure for these computation-intensive workloads, traditional HPC job schedulers such as Slurm are not designed for dynamic, agentic workflows characterized by unpredictable task durations, external API calls, and fault tolerance requirements of modern agents. This work presents RASER, a user-space framework that enables seamless execution of agentic workflows on production HPC clusters by extending Slurm's internal primitives. RASER introduces agentic job arrays with work stealing via shared filesystem queues, user-space checkpointing through application-level state serialization combined with Slurm requeue, and Apptainer container-based isolation without requiring any image modifications. Evaluations demonstrate that RASER reduces makespan by nearly 39% compared to static partitioning while achieving near-full CPU utilization. RASER provides resilience against preemption and failures while maintaining minimal checkpoint/restore overhead. It requires no kernel privileges or external database infrastructure, making it an accessible solution for deploying agentic workflows on existing HPC infrastructure.

cs.PF↗

arXiv2025

Seamless Transitions: A Comprehensive Review of Live Migration Technologies

Live migration, a technology enabling seamless transition of operational computational entities between various hosts while preserving continuous functionality and client connectivity, has been the subject of extensive research. However, existing reviews often overlook critical technical aspects and practical challenges integral to the usage of live migration techniques in real-world scenarios. This work bridges this gap by integrating the aspects explored in existing reviews together with a comprehensive analysis of live migration technologies across multiple dimensions, with focus on migration techniques, migration units, and infrastructure characteristics. Despite efforts to make live migration widely accessible, its reliance on multiple system factors can create challenges. In certain cases, the complexities and resource demands outweigh the benefits, making its implementation hard to justify. The focus of this work is mainly on container based and virtual machine-based migration technologies, examining the current state of the art and the disparity in adoption between these two approaches. Furthermore, this work explores the impact of migration objectives and operational constraints on the usability and efficacy of existing technologies. By outlining current technical challenges and providing guidelines for future research and development directions, this work serves a dual purpose: first, to equip enthusiasts with a valuable resource on live migration, and second, to contribute to the advancement of live migration technologies and their practical implementation across diverse computing environments.

cs.DC↗

arXiv2024

Towards Computational Performance Engineering for Unsupervised Concept Drift Detection -- Complexities, Benchmarking, Performance Analysis

Concept drift detection is crucial for many AI systems to ensure the system's reliability. These systems often have to deal with large amounts of data or react in real-time. Thus, drift detectors must meet computational requirements or constraints with a comprehensive performance evaluation. However, so far, the focus of developing drift detectors is on inference quality, e.g. accuracy, but not on computational performance, such as runtime. Many of the previous works consider computational performance only as a secondary objective and do not have a benchmark for such evaluation. Hence, we propose and explain performance engineering for unsupervised concept drift detection that reflects on computational complexities, benchmarking, and performance analysis. We provide the computational complexities of existing unsupervised drift detectors and discuss why further computational performance investigations are required. Hence, we state and substantiate the aspects of a benchmark for unsupervised drift detection reflecting on inference quality and computational performance. Furthermore, we demonstrate performance analysis practices that have proven their effectiveness in High-Performance Computing, by tracing two drift detectors and displaying their performance data.

cs.LG↗