arXiv subjects

Adithya Kumar

Publications and source records attributed to Adithya Kumar.

3 recordsLinked to original sources

arXiv2026

PRISM: Evaluating POSIX Storage Systems for AI Research Workflows

The rapid advancement of AI research is driven by massive investments in GPU clusters, yet the critical role of storage systems in enabling efficient research workflows is often overlooked. Unlike traditional HPC workloads, AI research prioritizes researcher productivity and ease of iteration. Practitioners rely on POSIX compliant file systems for seamless prototyping, debugging, and experimentation before scaling to specialized storage backends. The primary selection criterion is therefore not peak throughput alone, but rather performance within a POSIX compatible, researcher friendly interface. However, existing benchmarks evaluate storage systems exclusively on peak performance and fail to capture the bursty, heterogeneous IO patterns characteristic of real world AI research - where workflows are dynamic, continuously evolving spanning all stages of research. We introduce PRISM, an evaluation framework that reproduces representative AI research workloads - spanning data ingestion, checkpoint IO, and developer workflows to assess and qualify POSIX storage systems along both usability and performance dimensions on GPU clusters. Using PRISM we were able to compare Lustre and NFS based POSIX storage systems across multiple research workload dimensions and select the appropriate storage solution for different environments. As a specific case study in our environment we observed that a flash backed NFS solution outperformed the flash backed Lustre solution by up to 3x for the distributed checkpoint load usecase which helped us make an informed cluster design

cs.DC↗

arXiv2024

Revisiting Reliability in Large-Scale Machine Learning Research Clusters

Reliability is a fundamental challenge in operating large-scale machine learning (ML) infrastructures, particularly as the scale of ML models and training clusters continues to grow. Despite decades of research on infrastructure failures, the impact of job failures across different scales remains unclear. This paper presents a view of managing two large, multi-tenant ML clusters, providing quantitative analysis, operational experience, and our own perspective in understanding and addressing reliability concerns at scale. Our analysis reveals that while large jobs are most vulnerable to failures, smaller jobs make up the majority of jobs in the clusters and should be incorporated into optimization objectives. We identify key workload properties, compare them across clusters, and demonstrate essential reliability requirements for pushing the boundaries of ML training at scale. We hereby introduce a taxonomy of failures and key reliability metrics, analyze 11 months of data from two state-of-the-art ML environments with 4 million jobs and over 150 million A100 GPU hours. Building on our data, we fit a failure model to project Mean Time to Failure for various GPU scales. We further propose a method to estimate a related metric, Effective Training Time Ratio, as a function of job parameters, and we use this model to gauge the efficacy of potential software mitigations at scale. Our work provides valuable insights and future research directions for improving the reliability of AI supercomputer clusters, emphasizing the need for flexible, workload-agnostic, and reliability-aware infrastructure, system software, and algorithms.

cs.DC↗

arXiv2023

Explainable machine learning to enable high-throughput electrical conductivity optimization and discovery of doped conjugated polymers

The combination of high-throughput experimentation techniques and machine learning (ML) has recently ushered in a new era of accelerated material discovery, enabling the identification of materials with cutting-edge properties. However, the measurement of certain physical quantities remains challenging to automate. Specifically, meticulous process control, experimentation and laborious measurements are required to achieve optimal electrical conductivity in doped polymer materials. We propose a ML approach, which relies on readily measured absorbance spectra, to accelerate the workflow associated with measuring electrical conductivity. The classification model accurately classifies samples with a conductivity > 25 to 100 S/cm, achieving a maximum of 100 % accuracy rate. For the subset of highly conductive samples, we employed a regression model to predict their conductivities, yielding an impressive test R2 value of 0.984. We tested the models with samples of the two highest conductivities (498 and 506 S/cm) and showed that they were able to correctly classify and predict the two extrapolative conductivities at satisfactory levels of errors. The proposed ML-assisted workflow results in an improvement in the efficiency of the conductivity measurements by 89 % of the maximum achievable using our experimental techniques. Furthermore, our approach addressed the common challenge of the lack of explainability in ML models by exploiting bespoke mathematical properties of the descriptors and ML model, allowing us to gain corroborated insights into the spectral influences on conductivity. Through this study, we offer an accelerated pathway for optimizing the properties of doped polymer materials while showcasing the valuable insights that can be derived from purposeful utilization of ML in experimental science.

physics.app-ph↗