arXiv subjects

Swati Jain

Publications and source records attributed to Swati Jain.

5 recordsLinked to original sources

arXiv2026

Skill-based Agentic Evaluation for Real-time Data Science Tasks

We present a framework for evaluating data-science agents on live, continuously updated data using executable ground truth and format-agnostic factoid scoring. Consider this example query: "what were last week's audience sizes"---the reference answer changes as the underlying data changes, so static references become outdated and standard LLM-as-a-judge pipelines cannot verify responses against a fixed ground truth. Our central contribution, ground-truth-as-code, encodes each expected answer as an executable reference function that recomputes the answer directly from live data at evaluation time, ensuring the reference remains consistent with the system it describes. We combine this with a factoid-level, format-agnostic judge that decomposes both the agent's response and the computed ground truth into atomic claims and scores precision, recall, and accuracy over them, irrespective of the response format (prose, list, table, HTML, etc.). The approach is applicable to agents whose expected outputs can be expressed as executable data computations. We validate the framework through a human--LLM agreement study on an internally developed machine learning skill deployed in production, using a synthetic database constructed to reproduce production schemas and entity relationships. Relative to a natural-language ground-truth baseline, our method achieves a 29% improvement in the Matthews Correlation Coefficient (MCC)---a class-balanced measure of agreement between expert annotators and LLM-as-a-judge predictions---and a 16% reduction in token consumption per test case, while a self-directed baseline lacking explicit ground truth is anti-correlated with human judgment. Agents that perform multi-source data integration and computation over non-stationary data are routinely deployed in industry; we propose ground-truth-as-code as a practical methodology for their evaluation.

cs.AI↗

arXiv2026

Human-LLM Collaborative Feature Engineering for Tabular Data

Large language models (LLMs) are increasingly used to automate feature engineering in tabular learning. Given task-specific information, LLMs can propose diverse feature transformation operations to enhance downstream model performance. However, current approaches typically assign the LLM as a black-box optimizer, responsible for both proposing and selecting operations based solely on its internal heuristics, which often lack calibrated estimations of operation utility and consequently lead to repeated exploration of low-yield operations without a principled strategy for prioritizing promising directions. In this paper, we propose a human-LLM collaborative feature engineering framework for tabular learning. We begin by decoupling the transformation operation proposal and selection processes, where LLMs are used solely to generate operation candidates, while the selection is guided by explicitly modeling the utility and uncertainty of each proposed operation. Since accurate utility estimation can be difficult especially in the early rounds of feature engineering, we design a mechanism within the framework that selectively elicits and incorporates human expert preference feedback, comparing which operations are more promising, into the selection process to help identify more effective operations. Our evaluations on both the synthetic study and the real user study demonstrate that the proposed framework improves feature engineering performance across a variety of tabular datasets and reduces users' cognitive load during the feature engineering process.

cs.LG↗

arXiv2020

Using LSTM for the Prediction of Disruption in ADITYA Tokamak

Major disruptions in tokamak pose a serious threat to the vessel and its surrounding pieces of equipment. The ability of the systems to detect any behavior that can lead to disruption can help in alerting the system beforehand and prevent its harmful effects. Many machine learning techniques have already been in use at large tokamaks like JET and ASDEX, but are not suitable for ADITYA, which is comparatively small. Through this work, we discuss a new real-time approach to predict the time of disruption in ADITYA tokamak and validate the results on an experimental dataset. The system uses selected diagnostics from the tokamak and after some pre-processing steps, sends them to a time-sequence Long Short-Term Memory (LSTM) network. The model can make the predictions 12 ms in advance at less computation cost that is quick enough to be deployed in real-time applications.

cs.LG↗

arXiv2018

Partial synchronization of relaxation oscillators with repulsive coupling in autocatalytic integrate-and-fire model and electrochemical experiments

Experiments and supporting theoretical analysis is presented to describe the synchronization patterns that can be observed with a population of globally coupled electrochemical oscillators close to a homoclinic, saddle-loop bifurcation, where the coupling is repulsive in the electrode potential. While attractive coupling generates phase clusters and desynchronized states, repulsive coupling results in synchronized oscillations. The experiments are interpreted with a phenomenological model that captures the waveform of the oscillations (exponential increase) followed by a refractory period. The globally coupled autocatalytic integrate-and-fire model predicts the development of partially synchronized states that occur through attracting heteroclinic cycles between out-of-phase two-cluster states. Similar behavior can be expected in many other systems where the oscillations occur close to a saddle-loop bifurcation, e.g., with Morris-Lecar neurons.

nlin.AO↗

arXiv2014

Clustering in Globally Coupled Oscillators Near a Hopf Bifurcation: Theory and Experiments

A theoretical analysis is presented to show the general occurrence of phase clusters in weakly, globally coupled oscillators close to a Hopf bifurcation. Through a reductive perturbation method, we derive the amplitude equation with a higher order correction term valid near a Hopf bifurcation point. This amplitude equation allows us to calculate analytically the phase coupling function from given limit-cycle oscillator models. Moreover, using the phase coupling function, the stability of phase clusters can be analyzed. We demonstrate our theory with the Brusselator model. Experiments are carried out to confirm the presence of phase clusters close to Hopf bifurcations with electrochemical oscillators.

nlin.AO↗