arXiv subjects

Robert Jacob

Publications and source records attributed to Robert Jacob.

6 recordsLinked to original sources

arXiv2026

An offline approach to fNIRS-guided reinforcement learning for robot behavior

Human-in-the-loop Reinforcement Learning has become a popular approach for training, finetuning, and aligning robot behavior with user preferences. Our paper explores the feasibility of using brain signals via functional near-infrared spectroscopy (fNIRS) to modulate robot learning in simulation. We compare agents trained on passive (observational) versus active (demonstrative) interaction tasks, and test multiple methods for enhancing the RL algorithm with the neural signal, focusing on parameter augmentation in contrast to replacement. We further examine how model granularity and noise affect agent learning. Our results show that this framework is effective. The neural signal improves learning when augmenting trajectory priorities and state-action q-targets. Additionally, the framework learns successfully from offline data, offering a practical alternative for settings where real-time BCI setups are impractical or only limited data is available.

cs.RO↗

arXiv2025

Towards Reinforcement Learning from Neural Feedback: Mapping fNIRS Signals to Agent Performance

Reinforcement Learning from Human Feedback (RLHF) is a methodology that aligns agent behavior with human preferences by integrating user feedback into the agent's training process. This paper introduces a framework that guides agent training through implicit neural signals, with a focus on the neural classification problem. Our work presents and releases a novel dataset of functional near-infrared spectroscopy (fNIRS) recordings collected from 25 human participants across three domains: Pick-and-Place Robot, Lunar Lander, and Flappy Bird. We train multiple classifiers to predict varying levels of agent performance (optimal, suboptimal, or worst-case) from windows of preprocessed fNIRS features, achieving an average F1 score of 67% for binary and 46% for multi-class classification across conditions and domains. We also train multiple regressors to predict the degree of deviation between an agent's chosen action and a set of near-optimal policy actions, providing a continuous measure of performance. Finally, we evaluate cross-subject generalization and show that fine-tuning pre-trained models with a small sample of subject-specific data increases average F1 scores by 17% and 41% for binary and multi-class models, respectively. Our results demonstrate that mapping implicit fNIRS signals to agent performance is feasible and can be improved, laying the foundation for future Reinforcement Learning from Neural Feedback (RLNF) systems.

cs.AI↗

arXiv2025

Multi-messenger dynamic imaging of laser-driven shocks in water using a plasma wakefield accelerator

Understanding dense matter hydrodynamics is critical for predicting plasma behavior in environments relevant to laser-driven inertial confinement fusion. Traditional diagnostic sources face limitations in brightness, spatiotemporal resolution, and inability to detect relevant electromagnetic fields. In this work, we present a dual-probe, multi-messenger laser wakefield accelerator platform combining ultrafast X-rays and relativistic electron beams at 1 Hz, to interrogate a free-flowing water target in vacuum, heated by an intense 200 ps laser pulse. This scheme enables high-repetition-rate tracking of the interaction evolution using both particle types. Betatron X-rays reveal a cylindrically symmetric shock compression morphology assisted by low-density vapor, resembling foam-layer-assisted fusion targets. The synchronized electron beam detects time-evolving electromagnetic fields, uncovering charge separation and ion species differentiation during plasma expansion - phenomena not captured by photons or hydrodynamic simulations. We show that combining both probes provides complementary insights spanning kinetic to hydrodynamic regimes, highlighting the need for hybrid physics models to accurately predict fusion-relevant plasma behavior

physics.plasm-ph↗

arXiv2025

Mapping Neural Signals to Agent Performance, A Step Towards Reinforcement Learning from Neural Feedback

Implicit Human-in-the-Loop Reinforcement Learning (HITL-RL) is a methodology that integrates passive human feedback into autonomous agent training while minimizing human workload. However, existing methods often rely on active instruction, requiring participants to teach an agent through unnatural expression or gesture. We introduce NEURO-LOOP, an implicit feedback framework that utilizes the intrinsic human reward system to drive human-agent interaction. This work demonstrates the feasibility of a critical first step in the NEURO-LOOP framework: mapping brain signals to agent performance. Using functional near-infrared spectroscopy (fNIRS), we design a dataset to enable future research using passive Brain-Computer Interfaces for Human-in-the-Loop Reinforcement Learning. Participants are instructed to observe or guide a reinforcement learning agent in its environment while signals from the prefrontal cortex are collected. We conclude that a relationship between fNIRS data and agent performance exists using classical machine learning techniques. Finally, we highlight the potential that neural interfaces may offer to future applications of human-agent interaction, assistive AI, and adaptive autonomous systems.

cs.LG↗

arXiv2024

Spatiotemporally adaptive compression for scientific dataset with feature preservation -- a case study on simulation data with extreme climate events analysis

Scientific discoveries are increasingly constrained by limited storage space and I/O capacities. For time-series simulations and experiments, their data often need to be decimated over timesteps to accommodate storage and I/O limitations. In this paper, we propose a technique that addresses storage costs while improving post-analysis accuracy through spatiotemporal adaptive, error-controlled lossy compression. We investigate the trade-off between data precision and temporal output rates, revealing that reducing data precision and increasing timestep frequency lead to more accurate analysis outcomes. Additionally, we integrate spatiotemporal feature detection with data compression and demonstrate that performing adaptive error-bounded compression in higher dimensional space enables greater compression ratios, leveraging the error propagation theory of a transformation-based compressor. To evaluate our approach, we conduct experiments using the well-known E3SM climate simulation code and apply our method to compress variables used for cyclone tracking. Our results show a significant reduction in storage size while enhancing the quality of cyclone tracking analysis, both quantitatively and qualitatively, in comparison to the prevalent timestep decimation approach. Compared to three state-of-the-art lossy compressors lacking feature preservation capabilities, our adaptive compression framework improves perfectly matched cases in TC tracking by 26.4-51.3% at medium compression ratios and by 77.3-571.1% at large compression ratios, with a merely 5-11% computational overhead.

cs.CV↗

arXiv2019

Stability Analysis of Interface Conditions for Ocean-Atmosphere Coupling

In this paper we analyze the stability of different coupling strategies for multidomain PDEs that arise in general circulation models used in climate simulations. We focus on fully coupled ocean-atmosphere models that are needed to represent and understand the complicated interactions of these two systems, becoming increasingly important in climate change assessment in recent years. Numerical stability issues typically arise because of different time-stepping strategies applied to the coupled PDE system. In particular, the contributing factors include using large time steps, lack of accurate interface flux, and singe-iteration coupling. We investigate the stability of the coupled ocean-atmosphere models for various interface conditions such as the Dirichlet-Neumann condition and the bulk interface condition, which is unique to climate modeling. By analyzing a simplified model, we demonstrate how the parameterization of the bulk condition and other numerical and physical parameters affect the coupling stability.

math.NA↗