arXiv subjects

Aditya Prakash

Publications and source records attributed to Aditya Prakash.

At least 19 recordsLinked to original sources

arXiv2026

Multi-Contact Force Estimation for Continuum Robots via Gaussian-Parameterized Factor Graphs

Continuum robots offer key advantages in navigating unstructured environments, but their safe operation requires accurate estimation of the external contact forces acting anywhere along the robot body. Estimating these forces at unknown locations is an ill-conditioned problem, particularly for multiple contacts. We propose a unified shape and force estimation framework formulated on a factor graph. By incorporating a Gaussian mixture force parameterization into a discretized probabilistic Cosserat rod model, we reduce the dimensionality of the unknown external forces and mitigate the ill-conditioning of node-wise force estimation. The framework fuses strain, tendon tension, and pose measurements to simultaneously estimate the robot's shape and external forces while accounting for modeling and sensor uncertainties. Numerical simulations demonstrate that the proposed method outperforms existing methods in terms of force location and magnitude estimation for both single and multi-contact scenarios. We further present a progressive variant that introduces basis functions on demand to estimate contact forces sequentially during a simulated confined-navigation task.

cs.RO↗

arXiv2026

What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?

Human video datasets used for cotraining robot manipulation policies largely consist of curated demonstrations where motions are orchestrated to resemble robot behavior and 3D hand poses are captured with specialized hardware. A more plentiful source of data is everyday Internet video, but it is an open question what factors enable transfer from such videos to robots. We investigate this using a new dataset of 532 human videos with 28 hours of high-quality triangulated hand labels and natural motions. We find that hand pose quality affects transfer, but even with accurate hands, the inherent motion gap hinders transfer unless the vision and policy networks specialize to each embodiment. Our cotraining recipe yields consistent improvements, with an absolute success rate gain of $29.7\%$ in the low-robot-data regime across six manipulation tasks.

cs.RO↗

arXiv2026

QShield: Securing Neural Networks Against Adversarial Attacks using Quantum Circuits

Deep neural networks remain highly vulnerable to adversarial perturbations, limiting their reliability in security- and safety-critical applications. To address this challenge, we introduce QShield, a modular hybrid quantum-classical neural network (HQCNN) architecture designed to enhance the adversarial robustness of classical deep learning models. QShield integrates a conventional convolutional neural network (CNN) backbone for feature extraction with a quantum processing module that encodes the extracted features into quantum states, applies structured entanglement operations under realistic noise models, and outputs a hybrid prediction through a dynamically weighted fusion mechanism implemented via a lightweight multilayer perceptron (MLP). We systematically evaluate both classical and hybrid quantum-classical models on the MNIST, OrganAMNIST, and CIFAR-10 datasets, using a comprehensive set of robustness, efficiency, and computational performance metrics. Our results demonstrate that classical models are highly vulnerable to adversarial attacks, whereas the proposed hybrid models with entanglement patterns maintain high predictive accuracy while substantially reducing attack success rates across a wide range of adversarial attacks. Furthermore, the proposed hybrid architecture significantly increased the computational cost required to generate adversarial examples, thereby introducing an additional layer of defense. These findings indicate that the proposed modular hybrid architecture achieves a practical balance between predictive accuracy and adversarial robustness, positioning it as a promising approach for secure and reliable machine learning in sensitive and safety-critical applications.

cs.CR↗

arXiv2025

Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Images

We tackle the problem of forecasting bimanual 3D hand motion & articulation from a single image in everyday settings. To address the lack of 3D hand annotations in diverse settings, we design an annotation pipeline consisting of a diffusion model to lift 2D hand keypoint sequences to 4D hand motion. For the forecasting model, we adopt a diffusion loss to account for the multimodality in hand motion distribution. Extensive experiments across 6 datasets show the benefits of training on diverse data with imputed labels (14% improvement) and effectiveness of our lifting (42% better) & forecasting (16.4% gain) models, over the best baselines, especially in zero-shot generalization to everyday images.

cs.CV↗

arXiv2025

Long-Range Blockade Between Counter-Propagating Photons

Realizing strong interactions between individual photons, mediated by matter, is a cornerstone for advancing photonic quantum computing and quantum nonlinear optics. In such systems, the interaction range is typically limited by the narrow bandwidth of the material excitation, resulting in a tradeoff between interaction strength and pulse duration. Here, we address this limitation by exploring interactions between counter-propagating photons mediated by Rydberg polaritons. We experimentally demonstrate strong photon-photon interactions, achieving a record-long anti-correlation range exceeding $1~\mu s$. This extended range enables photon pulses that are long enough to fit within the polariton bandwidth, yet short enough to remain within the interaction range. Under these conditions, we observe complete photon blockade of entire pulses, tunable via their relative timing. Extending to the three-photon regime, we observe enhanced interactions when a photon encounters a counter-propagating pair. These results, supported by analytical theory and numerical simulations, establish counter-propagating Rydberg polaritons as a powerful platform for engineering interactions in quantum light fields, enabling deterministic photonic operations and opening new directions in few-photon quantum dynamics.

quant-ph↗

arXiv2025

How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions

We tackle the novel problem of predicting 3D hand motion and contact maps (or Interaction Trajectories) given a single RGB view, action text, and a 3D contact point on the object as input. Our approach consists of (1) Interaction Codebook: a VQVAE model to learn a latent codebook of hand poses and contact points, effectively tokenizing interaction trajectories, (2) Interaction Predictor: a transformer-decoder module to predict the interaction trajectory from test time inputs by using an indexer module to retrieve a latent affordance from the learned codebook. To train our model, we develop a data engine that extracts 3D hand poses and contact trajectories from the diverse HoloAssist dataset. We evaluate our model on a benchmark that is 2.5-10X larger than existing works, in terms of diversity of objects and interactions observed, and test for generalization of the model across object categories, action categories, tasks, and scenes. Experimental results show the effectiveness of our approach over transformer & diffusion baselines across all settings.

cs.CV↗

arXiv2025

Weakly Supervised Learning on Large Graphs

Graph classification plays a pivotal role in various domains, including pathology, where images can be represented as graphs. In this domain, images can be represented as graphs, where nodes might represent individual nuclei, and edges capture the spatial or functional relationships between them. Often, the overall label of the graph, such as a cancer type or disease state, is determined by patterns within smaller, localized regions of the image. This work introduces a weakly-supervised graph classification framework leveraging two subgraph extraction techniques: (1) Sliding-window approach (2) BFS-based approach. Subgraphs are processed using a Graph Attention Network (GAT), which employs attention mechanisms to identify the most informative subgraphs for classification. Weak supervision is achieved by propagating graph-level labels to subgraphs, eliminating the need for detailed subgraph annotations.

cs.LG↗

arXiv2024

Frequency-selective amplification of nonlinear response in strongly correlated bosons

We present a protocol to generate enhanced non-linear responses of incident pulses in the density wave phase within the extended Bose-Hubbard model using the concept of resonance-induced amplification (RIA). This method enables the selection of an incident pulse frequency to amplify the desired harmonic order. We characterize the enhancement of the non-linear harmonic spectra under various frequencies and field strengths of the incident pulses, and demonstrate that an optimal field strength is necessary to realize our protocol.

cond-mat.str-el↗

arXiv2024

Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects

We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the head movement. To this end, we designed the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks. Our analysis demonstrates the effectiveness of addressing distortion specific to egocentric cameras, adopting high-capacity transformers to learn complex hand-object interactions, and fusing predictions from different views. Our study further reveals challenging scenarios intractable with state-of-the-art methods, such as fast hand motion, object reconstruction from narrow egocentric views, and close contact between two hands and objects. Our efforts will enrich the community's knowledge foundation and facilitate future hand studies on egocentric hand-object interactions.

cs.CV↗

arXiv2023

3D Hand Pose Estimation in Everyday Egocentric Images

3D hand pose estimation in everyday egocentric images is challenging for several reasons: poor visual signal (occlusion from the object of interaction, low resolution & motion blur), large perspective distortion (hands are close to the camera), and lack of 3D annotations outside of controlled settings. While existing methods often use hand crops as input to focus on fine-grained visual information to deal with poor visual signal, the challenges arising from perspective distortion and lack of 3D annotations in the wild have not been systematically studied. We focus on this gap and explore the impact of different practices, i.e. crops as input, incorporating camera information, auxiliary supervision, scaling up datasets. We provide several insights that are applicable to both convolutional and transformer models leading to better performance. Based on our findings, we also present WildHands, a system for 3D hand pose estimation in everyday egocentric images. Zero-shot evaluation on 4 diverse datasets (H2O, AssemblyHands, Epic-Kitchens, Ego-Exo4D) demonstrate the effectiveness of our approach across 2D and 3D metrics, where we beat past methods by 7.4% - 66%. In system level comparisons, WildHands achieves the best 3D hand pose on ARCTIC egocentric split, outperforms FrankMocap across all metrics and HaMeR on 3 out of 6 metrics while being 10x smaller and trained on 5x less data.

cs.CV↗

arXiv2023

Mitigating Perspective Distortion-induced Shape Ambiguity in Image Crops

Objects undergo varying amounts of perspective distortion as they move across a camera's field of view. Models for predicting 3D from a single image often work with crops around the object of interest and ignore the location of the object in the camera's field of view. We note that ignoring this location information further exaggerates the inherent ambiguity in making 3D inferences from 2D images and can prevent models from even fitting to the training data. To mitigate this ambiguity, we propose Intrinsics-Aware Positional Encoding (KPE), which incorporates information about the location of crops in the image and camera intrinsics. Experiments on three popular 3D-from-a-single-image benchmarks: depth prediction on NYU, 3D object detection on KITTI & nuScenes, and predicting 3D shapes of articulated objects on ARCTIC, show the benefits of KPE.

cs.CV↗

arXiv2023

A protocol to characterize errors in quantum simulation of many-body physics

Quantum simulation of many-body systems, particularly using ultracold atoms and trapped ions, presents a unique form of quantum control -- it is a direct implementation of a multi-qubit gate generated by the Hamiltonian. As a consequence, it also faces a unique challenge in terms of benchmarking, because the well-established gate benchmarking techniques are unsuitable for this form of quantum control. Here we show that the symmetries of the target many-body Hamiltonian can be used to benchmark and characterize experimental errors in the quantum simulation. We consider two forms of errors: (i) unitary errors arising out of systematic errors in the applied Hamiltonian and (ii) canonical non-Markovian errors arising out of random shot-to-shot fluctuations in the applied Hamiltonian. We show that the dynamics of the expectation value of the target Hamiltonian itself, which is ideally constant in time, can be used to characterize these errors. In the presence of errors, the expectation value of the target Hamiltonian shows a characteristic thermalization dynamics, when it satisfies the operator thermalization hypothesis (OTH). That is, an oscillation in the short time followed by relaxation to a steady-state value in the long time limit. We show that while the steady-state value can be used to characterize the coherent errors, the amplitude of the oscillations can be used to estimate the non-Markovian errors. We develop scalable experimental protocols to characterize these errors.

cond-mat.quant-gas↗

arXiv2023

Look Ma, No Hands! Agent-Environment Factorization of Egocentric Videos

The analysis and use of egocentric videos for robotic tasks is made challenging by occlusion due to the hand and the visual mismatch between the human hand and a robot end-effector. In this sense, the human hand presents a nuisance. However, often hands also provide a valuable signal, e.g. the hand pose may suggest what kind of object is being held. In this work, we propose to extract a factored representation of the scene that separates the agent (human hand) and the environment. This alleviates both occlusion and mismatch while preserving the signal, thereby easing the design of models for downstream robotics tasks. At the heart of this factorization is our proposed Video Inpainting via Diffusion Model (VIDM) that leverages both a prior on real-world images (through a large-scale pre-trained diffusion model) and the appearance of the object in earlier frames of the video (through attention). Our experiments demonstrate the effectiveness of VIDM at improving inpainting quality on egocentric videos and the power of our factored representation for numerous tasks: object detection, 3D reconstruction of manipulated objects, and learning of reward functions, policies, and affordances from videos.

cs.CV↗

arXiv2023

3D Reconstruction of Objects in Hands without Real World 3D Supervision

Prior works for reconstructing hand-held objects from a single image train models on images paired with 3D shapes. Such data is challenging to gather in the real world at scale. Consequently, these approaches do not generalize well when presented with novel objects in in-the-wild settings. While 3D supervision is a major bottleneck, there is an abundance of a) in-the-wild raw video data showing hand-object interactions and b) synthetic 3D shape collections. In this paper, we propose modules to leverage 3D supervision from these sources to scale up the learning of models for reconstructing hand-held objects. Specifically, we extract multiview 2D mask supervision from videos and 3D shape priors from shape collections. We use these indirect 3D cues to train occupancy networks that predict the 3D shape of objects from a single RGB image. Our experiments in the challenging object generalization setting on in-the-wild MOW dataset show 11.6% relative improvement over models trained with 3D supervision on existing datasets.

cs.CV↗

arXiv2022

Impurity reveals distinct operational phases in quantum thermodynamic cycles

We analyze the effect of impurity on the work output and efficiency of quantum Otto and quantum Carnot heat cycles, modeled as a single quantum particle in an infinite square well (ISW) potential, which is the working substance. We solve this quantum mechanical system perturbatively up to first and second order in strength of the impurity for strong and weak coupling regimes, respectively. We derive the analytical expressions of work and efficiency for the strong coupling regime to the first order in the strength parameter. The threshold value of the strength parameter in weak coupling is obtained up to which the numerical result agrees with the perturbative result for a repulsive and attractive impurity. To our surprise, an embedded impurity unlocks new operational phases in the system, such as a quantum heat engine, quantum refrigerator, and quantum cold pump. In addition, the efficiency of the quantum Otto heat engine is seen to reach Carnot efficiency for some parameter regimes. The cooling power and coefficient of performance of the quantum refrigerator and quantum cold pump are non-trivially affected by the impurity.

quant-ph↗

arXiv2022

TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving

How should we integrate representations from complementary sensors for autonomous driving? Geometry-based fusion has shown promise for perception (e.g. object detection, motion forecasting). However, in the context of end-to-end driving, we find that imitation learning based on existing sensor fusion methods underperforms in complex driving scenarios with a high density of dynamic agents. Therefore, we propose TransFuser, a mechanism to integrate image and LiDAR representations using self-attention. Our approach uses transformer modules at multiple resolutions to fuse perspective view and bird's eye view feature maps. We experimentally validate its efficacy on a challenging new benchmark with long routes and dense traffic, as well as the official leaderboard of the CARLA urban driving simulator. At the time of submission, TransFuser outperforms all prior work on the CARLA leaderboard in terms of driving score by a large margin. Compared to geometry-based fusion, TransFuser reduces the average collisions per kilometer by 48%.

cs.CV↗

arXiv2021

NEAT: Neural Attention Fields for End-to-End Autonomous Driving

Efficient reasoning about the semantic, spatial, and temporal structure of a scene is a crucial prerequisite for autonomous driving. We present NEural ATtention fields (NEAT), a novel representation that enables such reasoning for end-to-end imitation learning models. NEAT is a continuous function which maps locations in Bird's Eye View (BEV) scene coordinates to waypoints and semantics, using intermediate attention maps to iteratively compress high-dimensional 2D image features into a compact representation. This allows our model to selectively attend to relevant regions in the input while ignoring information irrelevant to the driving task, effectively associating the images with the BEV representation. In a new evaluation setting involving adverse environmental conditions and challenging scenarios, NEAT outperforms several strong baselines and achieves driving scores on par with the privileged CARLA expert used to generate its training data. Furthermore, visualizing the attention maps for models with NEAT intermediate representations provides improved interpretability.

cs.CV↗

arXiv2021

Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

How should representations from complementary sensors be integrated for autonomous driving? Geometry-based sensor fusion has shown great promise for perception tasks such as object detection and motion forecasting. However, for the actual driving task, the global context of the 3D scene is key, e.g. a change in traffic light state can affect the behavior of a vehicle geometrically distant from that traffic light. Geometry alone may therefore be insufficient for effectively fusing representations in end-to-end driving models. In this work, we demonstrate that imitation learning policies based on existing sensor fusion methods under-perform in the presence of a high density of dynamic agents and complex scenarios, which require global contextual reasoning, such as handling traffic oncoming from multiple directions at uncontrolled intersections. Therefore, we propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. We experimentally validate the efficacy of our approach in urban settings involving complex scenarios using the CARLA urban driving simulator. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.

cs.CV↗