arXiv subjects

Philip Wang

Publications and source records attributed to Philip Wang.

6 recordsLinked to original sources

arXiv2026

MANIGUARD: A Benchmark and Data Suite for Specification-Grounded Safety Evaluation and Improvement of Robotic Manipulation

Foundation-model policies for robotic manipulation are advancing rapidly on task success, but rigorous evaluation of whether they succeed safely is still lacking. We introduce ManiGuard, a specification-grounded framework for evaluating and improving the safety of foundation-model manipulation, comprising the ManiGuard-Bench task suite and a paired safety-annotated trajectory-generation pipeline. ManiGuard-Bench organizes six contact-rich household task families into 200 locked base tasks along a skill $\times$ constraint taxonomy, with safety specified independently of task success. Each task is evaluated under one in-distribution and four single-axis out-of-distribution perturbations that hold the safety specification fixed, giving 1,000 locked scenarios. Every rollout is runtime-checked by LTL$_f$-grounded automaton monitors over physics-grounded predicates rather than learned classifiers or LLM judges, in simulation and on a physical Franka platform. The pipeline pairs an automated motion-planning generator with human teleoperation, annotated by the same per-step monitor, and directly supports safety-aware fine-tuning; we release 8,000 safety-annotated demonstrations, 40 per base task. Benchmarking zero-shot and fine-tuned VLAs across more than 23,000 rollouts, we find: (i) safety must be evaluated independently of task success, as 6-21% of successful rollouts violate the specification; (ii) fine-tuning on our suite raises safe task completion from near zero to 7.5-29.8% and engaged-and-safe behavior from 16-40% to 51-72%; but (iii) a gap remains that scaling demonstrations does not close, with 21-42% of engaged rollouts still violating, two of six families below 2% safe success for every policy, and these failures persisting under distribution shift and on hardware.

cs.RO↗

arXiv2025

SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents

We present SENTINEL, a framework for formally evaluating the physical safety of foundation model (FM)-based embodied agents. SENTINEL is the first to provide multi-level safety evaluation across semantic interpretation, plan generation, and physical execution within a unified formal framework. Unlike prior methods that rely on heuristic rules or subjective FM judgments, SENTINEL grounds practical safety requirements in formal temporal logic (TL) semantics that can precisely specify state invariants, temporal dependencies, and timing constraints. It employs a multi-level verification pipeline where (i) at the semantic level, intuitive natural language safety requirements are formalized into TL formulas and the agent's understanding of these requirements is probed for alignment with the TL formulas; (ii) at the plan level, high-level action plans and subgoals generated by the agent are verified against the TL formulas to detect unsafe plans before execution; and (iii) at the trajectory level, multiple execution trajectories are merged into a computation tree and efficiently verified against physically-detailed TL specifications for a final safety check. We apply SENTINEL in VirtualHome and AI2-THOR, and formally evaluate multiple FM-based embodied agents against diverse safety requirements. Our experiments show that by grounding physical safety in temporal logic and applying verification methods across multiple levels, SENTINEL provides a rigorous foundation for systematically evaluating the safety of FM-based embodied agents in simulation-based physical environments, and can effectively expose potential safety violations in interpreting, planning, and executing the tasks.

cs.AI↗

arXiv2025

Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization

Offline-to-online deployment of reinforcement-learning (RL) agents must bridge two gaps: (1) the sim-to-real gap, where real systems add latency and other imperfections not present in simulation, and (2) the interaction gap, where policies trained purely offline face out-of-distribution states during online execution because gathering new interaction data is costly or risky. Agents therefore have to generalize from static, delay-free datasets to dynamic, delay-prone environments. Standard offline RL learns from delay-free logs yet must act under delays that break the Markov assumption and hurt performance. We introduce DT-CORL (Delay-Transformer belief policy Constrained Offline RL), an offline-RL framework built to cope with delayed dynamics at deployment. DT-CORL (i) produces delay-robust actions with a transformer-based belief predictor even though it never sees delayed observations during training, and (ii) is markedly more sample-efficient than na\"ive history-augmentation baselines. Experiments on D4RL benchmarks with several delay settings show that DT-CORL consistently outperforms both history-augmentation and vanilla belief-based methods, narrowing the sim-to-real latency gap while preserving data efficiency.

cs.LG↗

arXiv2024

Inverse Delayed Reinforcement Learning

Inverse Reinforcement Learning (IRL) has demonstrated effectiveness in a variety of imitation tasks. In this paper, we introduce an IRL framework designed to extract rewarding features from expert trajectories affected by delayed disturbances. Instead of relying on direct observations, our approach employs an efficient off-policy adversarial training framework to derive expert features and recover optimal policies from augmented delayed observations. Empirical evaluations in the MuJoCo environment under diverse delay settings validate the effectiveness of our method. Furthermore, we provide a theoretical analysis showing that recovering expert policies from augmented delayed observations outperforms using direct delayed observations.

cs.LG↗

arXiv2024

Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shaping

In this paper, we aim to tackle the limitation of the Adversarial Inverse Reinforcement Learning (AIRL) method in stochastic environments where theoretical results cannot hold and performance is degraded. To address this issue, we propose a novel method which infuses the dynamics information into the reward shaping with the theoretical guarantee for the induced optimal policy in the stochastic environments. Incorporating our novel model-enhanced rewards, we present a novel Model-Enhanced AIRL framework, which integrates transition model estimation directly into reward shaping. Furthermore, we provide a comprehensive theoretical analysis of the reward error bound and performance difference bound for our method. The experimental results in MuJoCo benchmarks show that our method can achieve superior performance in stochastic environments and competitive performance in deterministic environments, with significant improvement in sample efficiency, compared to existing baselines.

cs.LG↗

arXiv2020

Shear response of granular packings compressed above jamming onset

We investigate the mechanical response of jammed packings of repulsive, frictionless spherical particles undergoing isotropic compression. Prior simulations of the soft-particle model, where the repulsive interactions scale as a power-law in the interparticle overlap with exponent $\alpha$, have found that the ensemble-averaged shear modulus $\langle G \rangle$ increases with pressure $P$ as $\sim P^{(\alpha-3/2)/(\alpha-1)}$ at large pressures. However, a deep theoretical understanding of this scaling behavior is lacking. We show that the shear modulus of jammed packings of frictionless, spherical particles has two key contributions: 1) continuous variations as a function of pressure along geometrical families, for which the interparticle contact network does not change, and 2) discontinuous jumps during compression that arise from changes in the contact network. We show that the shear modulus of the first geometrical family for jammed packings can be collapsed onto a master curve: $G^{(1)}/G_0 = (P/P_0)^{(\alpha-2)/(\alpha-1)} - P/P_0$, where $P_0 \sim N^{-2(\alpha-1)}$ is a characteristic pressure that separates the two power-law scaling regions and $G_0 \sim N^{-2(\alpha-3/2)}$. Deviations from this form can occur when there is significant non-affine particle motion near changes in the contact network. We further show that $\langle G (P)\rangle$ is not simply a sum of two power-laws, but $\langle G \rangle \sim (P/P_c)^a$, where $a \approx (\alpha -2)/(\alpha-1)$ in the $P \rightarrow 0$ limit and $\langle G \rangle \sim (P/P_c)^b$, where $b \gtrsim (\alpha -3/2)/(\alpha-1)$ above a characteristic pressure $P_c$. In addition, the magnitudes of both contributions to $\langle G\rangle$ from geometrical families and changes in the contact network remain comparable in the large-system limit for $P >P_c$.

cond-mat.soft↗