arXiv Science⌕ Search

arXiv subjects

Andrea Del Prete

Publications and source records attributed to Andrea Del Prete.

At least 19 recordsLinked to original sources
arXiv2026

CAST: Alternating State-Value Targets and Expanded Policy Gradients for Model-Based Reinforcement Learning

Model-based reinforcement learning (MBRL) is a family of RL methods that learn a model of the environment and use it for action selection, making it well suited to robotics due to its sample efficiency. Combining learned models with online planning can further improve action selection, as the planner can exploit the model to find better actions than the learned policy alone. Recent methods combining learned policies with online planning typically learn the value of the policy rather than the stronger planner-guided behavior. We present CAST (Critic with Alternating State-value Target), which uses planner-guided behavior to improve value learning while regularizing the value estimate with the current policy. CAST replaces the action-value critic with a state-value critic, trained using a target that combines a real planner-guided transition and an imagined transition under the current policy. The resulting value function corresponds to an alternating process between planner-guided behavior and the current policy, allowing it to benefit from the stronger planner behavior while being regularised by the policy being learned. We evaluate CAST on the DeepMind Control and HumanoidBench Suites against several state-of-the-art methods, and demonstrate successful transfer to a physical Unitree Go2 quadruped performing a dynamic handstand.

cs.RO↗
arXiv2026

Towards safe and optimal flight: Viability Kernel MPC for Fully Actuated Multirotor

Industrial aerial robotics demands safety guarantees for navigation in unstructured environments while optimizing performance and computational efficiency. This paper presents a method for generating safe pose trajectories for fully actuated multirotors within a Model Predictive Control (MPC) framework, leveraging both viability theory and data-driven methods. Obstacle avoidance is enforced through dynamically computed axis-aligned bounding boxes, providing formal safety guarantees without exhaustive offline reachability analysis. Numerical simulations on a fully actuated tilted hexarotor validate the approach, demonstrating successful navigation in cluttered environments with real-time computational performance.

cs.RO↗
arXiv2026

PAC-DP: PAC-Bayesian Diffusion Policy Learning

Diffusion Policies (DPs) are able to perform complex manipulation tasks. However, DPs are typically trained by minimizing a denoising objective, which provides limited control over generalization in the finite-data regimes common in robotics. In this letter, we propose PAC-DP, an approach that increases the performance of DPs in robotic manipulation tasks. By modeling the DP as a Bayesian neural network, and defining a PAC-Bayes generalization bound, we derive a novel training objective that augments the standard denoising loss with a Kullback-Leibler divergence regularizer between the posterior and prior parameter distributions. From the theoretical perspective, our approach provides a principled approach to regularize the training of DPs without significantly increasing the training time. From the practical point of view, experimental results demonstrate improved denoising performance, lower variational negative log-likelihood, and higher success rates across multiple robotic manipulation benchmarks. Crucially, the largest improvements are observed in low-data training regimes and complex tasks, establishing PAC-DP as a theoretically grounded framework for robot policy learning.

cs.RO↗
arXiv2026

PPO-EAL: Exact Augmented Lagrangian Proximal Policy Optimization for Safe Robotic Control

Reinforcement learning (RL) has emerged as a promising solution to accomplish complex robotic control tasks; however, most of the current work ignores the safety requirements. Safe RL seeks to maximize task performance while satisfying explicit physical constraints, but current algorithms struggle to learn the policy efficiently with precise constraint satisfaction. This work proposes PPO-EAL, a novel first-order constrained policy optimization framework that integrates exact augmented Lagrangian optimization into proximal policy optimization for safe robotic control. By combining clipped policy updates with exact quadratic penalty terms, PPO-EAL achieves theoretically grounded constraint enforcement without requiring impractically large penalty factors. A momentum-regulated multiplier update further improves dual-variable stability, reducing constraint oscillation and unsafe behavior while preserving task performance. We provide exactness and convergence analysis under standard stochastic approximation assumptions. Extensive validation across diverse GPU-accelerated robotic benchmarks-including cart-pole balancing, cart-double-pendulum stabilization, 7-DoF Franka end-effector reaching, and quadrupedal locomotion-demonstrates superior safety precision and reward performance compared with state-of-the-art first-order safe RL baselines. Finally, we demonstrate zero-shot sim-to-real deployment in a contact-rich gear assembly task, where PPO-EAL substantially improves task success, reduces peak contact force, and enhances operational robustness. These results establish PPO-EAL as a general and practically deployable safe RL framework for diverse safety-critical robotic systems.

cs.RO↗
arXiv2026

CACTO-BIC: Scalable Actor-Critic Learning via Biased Sampling and GPU-Accelerated Trajectory Optimization

Trajectory Optimization (TO) and Reinforcement Learning (RL) offer complementary strengths for solving optimal control problems. TO efficiently computes locally optimal solutions but can struggle with non-convexity, while RL is more robust to non-convexity at the cost of significantly higher computational demands. CACTO (Continuous Actor-Critic with Trajectory Optimization) was introduced to combine these advantages by learning a warm-start policy that guides the TO solver towards low-cost trajectories. However, scalability remains a key limitation, as increasing system complexity significantly raises the computational cost of TO. This work introduces CACTO-BIC to address these challenges. CACTO-BIC improves data efficiency by biasing initial-state sampling leveraging a property of the value function associated with locally optimal policies; moreover, it reduces computation time by exploiting GPU acceleration. Empirical evaluations show improved sample efficiency and faster computation compared to CACTO. Comparisons with PPO demonstrate that our approach can achieve similar solutions in less time. Finally, experiments on the AlienGO quadruped robot demonstrate that CACTO-BIC can scale to high-dimensional systems and is suitable for real-time applications.

cs.RO↗
arXiv2025

Sample Efficient Certification of Discrete-Time Control Barrier Functions

Control Invariant (CI) sets are instrumental in certifying the safety of dynamical systems. Control Barrier Functions (CBFs) are effective tools to compute such sets, since the zero sublevel sets of CBFs are CI sets. However, computing CBFs generally involves addressing a complex robust optimization problem, which can be intractable. Scenario-based methods have been proposed to simplify this computation. Then, one needs to verify if the CBF actually satisfies the robust constraints. We present an approach to perform this verification that relies on Lipschitz arguments, and forms the basis of a certification algorithm designed for sample efficiency. Through a numerical example, we validated the efficiency of the proposed procedure.

eess.SY↗
arXiv2025

Parallel-Constraint Model Predictive Control: Exploiting Parallel Computation for Improving Safety

Ensuring constraint satisfaction is a key requirement for safety-critical systems, which include most robotic platforms. For example, constraints can be used for modeling joint position/velocity/torque limits and collision avoidance. Constrained systems are often controlled using Model Predictive Control, because of its ability to naturally handle constraints, relying on numerical optimization. However, ensuring constraint satisfaction is challenging for nonlinear systems/constraints. A well-known tool to make controllers safe is the so-called control-invariant set (a.k.a. safe set). In our previous work, we have shown that safety can be improved by letting the safe-set constraint recede along the MPC horizon. In this paper, we push that idea further by exploiting parallel computation to improve safety. We solve several MPC problems at the same time, where each problem instantiates the safe-set constraint at a different time step along the horizon. Finally, the controller can select the best solution according to some user-defined criteria. We validated this idea through extensive simulations with a 3-joint robotic arm, showing that significant improvements can be achieved in terms of safety and performance, even using as little as 4 computational cores.

cs.RO↗
arXiv2024

Hybrid Lyapunov-based feedback stabilization of bipedal locomotion based on reference spreading

We propose a hybrid formulation of the linear inverted pendulum model for bipedal locomotion, where the foot switches are triggered based on the center of mass position, removing the need for pre-defined footstep timings. Using a concept similar to reference spreading, we define nontrivial tracking error coordinates induced by our hybrid model. These coordinates enjoy desirable linear flow dynamics and rather elegant jump dynamics perturbed by a suitable extended class ${\mathcal K}_\infty$ function of the position error. We stabilize this hybrid error dynamics using a saturated feedback controller, selecting its gains by solving a convex optimization problem. We prove local asymptotic stability of the tracking error and provide a certified estimate of the basin of attraction, comparing it with a numerical estimate obtained from the integration of the closed-loop dynamics. Simulations on a full-body model of a real robot show the practical applicability of the proposed framework and its advantages with respect to a standard model predictive control formulation.

eess.SY↗
arXiv2024

ALPINE: a climbing robot for operations in mountain environments

Mountain slopes are perfect examples of harsh environments in which humans are required to perform difficult and dangerous operations such as removing unstable boulders, dangerous vegetation or deploying safety nets. A good replacement for human intervention can be offered by climbing robots. The different solutions existing in the literature are not up to the task for the difficulty of the requirements (navigation, heavy payloads, flexibility in the execution of the tasks). In this paper, we propose a robotic platform that can fill this gap. Our solution is based on a robot that hangs on ropes, and uses a retractable leg to jump away from the mountain walls. Our package of mechanical solutions, along with the algorithms developed for motion planning and control, delivers swift navigation on irregular and steep slopes, the possibility to overcome or travel around significant natural barriers, and the ability to carry heavy payloads and execute complex tasks. In the paper, we give a full account of our main design and algorithmic choices and show the feasibility of the solution through a large number of physically simulated scenarios.

cs.RO↗
arXiv2024

Constant mean curvature graphs with prescribed asymptotic values in $\mathbb{E}(-1,\tau)$

In the homogeneous manifold $\mathbb{E}(-1,\tau),$ for $-\tfrac{1}{2}<H<\tfrac{1}{2},$ {we define a new product compactification in which the slices $\left\{t=c\right\}_{c\in\R}$ are rotational $H$-surfaces. This product compatification is the natural setting where it makes sense to study the asymptotic Dirichlet Problem for the constant mean curvature equation. Indeed, for every rectifiable curve $\Gamma$ projecting bijectively onto $\partial\H^2$ we prove the existence of a unique entire $H$-graph that is asymptotic to $\Gamma$.} We also find necessary and sufficient conditions for the existence of $H$-graphs over unbounded domains having prescribed, possibly infinite boundary data.

math.DG↗
arXiv2023

CACTO-SL: Using Sobolev Learning to improve Continuous Actor-Critic with Trajectory Optimization

Trajectory Optimization (TO) and Reinforcement Learning (RL) are powerful and complementary tools to solve optimal control problems. On the one hand, TO can efficiently compute locally-optimal solutions, but it tends to get stuck in local minima if the problem is not convex. On the other hand, RL is typically less sensitive to non-convexity, but it requires a much higher computational effort. Recently, we have proposed CACTO (Continuous Actor-Critic with Trajectory Optimization), an algorithm that uses TO to guide the exploration of an actor-critic RL algorithm. In turns, the policy encoded by the actor is used to warm-start TO, closing the loop between TO and RL. In this work, we present an extension of CACTO exploiting the idea of Sobolev learning. To make the training of the critic network faster and more data efficient, we enrich it with the gradient of the Value function, computed via a backward pass of the differential dynamic programming algorithm. Our results show that the new algorithm is more efficient than the original CACTO, reducing the number of TO episodes by a factor ranging from 3 to 10, and consequently the computation time. Moreover, we show that CACTO-SL helps TO to find better minima and to produce more consistent results.

cs.RO↗
arXiv2023

Parabolicity of invariant Surfaces

We present a clear and practical way to characterize the parabolicity of a complete immersed surface that is invariant with respect to a Killing vector field of the ambient space.

math.DG↗
arXiv2023

Receding-Constraint Model Predictive Control using a Learned Approximate Control-Invariant Set

In recent years, advanced model-based and data-driven control methods are unlocking the potential of complex robotics systems, and we can expect this trend to continue at an exponential rate in the near future. However, ensuring safety with these advanced control methods remains a challenge. A well-known tool to make controllers (either Model Predictive Controllers or Reinforcement Learning policies) safe, is the so-called control-invariant set (a.k.a. safe set). Unfortunately, for nonlinear systems, such a set cannot be exactly computed in general. Numerical algorithms exist for computing approximate control-invariant sets, but classic theoretic control methods break down if the set is not exact. This paper presents our recent efforts to address this issue. We present a novel Model Predictive Control scheme that can guarantee recursive feasibility and/or safety under weaker assumptions than classic methods. In particular, recursive feasibility is guaranteed by making the safe-set constraint move backward over the horizon, and assuming that such set satisfies a condition that is weaker than control invariance. Safety is instead guaranteed under an even weaker assumption on the safe set, triggering a safe task-abortion strategy whenever a risk of constraint violation is detected. We evaluated our approach on a simulated robot manipulator, empirically demonstrating that it leads to less constraint violations than state-of-the-art approaches, while retaining reasonable performance in terms of tracking cost, number of completed tasks, and computation time.

cs.RO↗
arXiv2023

Efficient Reinforcement Learning for Jumping Monopods

In this work, we consider the complex control problem of making a monopod reach a target with a jump. The monopod can jump in any direction and the terrain underneath its foot can be uneven. This is a template of a much larger class of problems, which are extremely challenging and computationally expensive to solve using standard optimisation-based techniques. Reinforcement Learning (RL) could be an interesting alternative, but the application of an end-to-end approach in which the controller must learn everything from scratch, is impractical. The solution advocated in this paper is to guide the learning process within an RL framework by injecting physical knowledge. This expedient brings to widespread benefits, such as a drastic reduction of the learning time, and the ability to learn and compensate for possible errors in the low-level controller executing the motion. We demonstrate the advantage of our approach with respect to both optimization-based and end-to-end RL approaches.

cs.RO↗
arXiv2023

Multi-contact Stochastic Predictive Control for Legged Robots with Contact Locations Uncertainty

Trajectory optimization under uncertainties is a challenging problem for robots in contact with the environment. Such uncertainties are inevitable due to estimation errors, control imperfections, and model mismatches between planning models used for control and the real robot dynamics. This induces control policies that could violate the contact location constraints by making contact at unintended locations, and as a consequence leading to unsafe motion plans. This work addresses the problem of robust kino-dynamic whole-body trajectory optimization using stochastic nonlinear model predictive control (SNMPC) by considering additive uncertainties on the model dynamics subject to contact location chance-constraints as a function of robot's full kinematics. We demonstrate the benefit of using SNMPC over classic nonlinear MPC (NMPC) for whole-body trajectory optimization in terms of contact location constraint satisfaction (safety). We run extensive Monte-Carlo simulations for a quadruped robot performing agile trotting and bounding motions over small stepping stones, where contact location satisfaction becomes critical. Our results show that SNMPC is able to perform all motions safely with 100% success rate, while NMPC failed 48.3% of all motions.

cs.RO↗
arXiv2023

Minimal graphs over non-compact domains in 3-manifolds fibered by a Killing vector field

Let $\mathbb{E}$ be a connected and orientable Riemannian 3-manifold with a non-singular Killing vector field whose associated one-parameter group of the isometries of $\mathbb{E}$ acts freely and properly on $\E$. Then, there exists a Killing Submersion from $\E$ onto a connected and orientable surface $M$ whose fibers are the integral curves of the Killing vector field. In this setting, assuming that $M$ is non-compact and the fibers have infinite length, we solve the Dirichlet problem for minimal Killing graphs over certain unbounded domains of $M$, prescribing piecewise continuous boundary values. We obtain general Collin-Krust type estimates. In the particular case of the Heisenberg group, we prove a uniqueness result for minimal Killing graphs with bounded boundary values over a strip. We also prove that isolated singularities of Killing graphs with prescribed mean curvature are removable.

math.DG↗
arXiv2023

The Jenkins-Serrin problem in 3-manifolds with a Killing vector field

We consider a Riemannian submersion from a 3-manifold $\mathbb{E}$ to a surface $M$, both connected and orientable, whose fibers are the integral curves of a Killing vector field without zeros, not necessarily unitary. We solve the Jenkins-Serrin problem for the minimal surface equation in $\mathbb{E}$ over a relatively compact open domain $\Omega\subset M$ with prescribed finite or infinite values on some arcs of the boundary under the only assumption that the same value $+\infty$ or $-\infty$ cannot be prescribed on two adjacent components of $\partial\Omega$ forming a convex angle. The domain $\Omega$ can have reentrant corners as well as closed curves in its boundary. We show that the solution exists if and only if some generalized Jenkins-Serrin conditions (in terms of a conformal metric in $M$) are fulfilled. We develop further the theory of divergence lines to study the convergence of a sequence of minimal graphs. We also provide maximum principles that guarantee the uniqueness of the solution. Finally, we obtain new examples of minimal surfaces in $\mathbb{R}^3$ and in other homogeneous $3$-manifolds.

math.DG↗
arXiv2023

A duality for prescribed mean curvature graphs in Riemannian and Lorentzian Killing submersions

We develop a conformal duality for spacelike graphs in Riemannian and Lorentzian three-manifolds that admit a Riemannian submersion over a Riemannian surface whose fibers are the integral curves of a Killing vector field, which is timelike in the Lorentzian case. The duality swaps mean curvature and bundle curvature and sends the length of the Killing vector field to its reciprocal while keeping invariant the base surface. We obtain two consequences of this result. On the one hand, we find entire graphs in Lorentz-Minkowski space $\mathbb{L}^3$ with prescribed mean curvature a bounded function $H\in C^\infty(\mathbb{R}^2)$ with bounded gradient. On the other hand, we obtain conditions for existence and non existence of entire graphs which are related to a notion of critical mean curvature.

math.DG↗