arXiv subjects

Guillaume Crevecoeur

Publications and source records attributed to Guillaume Crevecoeur.

27 records · Page 2Linked to original sources

arXiv2021

Entropy Regularised Deterministic Optimal Control: From Path Integral Solution to Sample-Based Trajectory Optimisation

Sample-based trajectory optimisers are a promising tool for the control of robotics with non-differentiable dynamics and cost functions. Contemporary approaches derive from a restricted subclass of stochastic optimal control where the optimal policy can be expressed in terms of an expectation over stochastic paths. By estimating the expectation with Monte Carlo sampling and reinterpreting the process as exploration noise, a stochastic search algorithm is obtained tailored to (deterministic) trajectory optimisation. For the purpose of future algorithmic development, it is essential to properly understand the underlying theoretical foundations that allow for a principled derivation of such methods. In this paper we make a connection between entropy regularisation in optimisation and deterministic optimal control. We then show that the optimal policy is given by a belief function rather than a deterministic function. The policy belief is governed by a Bayesian-type update where the likelihood can be expressed in terms of a conditional expectation over paths induced by a prior policy. Our theoretical investigation firmly roots sample based trajectory optimisation in the larger family of control as inference. It allows us to justify a number of heuristics that are common in the literature and motivate a number of new improvements that benefit convergence.

cs.RO↗

arXiv2021

Risk Sensitive Path Integral Control for Infinite Horizon Problem Formulations

Path Integral Control methods were developed for stochastic optimal control covering a wide class of finite horizon formulations with control affine nonlinear dynamics. Characteristic for this class is that the HJB equation is linear and consequently the value function can be expressed as a conditional expectation of the exponentially weighted cost-to-go evaluated over trajectories with uncontrolled system dynamics, hence the name. Subsequently it was shown that under the same assumptions Path Integral Control generalises to finite horizon risk sensitive stochastic optimal control problems. Here we study whether the HJB of infinite horizon formulations can be made linear as well. Our interest in infinite horizon formulations is motivated by the stationarity of the associated value function and their inherent dynamic stability seeking nature. Technically a stationary value function may ease the solution of the associated linear HJB. Second we argue this may offer an interesting starting point for off-linear Reinforcement Learning applications. We show formally that the discounted and average cost formulations are respectively intractable and tractable.

math.OC↗

arXiv2021

Neural Network Augmented Physics Models for Systems with Partially Unknown Dynamics: Application to Slider-Crank Mechanism

Dynamic models of mechatronic systems are abundantly used in the context of motion control and design of complex servo applications. In practice, these systems are often plagued by unknown interactions, which make the physics-based relations of the system dynamics only partially known. This paper presents a neural network augmented physics (NNAP) model as a combination of physics-inspired and neural layers. The neural layers are inserted in the model to compensate for the unmodeled interactions, without requiring direct measurements of these unknown phenomena. In contrast to traditional approaches, both the neural network and physical parameters are simultaneously optimized, solely by using state and control input measurements. The methodology is applied on experimental data of a slider-crank setup for which the state dependent load interactions are unknown. The NNAP model proves to be a stable and accurate modeling formalism for dynamic systems that ab initio can only be partially described by physical laws. Moreover, the results show that a recurrent implementation of the NNAP model enables improved robustness and accuracy of the system state predictions, compared to its feedforward counterpart. Besides capturing the system dynamics, the NNAP model provides a means to gain new insights by extracting the neural network from the converged NNAP model. In this way, we discovered accurate representations of the unknown spring force interaction and friction phenomena acting on the slider mechanism.

eess.SY↗

arXiv2020

On Entropy Regularized Path Integral Control for Trajectory Optimization

In this article we present a generalised view on Path Integral Control (PIC) methods. PIC refers to a particular class of policy search methods that are closely tied to the setting of Linearly Solvable Optimal Control (LSOC), a restricted subclass of nonlinear Stochastic Optimal Control (SOC) problems. This class is unique in the sense that it can be solved explicitly to yield a formal optimal state trajectory distribution. In this contribution we first review the PIC theory and discuss related algorithms tailored to policy search in general. We are able to identify a generic design strategy that relies on the existence of an optimal state trajectory distribution and finds a parametric policy by minimizing the cross entropy between the optimal and a state trajectory distribution parametrized through its policy. Inspired by this observation we then aim to formulate a SOC problem that shares traits with the LSOC setting yet that covers a less restrictive class of problem formulations. We refer to this SOC problem as Entropy Regularized Trajectory Optimization. The problem is closely related to the Entropy Regularized Stochastic Optimal Control setting which is lately often addressed by the Reinforcement Learning (RL) community. We analyse the theoretical convergence behaviour of the theoretical state trajectory distribution sequence and draw connections with stochastic search methods tailored to classic optimization problems. Finally we derive explicit updates and compare the implied Entropy Regularized PIC with earlier work in the context of both PIC and RL for derivative-free trajectory optimization.

math.OC↗

arXiv2020

Evolutionary-Based Sparse Regression for the Experimental Identification of Duffing Oscillator

In this paper, an evolutionary-based sparse regression algorithm is proposed and applied onto experimental data collected from a Duffing oscillator setup and numerical simulation data. Our purpose is to identify the Coulomb friction terms as part of the ordinary differential equation of the system. Correct identification of this nonlinear system using sparse identification is hugely dependent on selecting the correct form of nonlinearity included in the function library. Consequently, in this work, the evolutionary-based sparse identification is replacing the need for user knowledge when constructing the library in sparse identification. Constructing the library based on the data-driven evolutionary approach is an effective way to extend the space of nonlinear functions, allowing for the sparse regression to be applied on an extensive space of functions. ,e results show that the method provides an effective algorithm for the purpose of unveiling the physical nature of the Duffing oscillator. In addition, the robustness of the identification algorithm is investigated for various levels of noise in simulation. ,e proposed method has possible applications to other nonlinear dynamic systems in mechatronics, robotics, and electronics.

cs.CE↗

arXiv2019

Inverse Parametric Uncertain Identification using Polynomial Chaos and high-order Moment Matching benchmarked on a Wet Friction Clutch

A numerically efficient inverse method for parametric model uncertainty identification using maximum likelihood estimation is presented. The goal is to identify a probability model for a fixed number of model parameters based on a set of experiments. To perform maximum likelihood estimation, the output probability density function is required. Forward propagation of input uncertainty is established combining Polynomial Chaos and moment matching. High-order moments of the output distribution are estimated using the generalized Polynomial Chaos framework. Next, a maximum entropy parametric distribution is matched with the estimated moments. This method is numerically very attractive due to reduced forward sampling and deterministic nature of the propagation strategy. The methodology is applied on a wet clutch system for which certain model variables are considered as stochastic. The number of required model simulations to achieve the same accuracy as the brute force methodologies is decreased by one order of magnitude. The probability model identified with the high order estimates resulted into a true log-likelihood increase of about 4% since the accuracy of the estimated output probability density function could be improved up to 47%.

stat.AP↗

arXiv2018

Polynomial Chaos reformulation in Nonlinear Stochastic Optimal Control with application on a drivetrain subject to bifurcation phenomena

This paper discusses a method enabling optimal control of nonlinear systems that are subject to parametric uncertainty. A stochastic optimal tracking problem is formulated that can be expressed in function of the first two stochastic moments of the state. The proposed formulation allows to penalize system performance and system robustness independently. The use of polynomial chaos expansions is investigated to arrive at a computationally tractable formulation expressing the stochastic moments in function of the polynomial expansion coefficients rigorously. It is then demonstrated how the stochastic optimal control problem can be reformulated as a deterministic optimal control problem in function of these coefficients. The proposed method is applied to find a robust control input for the start-up of an eccentrically loaded drive train that is inherently prone to bifurcation behaviour. A reference trajectory is chosen to deliberately provoke a bifurcation. The proposed framework is able to avoid the bifurcation behaviour regardlessly.

math.OC↗

arXiv2018

A Radial Basis Function based Optimization Algorithm with Regular Simplex set geometry in Ellipsoidal Trust-Regions

We present a novel derivative-free interpolation based optimization algorithm. A trust-region method is used where a surrogate model is realized via an interpolation framework. The framework for interpolation is provided by Universal Kriging. A first contribution focuses on the development of an original sampling strategy. A valid model is guaranteed by maintaining a well-poised subset that exhibits the regular simplex geometry approximately. It follows that this strategy improves the scattering of points with respect to the state-of-the-art and, even importantly, assures that the surrogate model exhibits curvature. A second contribution focuses on the generalization of the spherical trust-region geometry to an ellipsoidal geometry, that to account for local anisotropy of the objective function and to improve the interpolation conditions as seen from the output space. The ensemble method is validated against its direct competitors on a set of multidimensional problems.

math.OC↗

arXiv2018

Sparse Identification of Nonlinear Duffing Oscillator From Measurement Data

In this paper we aim to apply an adaptation of the recently developed technique of sparse identification of nonlinear dynamical systems on a Duffing experimental setup with cubic feedback of the output. The Duffing oscillator described by nonlinear differential equation which demonstrates chaotic behavior and bifurcations, has received considerable attention in recent years as it arises in many real-world engineering applications. Therefore its identification is of interest for numerous practical problems. To adopt the existing identification method to this application, the optimization process which identifies the most important terms of the model has been modified. In addition, the impact of changing the amount of regularization parameter on the mean square error of the fit has been studied. Selection of the true model is done via balancing complexity and accuracy using Pareto front analysis. This study provides considerable insight into the employment of sparse identification method on the real-world setups and the results show that the developed algorithm is capable of finding the true nonlinear model of the considered application including a nonlinear friction term.

physics.class-ph↗