arXiv subjects

Aayushman Sharma

Publications and source records attributed to Aayushman Sharma.

11 recordsLinked to original sources

arXiv2026

On Data-Driven Model Identification for Nonlinear Optimal Control

In this paper, we study the use of nonlinear model identification techniques for the optimal control of nonlinear systems, also known as model-based Reinforcement Learning. We show that the nonlinear model identification problem is equivalent to estimating the generalized moments of an underlying sampling distribution and is bound to suffer from ill-conditioning and variance when approximating a system to high order and over a large domain, requiring samples combinatorial-exponential in the order of the approximation and domain size: a ``Curse of Variance and Ill-Conditioning (COVIC)" that shows up even in very low dimensional problems, quite apart from the usual ``Curse of Dimensionality". We show that the iterative identification of ``local" linear time varying (LTV) models around the current estimate of the optimal trajectory, coupled with a suitable optimal control algorithm such as iterative LQR (ILQR), alleviates these issues and is sufficient to locally accurately solve the underlying optimal control problem.

math.OC↗

arXiv2026

A Two-Stage, Model-Based Reinforcement Learning Approach for Active Flow Control of Bluff Body Wakes

This paper develops a data-driven, output-feedback approach to the infinite-horizon optimal control of high-dimensional nonlinear systems with unknown and unstable equilibria, using sparse partial observations. The approach builds on the transfer-plus-regulation decomposition of the infinite-horizon problem: a finite-horizon nonlinear transfer drives the system into a region where the dynamics are well-approximated by a linear model about the unknown operating point, and an infinite-horizon linear regulator identified within that region completes stabilization. We extend this framework to the partially observed setting by combining an ARMA-based information-state construction with a two-stage control architecture: an iterative linear quadratic regulator (iLQR) approach on the information state drives the system to the equilibrium neighborhood, discovered implicitly without prior knowledge of the target, and a locally identified time-invariant ARMA model provides the infinite-horizon regulator for asymptotic stabilization. The method requires no adjoint solver, reduced-order model, or full-state access. We validate the approach on high-fidelity Navier-Stokes simulations of the cylinder wake at $\mathrm{Re}=100$ using only eight surface pressure sensors, an order of magnitude fewer than recent model-based RL methods. The controller achieves complete suppression of vortex-shedding-induced lift oscillations and a $44\%$ reduction in total drag relative to the uncontrolled baseline.

eess.SY↗

arXiv2026

An Optimal Solution to Infinite Horizon Nonholonomic and Discounted Nonlinear Control Problems

This paper considers the infinite horizon optimal control problem for nonlinear systems. Under the condition of nonlinear controllability of the system to any terminal set containing the origin and forward invariance of the terminal set, we establish a regularized solution approach consisting of a ``finite free final time" optimal transfer problem to the terminal set, which renders the set globally asymptotically stable. Further, we show that the approximations converge to the optimal infinite horizon cost as the size of the terminal set decreases to zero. We also perform the analysis for the discounted problem and show that the terminal set is asymptotically stable only for a subset of the state space and not globally. The theory is empirically evaluated on various nonholonomic robotic systems to show that the cost of our approximate problem converges and the transfer time into the terminal set is dependent on the initial state of the system, necessitating the free final time formulation. We also do comparisons of our free-final time approach with nonlinear MPC.

math.OC↗

arXiv2025

Optimal Control with Lyapunov Stability Guarantees for Space Applications

This paper investigates the infinite horizon optimal control problem (OCP) for space applications characterized by nonlinear dynamics. The proposed approach divides the problem into a finite horizon OCP with a regularized terminal cost, guiding the system towards a terminal set, and an infinite horizon linear regulation phase within this set. This strategy guarantees global asymptotic stability under specific assumptions. Our method maintains the system's fully nonlinear dynamics until it reaches the terminal set, where the system dynamics is linearized. As the terminal set converges to the origin, the difference in optimal cost incurred reduces to zero, guaranteeing an efficient and stable solution. The approach is tested through simulations on three problems: spacecraft attitude control, rendezvous maneuver, and soft landing. In spacecraft attitude control, we focus on achieving precise orientation and stabilization. For rendezvous maneuvers, we address the navigation of a chaser to meet a target spacecraft. For the soft landing problem, we ensure a controlled descent and touchdown on a planetary surface. We provide numerical results confirming the effectiveness of the proposed method in managing these nonlinear dynamics problems, offering robust solutions essential for successful space missions.

math.OC↗

arXiv2025

A Reduced Order Iterative Linear Quadratic Regulator (ILQR) Technique for the Optimal Control of Nonlinear Partial Differential Equations

In this paper, we introduce a reduced order model-based reinforcement learning (MBRL) approach, utilizing the Iterative Linear Quadratic Regulator (ILQR) algorithm for the optimal control of nonlinear partial differential equations (PDEs). The approach proposes a novel modification of the ILQR technique: it uses the Method of Snapshots to identify a reduced order Linear Time Varying (LTV) approximation of the nonlinear PDE dynamics around a current estimate of the optimal trajectory, utilizes the identified LTV model to solve a time-varying reduced order LQR problem to obtain an improved estimate of the optimal trajectory along with a new reduced basis, and iterates till convergence. The convergence behavior of the reduced order approach is analyzed and the algorithm is shown to converge to a limit set that is dependent on the truncation error in the reduction. The proposed approach is tested on the viscous Burger's equation and two phase-field models for microstructure evolution in materials, and the results show that there is a significant reduction in the computational burden over the standard ILQR approach, without significantly sacrificing performance.

eess.SY↗

arXiv2024

Convexity in Optimal Control Problems

This paper investigates the central role played by the Hamiltonian in continuous-time nonlinear optimal control problems. We show that the strict convexity of the Hamiltonian in the control variable is a sufficient condition for the existence of a unique optimal trajectory, and the nonlinearity/non-convexity of the dynamics and the cost are immaterial. The analysis is extended to discrete-time problems, revealing that discretization destroys the convex Hamiltonian structure, leading to multiple spurious optima, unless the time discretization is sufficiently small. We present simulated results comparing the "indirect" Iterative Linear Quadratic Regulator (iLQR) and the "direct" Sequential Quadratic Programming (SQP) approach for solving the optimal control problem for the cartpole and pendulum models to validate the theoretical analysis. Results show that the ILQR always converges to the "globally" optimum solution while the SQP approach gets stuck in spurious minima given multiple random initial guesses for a time discretization that is insufficiently small, while both converge to the same unique solution if the discretization is sufficiently small.

math.OC↗

arXiv2024

Benchmarking Model Predictive Control Algorithms in Building Optimization Testing Framework (BOPTEST)

We present a data-driven modeling and control framework for physics-based building emulators. Our approach consists of: (a) Offline training of differentiable surrogate models that accelerate model evaluations, provide cost-effective gradients, and maintain good predictive accuracy for the receding horizon in Model Predictive Control (MPC), and (b) Formulating and solving nonlinear building HVAC MPC problems. We extensively evaluate the modeling and control performance using multiple surrogate models and optimization frameworks across various test cases available in the Building Optimization Testing Framework (BOPTEST). Our framework is compatible with other modeling techniques and can be customized with different control formulations, making it adaptable and future-proof for test cases currently under development for BOPTEST. This modularity provides a path towards prototyping predictive controllers in large buildings, ensuring scalability and robustness in real-world applications.

eess.SY↗

arXiv2023

An Information-state based Approach to the Optimal Output Feedback Control of Nonlinear Systems

This paper develops a data-based approach to the closed-loop output feedback control of nonlinear dynamical systems with a partial nonlinear observation model. We propose an information state based approach to rigorously transform the partially observed problem into a fully observed problem where the information state consists of the past several observations and control inputs. We further show the equivalence of the transformed and the initial partially observed optimal control problems and provide the conditions to solve for the deterministic optimal solution. We develop a data based generalization of the iterative Linear Quadratic Regulator (iLQR) to partially observed systems using a local linear time varying model of the information state dynamics approximated by an Autoregressive moving average (ARMA) model, that is generated using only the input-output data. This open-loop trajectory optimization solution is then used to design a local feedback control law, and the composite law then provides an optimum solution to the partially observed feedback design problem. The efficacy of the developed method is shown by controlling complex high dimensional nonlinear dynamical systems in the presence of model and sensing uncertainty.

cs.RO↗

arXiv2022

Optimal Control of Material Micro-Structures

In this paper, we consider the optimal control of material micro-structures. Such material micro-structures are modeled by the so-called phase field model. We study the underlying physical structure of the model and propose a data based approach for its optimal control, along with a comparison to the control using a state of the art Reinforcement Learning (RL) algorithm. Simulation results show the feasibility of optimally controlling such micro-structures to attain desired material properties and complex target micro-structures.

eess.SY↗

arXiv2022

On the Search for Feedback in Reinforcement Learning

The problem of Reinforcement Learning (RL) in an unknown nonlinear dynamical system is equivalent to the search for an optimal feedback law utilizing the simulations/ rollouts of the dynamical system. Most RL techniques search over a complex global nonlinear feedback parametrization making them suffer from high training times as well as variance. Instead, we advocate searching over a local feedback representation consisting of an open-loop sequence, and an associated optimal linear feedback law completely determined by the open-loop. We show that this alternate approach results in highly efficient training, the answers obtained are repeatable and hence reliable, and the resulting closed performance is superior to global state-of-the-art RL techniques. Finally, if we replan, whenever required, which is feasible due to the fast and reliable local solution, it allows us to recover global optimality of the resulting feedback law.

cs.LG↗

arXiv2020

D2C 2.0: Decoupled Data-Based Approach for Learning to Control Stochastic Nonlinear Systems via Model-Free ILQR

In this paper, we propose a structured linear parameterization of a feedback policy to solve the model-free stochastic optimal control problem. This parametrization is corroborated by a decoupling principle that is shown to be near-optimal under a small noise assumption, both in theory and by empirical analyses. Further, we incorporate a model-free version of the Iterative Linear Quadratic Regulator (ILQR) in a sample-efficient manner into our framework. Simulations on systems over a range of complexities reveal that the resulting algorithm is able to harness the superior second-order convergence properties of ILQR. As a result, it is fast and is scalable to a wide variety of higher dimensional systems. Comparisons are made with a state-of-the-art reinforcement learning algorithm, the Deep Deterministic Policy Gradient (DDPG) technique, in order to demonstrate the significant merits of our approach in terms of training-efficiency.

math.OC↗