Optimization of fluid mixing by reinforcement learning using limit cycles of a dynamical system

Abstract

We propose a method to overcome the difficulties encountered when applying reinforcement learning to fluid mixing processes. The proposed method has two main features: (i) it does not require detailed measurements of the flow state, and (ii) by effectively exploiting a stable limit cycle of a two-dimensional dynamical system (the Li'enard system), it can stably perform optimization without imposing explicit constraints on the control parameters. As an illustrative example, we optimize a process in which a fluid contained in a cylindrical vessel is mixed by periodically rotating the vessel. The resulting optimal vessel motion is physically reasonable: it reverses its direction of rotation before a solid-body rotation state is established. Furthermore, even when the fluid viscosity increases with time during the mixing process, the method can continuously adapt the control parameters to the changing viscosity.