Automated Driving with Evolution Capability: A Reinforcement Learning Method with Monotonic Performance Enhancement

Abstract

Reinforcement Learning (RL) offers a promising solution to enable evolutionary automated driving. However, the conventional RL method is always concerned with risk performance. The updated policy may not obtain a performance enhancement, even leading to performance deterioration. To address this challenge, this research proposes a High Confidence Policy Improvement Reinforcement Learning-based (HCPI-RL) planner. It is intended to achieve the monotonic evolution of automated driving. A novel RL policy update paradigm is designed to enable the newly learned policy performance consistently surpass that of previous policies, which is deemed as monotonic performance enhancement. Hence, the proposed HCPI-RL planner has the following features: i) Evolutionary automated driving with monotonic performance enhancement; ii) With the capability of handling scenarios with emergency; iii) With enhanced decision-making optimality. Results demonstrate that the proposed HCPI-RL planner enhances the policy return by 44.7% in emergent cut-in scenarios, 108.2% in emergent braking scenarios, and 64.4% in daily cruising scenarios, compared to the PPO planner. Adopting the proposed planner, automated driving efficiency is enhanced by 19.2% compared to the PPO planner, and by 30.7% compared to the rule-based planner.