Jacob Azoulay | arXiv Science

arXiv2022

Lithium-Ion Battery Charging Schedule Optimization to Balance Battery Usage and Degradation

This work optimizes a lithium-ion battery charging schedule while considering a joint revenue and battery degradation model. The study extends the work of Meheswari et. al. to encourage battery usage/charging at optimal intervals depending on energy cost forecasts. This paper utilizes central difference Nesterov momentum gradient descent to come to optimal charging strategies and deal with the non-linearities of the battery degradation model. This optimization strategy is tested against constant, random varied price forecasts and a novel Gaussian process cost forecasting model. Contrary to many other papers regarding battery charging, formulating schedule optimization as a multivariate optimization problem provides meaningful insight to the inherent balance between these two competing objectives.

math.OC↗

arXiv2021

Teaching a Robot to Walk Using Reinforcement Learning

Classical control techniques such as PID and LQR have been used effectively in maintaining a system state, but these techniques become more difficult to implement when the model dynamics increase in complexity and sensitivity. For adaptive robotic locomotion tasks with several degrees of freedom, this task becomes infeasible with classical control techniques. Instead, reinforcement learning can train optimal walking policies with ease. We apply deep Q-learning and augmented random search (ARS) to teach a simulated two-dimensional bipedal robot how to walk using the OpenAI Gym BipedalWalker-v3 environment. Deep Q-learning did not yield a high reward policy, often prematurely converging to suboptimal local maxima likely due to the coarsely discretized action space. ARS, however, resulted in a better trained robot, and produced an optimal policy which officially "solves" the BipedalWalker-v3 problem. Various naive policies, including a random policy, a manually encoded inch forward policy, and a stay still policy, were used as benchmarks to evaluate the proficiency of the learning algorithm results.

cs.LG↗