arXiv subjects

Yeoneung Kim

Publications and source records attributed to Yeoneung Kim.

28 records · Page 2Linked to original sources

arXiv2024

Acceleration of Grokking in Learning Arithmetic Operations via Kolmogorov-Arnold Representation

We propose novel methodologies aimed at accelerating the grokking phenomenon, which refers to the rapid increment of test accuracy after a long period of overfitting as reported in~\cite{power2022grokking}. Focusing on the grokking phenomenon that arises in learning arithmetic binary operations via the transformer model, we begin with a discussion on data augmentation in the case of commutative binary operations. To further accelerate, we elucidate arithmetic operations through the lens of the Kolmogorov-Arnold (KA) representation theorem, revealing its correspondence to the transformer architecture: embedding, decoder block, and classifier. Observing the shared structure between KA representations associated with binary operations, we suggest various transfer learning mechanisms that expedite grokking. This interpretation is substantiated through a series of rigorous experiments. In addition, our approach is successful in learning two nonstandard arithmetic tasks: composition of operations and a system of equations. Furthermore, we reveal that the model is capable of learning arithmetic operations using a limited number of tokens under embedding transfer, which is supported by a set of experiments as well.

cs.LG↗

arXiv2024

On the stability of Lipschitz continuous control problems and its application to reinforcement learning

We address the crucial yet underexplored stability properties of the Hamilton--Jacobi--Bellman (HJB) equation in model-free reinforcement learning contexts, specifically for Lipschitz continuous optimal control problems. We bridge the gap between Lipschitz continuous optimal control problems and classical optimal control problems in the viscosity solutions framework, offering new insights into the stability of the value function of Lipschitz continuous optimal control problems. By introducing structural assumptions on the dynamics and reward functions, we further study the rate of convergence of value functions. Moreover, we introduce a generalized framework for Lipschitz continuous control problems that incorporates the original problem and leverage it to propose a new HJB-based reinforcement learning algorithm. The stability properties and performance of the proposed method are tested with well-known benchmark examples in comparison with existing approaches.

math.OC↗

arXiv2023

On a minimum eradication time for the SIR model with time-dependent coefficients

We study the minimum eradication time problem for controlled Susceptible-Infected-Recovered (SIR) epidemic models that incorporate vaccination control and time-varying infected and recovery rates. Unlike the SIR model with constant rates, the time-varying model is more delicate as the number of infectious individuals can oscillate, which causes ambiguity for the definition of the eradication time. We accordingly introduce two definitions that describe the minimum eradication time, and we prove that for a suitable choice of the threshold, the two definitions coincide. We also study the well-posedness of time-dependent Hamilton-Jacobi equation that the minimum eradication time satisfies in the viscosity sense and verify that the value function is locally semiconcave under certain conditions.

math.AP↗

arXiv2023

Improved Regret Analysis for Variance-Adaptive Linear Bandits and Horizon-Free Linear Mixture MDPs

In online learning problems, exploiting low variance plays an important role in obtaining tight performance guarantees yet is challenging because variances are often not known a priori. Recently, considerable progress has been made by Zhang et al. (2021) where they obtain a variance-adaptive regret bound for linear bandits without knowledge of the variances and a horizon-free regret bound for linear mixture Markov decision processes (MDPs). In this paper, we present novel analyses that improve their regret bounds significantly. For linear bandits, we achieve $\tilde O(\min\{d\sqrt{K}, d^{1.5}\sqrt{\sum_{k=1}^K σ_k^2}\} + d^2)$ where $d$ is the dimension of the features, $K$ is the time horizon, and $σ_k^2$ is the noise variance at time step $k$, and $\tilde O$ ignores polylogarithmic dependence, which is a factor of $d^3$ improvement. For linear mixture MDPs with the assumption of maximum cumulative reward in an episode being in $[0,1]$, we achieve a horizon-free regret bound of $\tilde O(d \sqrt{K} + d^2)$ where $d$ is the number of base models and $K$ is the number of episodes. This is a factor of $d^{3.5}$ improvement in the leading term and $d^7$ in the lower order term. Our analysis critically relies on a novel peeling-based regret analysis that leverages the elliptical potential `count' lemma.

stat.ML↗

arXiv2022

State-constraint static Hamilton-Jacobi equations in nested domains

We study state-constraint static Hamilton-Jacobi equations in a sequence of domains $\{Ω_k\}_{k \in \mathbb{N}}$ in $\mathbb{R}^n$ such that $Ω_k \subset Ω_{k+1}$ for all $k\in \mathbb{N}$. We obtain rates of convergence of $u_k$, the solution to the state-constraint problem in $Ω_k$, to $u$, the solution to the corresponding problem in $Ω= \bigcup_{k \in \mathbb{N}} Ω_k$. In many cases, the rates obtained are proven to be optimal. Various new examples and discussions are provided at the end of the paper.

math.AP↗

arXiv2022

Infusing model predictive control into meta-reinforcement learning for mobile robots in dynamic environments

The successful operation of mobile robots requires them to adapt rapidly to environmental changes. To develop an adaptive decision-making tool for mobile robots, we propose a novel algorithm that combines meta-reinforcement learning (meta-RL) with model predictive control (MPC). Our method employs an off-policy meta-RL algorithm as a baseline to train a policy using transition samples generated by MPC when the robot detects certain events that can be effectively handled by MPC, with its explicit use of robot dynamics. The key idea of our method is to switch between the meta-learned policy and the MPC controller in a randomized and event-triggered fashion to make up for suboptimal MPC actions caused by the limited prediction horizon. During meta-testing, the MPC module is deactivated to significantly reduce computation time in motion control. We further propose an online adaptation scheme that enables the robot to infer and adapt to a new task within a single trajectory. The performance of our method has been demonstrated through simulations using a nonlinear car-like vehicle model with (i) synthetic movements of obstacles, and (ii) real-world pedestrian motion data. The simulation results indicate that our method outperforms other algorithms in terms of learning efficiency and navigation quality.

cs.RO↗

arXiv2022

On representation formulas for optimal control: A Lagrangian perspective

In this paper, we study representation formulas for finite-horizon optimal control problems with or without state constraints, unifying two different viewpoints: the Lagrangian and dynamic programming (DP) frameworks. In a recent work [1], the generalized Lax formula is obtained via DP for optimal control problems with state constraints and nonlinear systems. We revisit the formula from the Lagrangian perspective to provide a unified framework for understanding and implementing the nontrivial representation of the value function. Our simple derivation makes direct use of the Lagrangian formula from the theory of Hamilton-Jacobi (HJ) equations. We also discuss a rigorous way to construct an optimal control using a $δ$-net, as well as a numerical scheme for controller synthesis via convex optimization.

math.OC↗

arXiv2021

Training Wasserstein GANs without gradient penalties

We propose a stable method to train Wasserstein generative adversarial networks. In order to enhance stability, we consider two objective functions using the $c$-transform based on Kantorovich duality which arises in the theory of optimal transport. We experimentally show that this algorithm can effectively enforce the Lipschitz constraint on the discriminator while other standard methods fail to do so. As a consequence, our method yields an accurate estimation for the optimal discriminator and also for the Wasserstein distance between the true distribution and the generated one. Our method requires no gradient penalties nor corresponding hyperparameter tuning and is computationally more efficient than other methods. At the same time, it yields competitive generators of synthetic images based on the MNIST, F-MNIST, and CIFAR-10 datasets.

cs.LG↗

arXiv2018

On the Uniqueness for One-Dimensional Constrained Hamilton-Jacobi Equations

The goal of this paper is to study uniqueness of a one-dimensional Hamilton-Jacobi equation \begin{equation*} \begin{cases} u_t=|u_x|^2+R(x,I(t)) &\text{in }\mathbb{R} \times (0,\infty), \max_{\mathbb{R}} u(\cdot,t)=0 &\text{on }[0,\infty), \end{cases} \end{equation*} with an initial condition $u_0(x,0)=u_0(x)$ on $\mathbb{R}$. A reaction term $R(x,I(t))$ is given while $I(t)$ is an unknown constraint (Lagrange multiplier) that forces maximum of $u$ to be always zero. In the paper, we prove uniqueness of a pair of unknowns (u,I) using dynamic programming principle in one dimensional space for some particular class of nonseparable reaction $R(x,I(t))$.

math.AP↗

arXiv2018

Well-posedness for constrained Hamilton-Jacobi equations

The goal of this paper is to study a Hamilton-Jacobi equation \begin{equation*} \begin{cases} u_t=H(Du)+R(x,I(t)) &\text{in }\mathbb{R}^n \times (0,\infty), \sup_{\mathbb{R}^n} u(\cdot,t)=0 &\text{on }[0,\infty), \end{cases} \end{equation*} with initial conditions $I(0)=0$, $u_0(x,0)=u_0(x)$ on $\mathbb{R}^n$. Here $(u,I)$ is a pair of unknowns and the Hamiltonian $H$ and the reaction $R$ are given. And $I(t)$ is an unknown constraint (Lagrange multiplier) that forces supremum of $u$ to be always zero. We construct a solution in the viscosity setting using a fixed point argument when the reaction term $R(x,I)$ is strictly decreasing in $I$. We also discuss both uniqueness and nonuniqueness. For uniqueness, a certain structural assumption on $R(x,I)$ is needed. We also provide an example with infinitely many solutions when the reaction term is not strictly decreasing in $I$.

math.AP↗