Counterexamples for BFGS-type methods under arbitrary strong Wolfe constants

Abstract

Whether the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method and its variants can fail to converge on smooth nonconvex functions under realistic line-search parameters has remained a central open problem in quasi-Newton theory since the landmark counterexample of Dai (2002), which was confined to small Armijo parameters ($c_1 \le 1/84 \approx 0.0119$ on Powell's geometry, or $c_1 \le 69/7480 \approx 0.0092$ on his six-point cycle) and an objective function unbounded below. In this paper, we resolve the long-standing open question, posed by Dai (2002) following a discussion with J. C. Gilbert, of whether such counterexamples exist in theory for every Armijo parameter $c_1 \in (0, 1)$. Specifically, for every prescribed pair of line-search parameters $0 < c_1 < c_2 < 1$, we construct an objective function $f \in C^\infty(\mathbb{R}^2)$, bounded below and with Lipschitz continuous gradient, on which every method in a broad conjugacy class $\mathcal{C}$ equipped with the first-local-minimizer line search generates an infinite sequence of iterates with $\|\nabla f(x_k)\| = 1$ for all $k \ge 0$. The class $\mathcal{C}$ encompasses the classical full-memory BFGS method, limited-memory BFGS (L-BFGS) with arbitrary memory $m \ge 1$, the Broyden positive family, and the Hestenes-Stiefel conjugate gradient method. The steps are the standard first local minimizers along the search rays and simultaneously satisfy the strong Wolfe, weak Wolfe, Armijo, and Goldstein conditions with constants $(c_1, c_2)$. The construction operates in the minimal possible dimension $n = 2$, exploiting a non-decaying conjugate descent orbit in the plane coupled with an explicit tubular interpolation whose two-bump axial curvature profile places the Armijo ratio anywhere in $(0, 1)$.