arXiv subjects

Haihao Lu

Publications and source records attributed to Haihao Lu.

At least 19 recordsLinked to original sources

arXiv2026

Reformulation Techniques for PDLP with Applications to QUBO and Joint Replenishment

Practitioners of linear programming widely believe that reformulating a linear program (LP) offers little benefit to interior-point methods: their iteration counts are famously insensitive to the formulation, so the payoff of reformulation beyond automatic presolve and customized linear solve is modest. This paper presents evidence that the same is not true of first-order solvers built on the primal-dual hybrid gradient method (PDHG), such as PDLP. Unlike interior-point methods, PDHG-type methods have per-iteration costs governed by the matrix representation of the formulation and iteration counts governed by its geometry, and both change under equivalent reformulations. We describe a general reformulation recipe: constraints drawn from a catalogue of projection-friendly polyhedra can be absorbed into the projection step of PDHG. We analyze what absorption does to the four constants that govern restarted PDHG: the coupling norm, the termination certificate, the distance scale, and the sharpness modulus. We perform the reformulation on two case studies. On the LP relaxation of quadratic unconstrained binary optimization (QUBO), absorption makes any step size below $1$ admissible on every instance, and yields a fully explicit linear rate. On the LP relaxation of the joint replenishment problem (JRP), whose optimum certifies a $2$-approximation for an NP-hard inventory problem, absorption improves the coupling norm and eliminates every matrix-vector product. Both reformulations run on GPUs with custom kernels in place of sparse matrix-vector products. We benchmark both against the GPU LP solver cuPDLPx applied to the original formulations. By solve time, the reformulated solvers are $6.4$-$9.6\times$ faster on QUBO and $6.1$-$28.9\times$ faster on the JRP. They take $1.7$-$4.3\times$ and $13.6$-$52.3\times$ fewer iterations respectively.

math.OC↗

arXiv2026

PDFP: GPU-Accelerated Feasibility Pump via PDHG

The feasibility pump (FP) is among the most widely used primal heuristics for mixed-integer linear programming. It alternates between rounding an LP-feasible point and projecting the rounded point back onto the feasible region of the LP relaxation. Recent pump-based heuristics compute the projection with the primal--dual hybrid gradient method (PDHG), which is scalable on modern GPUs. They pose it as a linear program, which needs auxiliary variables and constraints for general integers. Once the projection runs on the GPU, the constraint-propagation rounding (CPR) of FP~2.0 takes a large share of each iteration. For the projection, we propose PDFP, a custom first-order method that solves the pump's projection problem directly, without auxiliary primal variables or constraints. We prove that it keeps the accelerated linear rate of restarted Halpern PDHG. For the rounding, we propose a new procedure that is more efficient than CPR and, under certain assumptions, still returns a CPR rounding. It also chooses rounded values by constraint violation, which can produce feasible roundings where nearest rounding cannot. Finally, our pump runs two independent trajectories concurrently on one GPU through CPU--GPU synchronization. This exploits the variance in trajectory length and lets the pump terminate sooner on average. Computational experiments on MIPLIB~2017 instances show early, promising results: these contributions speed up the discovery of the first incumbent solution. The code is publicly available at https://github.com/MIT-Lu-Lab/PDFP.

math.OC↗

arXiv2026

On the Local Linear Convergence of Operator Splitting Methods for Conic Programming

Operator-splitting methods such as the primal-dual hybrid gradient method (PDHG) and the alternating direction method of multipliers (ADMM) often exhibit linear convergence on conic programs, although general theory guarantees only sublinear rates. We identify two geometric conditions -- strict complementarity and quadratic facial violation -- that explain this local behavior: under these conditions, PDHG and ADMM converge linearly to an optimal solution when initialized sufficiently close to the converging strictly complementary solution. We establish this result through a unified and verifiable primal-dual error-bound framework. First, we show that strict complementarity, together with a quadratic facial-violation property of the associated complementary faces, implies uniform quadratic growth of both the primal and dual augmented Lagrangians near a strictly complementary solution. Second, we prove the local equivalence of three regularity conditions: uniform quadratic growth of the augmented Lagrangians, quadratic growth of a localized smoothed primal-dual gap, and metric subregularity of the saddle-point mapping. This equivalence clarifies the relationship among previously proposed conditions for local linear convergence. Third, using a unified formulation, we give a concise analysis showing that these equivalent conditions yield local linear convergence of PDHG and ADMM. We verify the quadratic facial-violation property for standard polyhedral and symmetric cones, as well as relevant faces of exponential and power cones, and show that it is preserved under Cartesian products. We also obtain an improved local rate using a restarted Halpern scheme. Finally, we extend the framework to convex composite optimization through a quadratic subdifferential-violation condition, which generalizes the quadratic facial-violation.

math.OC↗

arXiv2026

cuPDQPx: A GPU-Accelerated Restarted Halpern PDHG for Large-Scale Convex Quadratic Programming

We present cuPDQPx, a GPU-accelerated, factorization-free first-order solver for large-scale convex quadratic programming (QP). Building on the restarted reflected Halpern primal-dual framework of cuPDLPx, we introduce an auxiliary-variable splitting that yields explicit updates without linearizing the quadratic objective or solving inner quadratic subproblems. A preconditioned proximal point interpretation yields both sublinear and linear fixed-point residual convergence guarantees. For practical efficiency, cuPDQPx integrates diagonal preconditioning, step-size selection, adaptive restart, and adaptive primal and quadratic weight updates. cuPDQPx is implemented in C and CUDA with optimized sparse matrix-vector products, kernel fusion, and CUDA Graphs. Numerical experiments on the Maros-Mészáros collection, a convex subset of QPLIB, and synthetic large-scale QPs demonstrate that cuPDQPx has strong numerical performance at both $10^{-4}$ and $10^{-8}$ tolerances.

math.OC↗

arXiv2026

H-PDLP: GPU-Accelerated Linear Programming via Generalized PDHG and Homogeneous Self-Dual Embedding

GPU-based first-order methods are emerging as an approach for solving large-scale linear programming (LP) problems. We present H-PDLP, a GPU first-order LP solver based on a generalized primal-dual hybrid gradient (PDHG) method for a homogeneous self-dual embedding of the LP, and study its numerical performance. The solver applies the restarted reflected Halpern iteration to the generalized PDHG operator, with the embedding encoding both primal-dual optimal solutions and certificates of primal or dual infeasibility. We derive a compact embedding for LPs with two-sided constraint and variable bounds in which the bounds enter through box cones, and we show that the resulting box-cone projections reduce to one-dimensional root-finding problems, so that each iteration remains matrix-free and well suited to GPUs. On the theoretical side, we establish conditions under which the iterates remain uniformly bounded away from the trivial zero solution of the embedding and derive complexity guarantees for the method. We also introduce MIPLIBInfeas, a reproducible benchmark of 229 infeasible and 105 unbounded LPs derived from MIPLIB 2017. Numerical experiments demonstrate the effectiveness of the proposed approach on both feasible LPs and infeasible or unbounded LPs. The source code for H-PDLP is publicly available at https://github.com/MIT-Lu-Lab/H-PDLP.

math.OC↗

arXiv2026

PDNQP: A GPU-based Factorization-free Method for Large-scale Nonconvex Quadratic Programming

Large-scale nonconvex quadratic programming remains challenging because the sparse matrix factorizations used by many existing solvers can incur substantial computational and memory costs and are difficult to parallelize efficiently on GPUs. We present PDNQP, a factorization-free first-order solver for finding stationary points of large-scale nonconvex quadratic programs. The method combines a proximal augmented-Lagrangian framework with restarted accelerated primal--dual hybrid gradient to solve the resulting strongly convex QP subproblems. Its key ingredients are a residual-form reformulation that avoids penalty-dependent normal matrices and preserves a fixed sparse constraint operator across outer iterations, and an adaptive strategy that coordinates inner-solve accuracy with outer progress. The resulting computations rely on sparse matrix--vector products, projections, and vector operations suited to GPU execution. On $107$ nonconvex CUTEst instances, PDNQP achieves the highest success rate among the tested solvers at both accuracy levels. With absolute and relative tolerances set to $10^{-4}$, it achieves speedups of $2.6\times$--$6.0\times$ in shifted geometric mean runtime; at $10^{-6}$, its aggregate runtime remains comparable to that of the fastest tested solvers. On $30$ large-scale synthetic nonconvex QPs with approximately one million variables each, PDNQP returns points satisfying the common external termination criteria on $27$ instances, while none of the other tested solvers succeeds under the same experimental protocol.

math.OC↗

arXiv2026

FlashFolio: A GPU-Accelerated Solver for Portfolio Optimization

We present FlashFolio, a GPU-accelerated solver for single-period and multi-period portfolio optimization with factor-based risk modeling, bid-offer spread costs, and nonlinear market impact. These models are widely used in portfolio construction and optimal execution, but become computationally challenging at large scale, especially in the multi-period setting. We benchmark FlashFolio against MOSEK on instances constructed from realistic market inputs. FlashFolio delivers consistent runtime improvements, achieving speedups of up to 12.9x in the single-period setting and 48x in the multi-period setting, while also exhibiting stronger robustness on challenging multi-period instances. Our results show that GPU-based optimization can help improve the practicality of large-scale portfolio optimization.

math.OC↗

arXiv2026

End-to-End Learning of Correlated Operating Reserve Requirements in Security-Constrained Economic Dispatch

Operating reserve requirements in security-constrained economic dispatch (SCED) depend strongly on the assumed correlation structure of renewable forecast errors, yet that structure is usually specified exogenously rather than learned for the dispatch task itself. This paper formulates correlated reserve-set design as an end-to-end trainable robust optimization problem: choose the ellipsoidal uncertainty-set shape to minimize robust dispatch cost subject to a target coverage requirement. By profiling the coverage constraint into a shape-dependent radius, the original bilevel problem becomes a single-stage differentiable objective, and KKT/dual information from the SCED solve provides task gradients without differentiating through the solver. For unknown distributions, a four-way train/tune/calibrate/test split combines a smoothed quantile-sensitivity estimator for training with split conformal calibration for deployment, yielding finite-sample marginal coverage under exchangeability and a consistent gradient estimator for the smoothed objective. The same task gradient can also be passed upstream to context-dependent encoders, which we report as a secondary extension. The framework is evaluated on the IEEE~118-bus system with a coupled SCED formulation that includes inter-zone transfer constraints. The learned static ellipsoid reduces dispatch cost by about 4.8\% relative to the Sample Covariance baseline while maintaining empirical coverage above the target level.

math.OC↗

arXiv2026

PDLP: A Practical First-Order Method for Large-Scale Linear Programming

We present PDLP, a practical first-order method for linear programming (LP) designed to solve large-scale LP problems. PDLP is based on the primal-dual hybrid gradient (PDHG) method applied to the minimax formulation of LP. PDLP incorporates several enhancements to PDHG, including diagonal preconditioning, presolving, adaptive step sizes, adaptive restarting, and feasibility polishing. Our algorithm is implemented in C++, available in Google's open-source OR-Tools library, and supports multithreading. To evaluate our method, we introduce a new collection of eleven large-scale LP problems with sizes ranging from 125 million to 6.3 billion nonzeros. PDLP solves eight of these instances to optimality gaps of 1% (with primal and dual feasibility errors of less than $10^{-8}$) within six days on a single machine. We also compare PDLP with Gurobi barrier, primal simplex, and dual simplex implementations. These traditional methods are designed to solve linear programs to much tighter optimality gaps but struggle to solve these instances. Gurobi barrier solves only three instances, exceeding our 1TB RAM limit on the other eight. While primal and dual simplex are more memory-efficient than the barrier method, they are slower and solve only three instances within six days. Compared with the conference version of this work (in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021)), the key new contributions are: (i) feasibility polishing, a technique that quickly finds solutions that are approximately optimal but almost exactly feasible (without which only two of the eleven problems can be solved); (ii) a multithreaded C++ implementation available in Google OR-Tools; and (iii) a new collection of large-scale LP problems. Note that the conference version should be referred to for comparisons with SCS and ablation studies, which we do not repeat in this paper.

math.OC↗

arXiv2026

Active set identification and rapid convergence for degenerate primal-dual problems

Primal-dual methods for solving convex optimization problems with functional constraints often exhibit a distinct two-stage behavior. Initially, they converge towards a solution at a sublinear rate. Then, after a certain point, the method identifies the set of active constraints and the convergence enters a faster local linear regime. Theory characterizing this phenomenon spans over three decades. However, most existing work only guarantees eventual identification of the active set and relies heavily on nondegeneracy conditions, such as strict complementarity, which often fail to hold in practice. We characterize mild conditions on the problem geometry and the algorithm under which this phenomenon provably occurs. Our guarantees are entirely nonasymptotic and, importantly, do not rely on strict complementarity. Our framework encompasses several widely-used algorithms, including the proximal point method, the primal-dual hybrid gradient method, the alternating direction method of multipliers, and the extragradient method.

math.OC↗

arXiv2026

MPAX: Mathematical Programming in JAX

We present MPAX (Mathematical Programming in JAX), an open-source first-order solver for large-scale linear programming (LP) and convex quadratic programming (QP) built natively in JAX. The primary goal of MPAX is to exploit modern machine learning infrastructure for large-scale mathematical programming, while also providing advanced mathematical programming algorithms that are easy to integrate into machine learning workflows. MPAX implements two PDHG variants, r2HPDHG for LP and rAPDHG for QP, together with diagonal preconditioning, adaptive restarts, adaptive step sizes, primal-weight updates, infeasibility detection, and feasibility polishing. Leveraging JAX's compilation and parallelization ecosystem, MPAX provides across-hardware portability, batched solving, distributed optimization, and automatic differentiation. We evaluate MPAX on CPUs, NVIDIA GPUs, and Google TPUs, observing substantial GPU speedups over CPU baselines and competitive performance relative to GPU-based codebases on standard LP/QP benchmarks. Our numerical experiments further demonstrate MPAX's capabilities in high-throughput batched solving, near-linear multi-GPU scaling for dense LPs, and efficient end-to-end differentiable training. The solver is publicly available at https://github.com/MIT-Lu-Lab/MPAX.

math.OC↗

arXiv2025

The Power of Linear Programming in Sponsored Listings Ranking: Evidence from a Large-Scale Field Experiment

Sponsored product advertisements constitute a major revenue source for online marketplaces such as Amazon, Walmart, and Alibaba. A key operational challenge in these systems lies in the Sponsored Listings Ranking (SLR) problem, that is, determining which items to include and how to rank them to balance short-term revenue with long-term relevance and user experience. Industry practice predominantly relies on score-based algorithms, which construct heuristic composite scores to rank items efficiently within strict real-time latency constraints. However, such methods offer limited control over objective trade-offs and cannot readily accommodate additional operational constraints. We propose and evaluate a Linear Programming (LP)-based algorithm as a principled alternative to score-based approaches. We first formulate the SLR problem as a constrained mixed integer programming (MIP) model and develop a dual-based algorithm that approximately solves its LP relaxation within 0.1 second, satisfying production-level latency requirements. In collaboration with a leading online marketplace, we conduct a 19-day field experiment encompassing approximately 329 million impressions. The LP-based algorithm significantly outperforms the industry-standard benchmark in key marketplace metrics, demonstrating both higher revenue and maintained relevance. Mechanism analyses reveal that the performance gains are most pronounced when the revenue-relevance tradeoff is stronger. Our framework also generalizes to settings with inventory, sales, or fairness constraints, offering a flexible and deployable optimization paradigm. The LP-based algorithm was deployed in production at our partner marketplace in January 2023, marking a rare large-scale implementation of a mathematically grounded ranking algorithm in real-world online advertising.

econ.GN↗

arXiv2025

cuPDLPx: A Further Enhanced GPU-Based First-Order Solver for Linear Programming

We introduce cuPDLPx, a further enhanced GPU-based first-order solver for linear programming. Building on the recently developed restarted Halpern PDHG for LP, cuPDLPx incorporates a number of new techniques, including a new restart criterion and a PID-controlled primal weight update. These improvements are carefully tailored for GPU architectures and deliver substantial computational gains. Across benchmark datasets, cuPDLPx achieves 2.5x-5x speedups on MIPLIB LP relaxations and 3x-6.8x on Mittelmann's benchmark set, with particularly strong improvements in high-accuracy and presolve-enabled settings. The solver is publicly available at https://github.com/MIT-Lu-Lab/cuPDLPx.

math.OC↗

arXiv2025

A New Crossover Algorithm for LP Inspired by the Spiral Dynamic of PDHG

Motivated by large-scale applications, there is a recent trend of research on using first-order methods for solving LP. Among them, PDLP, which is based on a primal-dual hybrid gradient (PDHG) algorithm, may be the most promising one. In this paper, we present a geometric viewpoint on the behavior of PDHG for LP. We demonstrate that PDHG iterates exhibit a spiral pattern with a closed-form solution when the variable basis remains unchanged. This spiral pattern consists of two orthogonal components: rotation and forward movement, where rotation improves primal and dual feasibility, while forward movement advances the duality gap. We also characterize the different situations in which basis change events occur. Inspired by the spiral behavior of PDHG, we design a new crossover algorithm to obtain a vertex solution from any optimal LP solution. This approach differs from traditional simplex-based crossover methods. Our numerical experiments demonstrate the effectiveness of the proposed algorithm, showcasing its potential as an alternative option for crossover.

math.OC↗

arXiv2025

A Practical and Optimal First-Order Method for Large-Scale Convex Quadratic Programming

Convex quadratic programming (QP) is an important class of optimization problem with wide applications in practice. The classic QP solvers are based on either simplex or barrier method, both of which suffer from the scalability issue because their computational bottleneck is solving linear equations. In this paper, we design and analyze a first-order method for QP, called restarted accelerated primal-dual hybrid gradient (rAPDHG), whose computational bottleneck is matrix-vector multiplication. We show that rAPDHG has a linear convergence rate to an optimal solution when solving QP, and the obtained linear rate is optimal among a wide class of primal-dual methods. Furthermore, we connect the linear rate with a sharpness constant of the KKT system of QP, which is a standard quantity to measure the hardness of a continuous optimization problem. Numerical experiments demonstrate that both restarts and acceleration can significantly improve the performance of the algorithm. Lastly, we present PDQP.jl, an open-source solver based on rAPDHG that can be run on both GPU and CPU. With a numerical comparison with SCS and OSQP on standard QP benchmark sets and large-scale synthetic QP instances, we demonstrate the effectiveness of rAPDHG for solving QP.

math.OC↗

arXiv2025

Enhanced PDHG for Linear Programming with Online Preconditioning

We present an online preconditioning technique for the primal-dual hybrid gradient (PDHG) algorithm for linear programming (LP). The method adaptively updates primal and dual preconditioners using an online optimization framework. To improve its practical performance, we introduce several algorithmic enhancements, including using normalized online loss functions and updating preconditioners infrequently. We implement the technique on top of vanilla PDHG and the GPU-based LP solver cuPDLP.jl, and benchmark its performance on standard LP datasets. Our numerical experiments demonstrate that online preconditioning effectively reduces both iteration counts and overall solving time.

math.OC↗

arXiv2025

An Overview of GPU-based First-Order Methods for Linear Programming and Extensions

The rapid progress in GPU computing has revolutionized many fields, yet its potential in mathematical programming, such as linear programming (LP), has only recently begun to be realized. This survey aims to provide a comprehensive overview of recent advancements in GPU-based first-order methods for LP, with a particular focus on the design and development of cuPDLP. We begin by presenting the design principles and algorithmic foundation of the primal-dual hybrid gradient (PDHG) method, which forms the core of the solver. Practical enhancements, such as adaptive restarts, preconditioning, Halpern-type acceleration and infeasibility detection, are discussed in detail, along with empirical comparisons against industrial-grade solvers, highlighting the scalability and efficiency of cuPDLP. We also provide a unified theoretical framework for understanding PDHG, covering both classical and recent results on sublinear and linear convergence under sharpness conditions. Finally, we extend the discussion to GPU-based optimization beyond LP, including quadratic, semidefinite, conic, and nonlinear programming.

math.OC↗

arXiv2025

New Understandings and Computation on Augmented Lagrangian Methods for Low-Rank Semidefinite Programming

Augmented Lagrangian Method (ALM) combined with Burer-Monteiro (BM) factorization, dubbed ALM-BM, offers a powerful approach for solving large-scale low-rank semidefinite programs (SDPs). Despite its empirical success, the theoretical understandings of the resulting non-convex ALM-BM subproblems, particularly concerning their structural properties and efficient subproblem solvability by first-order methods, still remain limited. This work addresses these notable gaps by providing a rigorous theoretical analysis. We demonstrate that, under appropriate regularity of the original SDP, termed as primal simplicity, ALM subproblems inherit crucial properties such as low-rankness and strict complementarity when the dual variable is localized. Furthermore, ALM subproblems are shown to enjoy a quadratic growth condition, building on which we prove that the non-convex ALM-BM subproblems can be solved to global optimality by gradient descent, achieving linear convergence under conditions of local initialization and dual variable proximity. Through illustrative examples, we further establish the necessity of these local assumptions, revealing them as inherent characteristics of the problem structure. Motivated by these theoretical insights, we propose ALORA, a rank-adaptive augmented Lagrangian method that builds upon the ALM-BM framework, which dynamically adjusts the rank using spectral information and explores negative curvature directions to navigate the nonconvex landscape. Exploiting modern GPU computing architectures, ALORA exhibits strong numerical performance, solving SDPs with tens of millions of dimensions in hundreds of seconds.

math.OC↗