A First Course in Sparse Optimization

Abstract

This article aims to provide a comprehensive overview of sparse optimization, with a focus on both sparse signal recovery and sparse regularization techniques. We will begin by exploring the foundations of sparse optimization, delving into the mathematical tools and models that underpin sparse signal recovery and LASSO. We will then discuss key algorithms for both sparse recovery (e.g., basis pursuit, matching pursuit) and sparse regularization (e.g., LASSO, elastic net), along with their applications in real-world problems. Throughout the text, we balance intuitive explanations with rigorous mathematical formulations to provide a comprehensive resource for both newcomers and experts in the field. Our aim is twofold: to provide a self-contained entry point for students and researchers new to the field, and to offer a rigorous reference for practitioners seeking to apply sparse optimization in science and engineering.