Manifold Regression

Abstract

Conventional statistical modeling typically assumes a one-to-one mapping between input and output variables, where one-to-one is used in the predictive sense that a specified input results in a unique output. Many scientific and engineering systems, however, exhibit non-one-to-one(noto) input-output relations. In a noto relation, the same input may correspond to multiple admissible outputs, so the conventional statistical models may not be appropriate. This paper develops manifold regression, a parametric modeling framework for such noto input-output relations, which represents the underlying input-output relation through a latent manifold. The manifold is specified upto an unknown finite dimensional parameter and we estimate the unknown parameters from ordinary input-output observations by regularized profile optimization with a robust solver. Once the manifold has been learned, prediction is obtained by slicing the estimated manifold along a specified coordinate value. Because a slice may contain multiple latent roots, the resulting manifold prediction is naturally set-valued. This formulation extends predictive learning beyond conventional one-to-one statistical models while containing classical regression and inverse prediction as special cases. Theoretical results are established to justify the manifold regression model and its sliced prediction sets; and case studies are used to demonstrate stable performance across representative noto cases.