arXiv subjects

Zhanran Lin

Publications and source records attributed to Zhanran Lin.

3 recordsLinked to original sources

arXiv2024

Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivity

One of the most intriguing findings in the structure of neural network landscape is the phenomenon of mode connectivity: For two typical global minima, there exists a path connecting them without barrier. This concept of mode connectivity has played a crucial role in understanding important phenomena in deep learning. In this paper, we conduct a fine-grained analysis of this connectivity phenomenon. First, we demonstrate that in the overparameterized case, the connecting path can be as simple as a two-piece linear path, and the path length can be nearly equal to the Euclidean distance. This finding suggests that the landscape should be nearly convex in a certain sense. Second, we uncover a surprising star-shaped connectivity: For a finite number of typical minima, there exists a center on minima manifold that connects all of them simultaneously via linear paths. These results are provably valid for linear networks and two-layer ReLU networks under a teacher-student setup, and are empirically supported by models trained on MNIST and CIFAR-10.

cs.LG↗

arXiv2023

Fast Rerandomization via the BRAIN

Randomized experiments are a crucial tool for causal inference in many different fields. Rerandomization addresses any covariate imbalance in such experiments by resampling treatment assignments until certain balance criteria are satisfied. However, rerandomization based on na\"ive acceptance-rejection sampling is computationally inefficient, especially when numerous independent assignments are required to perform randomization-based statistical inference. Existing acceleration methods are suboptimal and not applicable in structured experiments, including stratified and clustered experiments. Based on metaheuristics in integer programming, we propose BRAIN -- a novel computationally-lightweight methodology that ensures covariate balance in randomized experiments while significantly accelerating the computation. Our BRAIN method provides unbiased treatment effect estimators with reduced variance compared to complete randomization, preserving the desirable statistical properties of traditional rerandomization. Simulation studies and a real data example demonstrate the benefits of our method in fast sampling while retaining the appealing statistical guarantees.

stat.ME↗

arXiv2023

Joint Coverage Regions: Simultaneous Confidence and Prediction Sets

We introduce Joint Coverage Regions (JCRs), which unify confidence intervals and prediction regions in frequentist statistics. Specifically, joint coverage regions aim to cover a pair formed by an unknown fixed parameter (such as the mean of a distribution), and an unobserved random datapoint (such as the outcomes associated to a new test datapoint). The first corresponds to a confidence component, while the second corresponds to a prediction part. In particular, our notion unifies classical statistical methods such as the Wald confidence interval with distribution-free prediction methods such as conformal prediction. We show how to construct finite-sample valid JCRs when a conditional pivot is available; under the same conditions where exact finite-sample confidence and prediction sets are known to exist. We further develop efficient JCR algorithms, including split-data versions by introducing adequate sets to reduce the cost of repeated computation. We illustrate the use of JCRs in statistical problems such as constructing efficient prediction sets when the parameter space is structured.

stat.ME↗