arXiv subjects

Srijan Sengupta

Publications and source records attributed to Srijan Sengupta.

At least 19 recordsLinked to original sources

arXiv2026

Short-term forecasting of wildfire spread: A network epidemiology approach

Wildfire spread poses substantial environmental and public-health risks, motivating interpretable models for short-term forecasting. We develop a statistical framework that combines cellular automata with ideas from network epidemiology to model wildfire evolution across a spatial lattice. Each grid cell is classified as available, burning, or consumed. State transitions distinguish spread from burning neighbors, intrinsic ignition, and cessation of burning, with transition rates linked to meteorological and environmental covariates. A likelihood-based estimation procedure yields transition-specific covariate effects and probabilistic forecasts of cell states. We assess forecasting performance in a simulation study and in applications to the 2018 California wildfires and the 2019-2020 Australian wildfires. We also compare the method with a published forecasting approach using the 2017 Haypress fire. The results show strong short-term discrimination in many settings, with reduced accuracy at longer forecast horizons and during abrupt fire expansion. The framework provides an interpretable basis for studying wildfire dynamics and identifies opportunities to improve ignition forecasts through richer spatial and observation models.

stat.AP↗

arXiv2026

Predictive Subsampling for Scalable Inference in Networks

Current methods for statistical inference in networks often encounter substantial computational bottlenecks when applied to the massive network datasets that are increasingly common across scientific domains. In this paper, we develop \textit{Predictive Subsampling} (\texttt{PredSub}), a scalable framework for estimation and two-sample testing in networks. The central idea is to replace a full-sample estimation procedure by estimation on a random subsample, followed by out-of-sample prediction of the remaining vertices. This construction exploits the fact that the subsample provides an inferential anchor, which enables each remaining vertex to be incorporated in a \textit{predictive} manner through a fast vector operation. Building on this estimator, we develop two scalable procedures for two-sample testing, namely \texttt{PredSubTest} and \texttt{PureSubTest}. We establish finite-sample error bounds as well as estimation and testing consistency of the proposed methods in both Frobenius and two-to-infinity norms. These results formally characterize the trade-offs between statistical accuracy and computational efficiency with respect to the subsample size, the choice of test statistic, and the choice of norm. We demonstrate the empirical performance of the proposed methods through detailed simulation studies and two real-world applications involving DBLP coauthorship networks and the Cannes 2013 social media networks.

stat.ME↗

arXiv2026

Understanding Geopolitical Alignments Through Covariate Augmented Spectral Clustering of Heterogeneous UNGA Voting Data

Community detection is a fundamental problem in network analysis. While many existing methods focus on homogeneous networks, real world networks are often heterogeneous, involving multiple node types and interaction mechanisms. In addition, node specific covariates frequently provide valuable information about the underlying community structure. Existing methodologies typically account for either network heterogeneity or covariate information, but seldom both simultaneously. In this paper, we propose a covariate assisted spectral clustering framework for heterogeneous networks that jointly utilizes network connectivity in a heterogeneous setting and node level covariates. The proposed method extends covariate assisted spectral clustering to heterogeneous settings and operates directly on the heterogeneous network without relying on projection based simplifications. Under a heterogeneous node contextualized stochastic blockmodel, we establish theoretical guarantees for the proposed procedure, including concentration results, eigenspace perturbation bounds, and an explicit upper bound on the misclustering rate. Simulation studies demonstrate that incorporating covariate information substantially improves community recovery and consistently outperforms several benchmark methods. We further apply the proposed framework to United Nations General Assembly voting data, where it reveals meaningful geopolitical structures by combining voting interactions with auxiliary covariate information.

stat.AP↗

arXiv2026

Two-Sample Testing for Multivariate Cross-Correlation Functions with Applications to Gut-Brain Reward Learning

Cross-correlation functions (CCFs) are classical tools for studying lead-lag relationships between paired time series, but they are most often used descriptively rather than inferentially. Motivated by mouse experiments on gut-brain interactions in reward learning, we carry out a two-sample hypothesis test for formal statistical inference on collections of subject-specific CCF curves. In our application, each experimental session yields two related CCFs describing the temporal association of dopamine activity with locomotor velocity and acceleration, which leads naturally to a multivariate functional data formulation. We treat each empirical CCF as a functional observation indexed by lag and test equality of mean multivariate CCF functions across groups using integrated and maximum-type global statistics, \(F_{\mathrm{int}}\) and \(F_{\max}\), constructed from pointwise Hotelling \(T^2\) statistics. The integrated test targets broad differences across the lag domain, whereas the maximum test is sensitive to local differences. Applied to free-feeding and intragastric infusion datasets, the proposed methods detect substantial differences in dopamine-locomotion coupling across brain region and biological sex in the free-feeding experiment, with more selective effects in the infusion setting. The proposed framework provides a flexible and rigorous FDA-based approach for comparing dynamic dependence structures across experimental conditions.

stat.AP↗

arXiv2026

Network Cross-Validation and Model Selection via Subsampling

Complex and larger networks are becoming increasingly prevalent in scientific applications in various domains. Although a number of models and methods exist for such networks, cross-validation on networks remains challenging due to the unique structure of network data. In this paper, we propose a general cross-validation procedure called NETCROP (NETwork CRoss-Validation using Overlapping Partitions). The key idea is to divide the original network into multiple subnetworks with a shared overlap part, producing training sets consisting of the subnetworks and a test set with the node pairs between the subnetworks. This train-test split provides the basis for a network cross-validation procedure that can be applied on a wide range of model selection and parameter tuning problems for networks. The method is computationally efficient for large networks as it uses smaller subnetworks for the training step. We provide methodological details and theoretical guarantees for several model selection and parameter tuning tasks using NETCROP. Numerical results demonstrate that NETCROP performs accurate cross-validation on a diverse set of network model selection and parameter tuning problems. The results also indicate that NETCROP is computationally much faster while being often more accurate than the existing methods for network cross-validation.

stat.ME↗

arXiv2026

A label-switching algorithm for fast core-periphery identification

Core-periphery (CP) structure is frequently observed in networks where the nodes form two distinct groups: a small, densely interconnected core and a sparse periphery. Borgatti and Everett (2000) proposed one of the most popular methods to identify and quantify CP structure by comparing the observed network with an ``ideal'' CP structure. While this metric has been widely used, an improved algorithm is still needed. In this work, we detail a greedy, label-switching algorithm to identify CP structure that is both fast and accurate. By leveraging a mathematical reformulation of the CP metric, our proposed heuristic offers an order-of-magnitude improvement on the number of operations compared to a naive implementation. We prove that the algorithm monotonically ascends to a local maximum while consistently yielding solutions within 90% of the global optimum on small toy networks. On synthetic networks, our algorithm exhibits superior classification accuracies and run-times compared to a popular competing method, and on one-real world network, it is 340 times faster.

stat.CO↗

arXiv2026

Uncertainty Quantification for Named Entity Recognition via Full-Sequence and Subsequence Conformal Prediction

Named Entity Recognition (NER) serves as a foundational component in many natural language processing (NLP) pipelines. However, current NER models typically output a single predicted label sequence without any accompanying measure of uncertainty, leaving downstream applications vulnerable to cascading errors. In this paper, we introduce a general framework for adapting sequence-labeling-based NER models to produce uncertainty-aware prediction sets. These prediction sets are collections of full-sentence labelings that are guaranteed to contain the correct labeling with a user-specified confidence level. This approach serves a role analogous to confidence intervals in classical statistics by providing formal guarantees about the reliability of model predictions. Our method builds on conformal prediction, which offers finite-sample coverage guarantees under minimal assumptions. We design efficient nonconformity scoring functions to construct efficient, well-calibrated prediction sets that support both unconditional and class-conditional coverage. This framework accounts for heterogeneity across sentence length, language, entity type, and number of entities within a sentence. Empirical experiments on four NER models across three benchmark datasets demonstrate the broad applicability, validity, and efficiency of the proposed methods.

cs.CL↗

arXiv2025

A Unified Framework for Community Detection and Model Selection in Blockmodels

Blockmodels are a foundational tool for modeling community structure in networks, with the stochastic blockmodel (SBM), degree-corrected blockmodel (DCBM), and popularity-adjusted blockmodel (PABM) forming a natural hierarchy of increasing generality. While community detection under these models has been extensively studied, much less attention has been paid to the model selection problem, i.e., determining which model best fits a given network. Building on recent theoretical insights about the spectral geometry of these models, we propose a unified framework for simultaneous community detection and model selection across the full blockmodel hierarchy. A key innovation is the use of loss functions that serve a dual role: they act as objective functions for community detection and as test statistics for hypothesis testing. We develop a greedy algorithm to minimize these loss functions and establish theoretical guarantees for exact label recovery and model selection consistency under each model. Extensive simulation studies demonstrate that our method achieves high accuracy in both tasks, outperforming or matching state-of-the-art alternatives. Applications to five real-world networks further illustrate the interpretability and practical utility of our approach. R code for implementing the method is available at https://github.com/subhankarbhadra/model-selection.

stat.ME↗

arXiv2025

Detecting and Localizing Anomalous Cliques in Inhomogeneous Networks using Egonets

Cliques, or fully connected subgraphs, are among the most important and well-studied graph motifs in network science. We consider the problem of finding a statisti- cally anomalous clique hidden in a large network. There are two parts to this problem: (1) detection, i.e., determining whether an anomalous clique is present, and (2) localization, i.e., determining which vertices of the network constitute the detected clique. While this problem has been extensively studied under the homogeneous Erdos-Renyi model, little progress has been made beyond this simple setting, and no existing method can perform detection and localization in inhomogeneous networks within finite time. To address this gap, we first show that in homogeneous networks, the anomalousness of a clique depends solely on its size. This property does not carry over to inhomogeneous networks, where the identity of the vertices forming the clique plays a critical role, and a smaller clique can be more anomalous than a larger one. Building on this insight, we propose a unified method for clique detection and localization based on a class of subgraphs called egonets. The proposed method generalizes to a wide variety of inhomogeneous network models and is naturally amenable to parallel computing. We establish the theoretical properties of the proposed method and demonstrate its empirical performance through simulation studies and application to two real world networks.

stat.ME↗

arXiv2025

Statistical inference for core-periphery structures

Core-periphery (CP) structure is an important meso-scale network property where nodes group into a small, densely interconnected {core} and a sparse {periphery} whose members primarily connect to the core rather than to each other. While this structure has been observed in numerous real-world networks, there has been minimal statistical formalization of it. In this work, we develop a statistical framework for CP structures by introducing a model-agnostic and generalizable population parameter which quantifies the strength of a CP structure at the level of the data-generating mechanism. We study this parameter under four canonical random graph models and establish theoretical guarantees for label recovery, including exact label recovery. Next, we construct intersection tests for validating the presence and strength of a CP structure under multiple null models, and prove theoretical guarantees for type I error and power. These tests provide a formal distinction between exogenous (or induced) and endogenous (or intrinsic) CP structure in heterogeneous networks, enabling a level of structural resolution that goes beyond merely detecting the presence of CP structure. The proposed methods show excellent performance on synthetic data, and our applications demonstrate that statistically significant CP structure is somewhat rare in real-world networks.

stat.ME↗

arXiv2025

Scalable community detection in massive networks via predictive assignment

Massive network datasets are becoming increasingly common in scientific applications. Existing community detection methods encounter significant computational challenges for such massive networks due to two reasons. First, the full network needs to be stored and analyzed on a single server, leading to high memory costs. Second, existing methods typically use matrix factorization or iterative optimization using the full network, resulting in high runtimes. We propose a strategy called \textit{predictive assignment} to enable computationally efficient community detection while ensuring statistical accuracy. The core idea is to avoid large-scale matrix computations by breaking up the task into a smaller matrix computation plus a large number of vector computations that can be carried out in parallel. Under the proposed method, community detection is carried out on a small subgraph to estimate the relevant model parameters. Next, each remaining node is assigned to a community based on these estimates. We prove that predictive assignment achieves strong consistency under the stochastic blockmodel and its degree-corrected version. We also demonstrate the empirical performance of predictive assignment on simulated networks and two large real-world datasets: DBLP (Digital Bibliography \& Library Project), a computer science bibliographical database, and the Twitch Gamers Social Network.

stat.ME↗

arXiv2025

A Bootstrap-based Method for Testing Network Similarity

This paper studies the matched network inference problem, where the goal is to determine if two networks, defined on a common set of nodes, exhibit a specific form of stochastic similarity. Two notions of similarity are considered: (i) equality, i.e., testing whether the networks arise from the same random graph model, and (ii) scaling, i.e., testing whether their probability matrices are proportional for some unknown scaling constant. We develop a testing framework based on a parametric bootstrap approach and a Frobenius norm-based test statistic. The proposed approach is highly versatile as it covers both the equality and scaling problems, and ensures adaptability under various model settings, including stochastic blockmodels, Chung-Lu models, and random dot product graph models. We establish theoretical consistency of the proposed tests and demonstrate their empirical performance through extensive simulations under a wide range of model classes. Our results establish the flexibility and computational efficiency of the proposed method compared to existing approaches. We also report a real-world application involving the Aarhus network dataset, which reveals meaningful sociological patterns across different communication layers.

stat.ME↗

arXiv2024

Testing for the Network Small-World Property

Researchers have long observed that the ``small-world" property, which combines the concepts of high transitivity or clustering with a low average path length, is ubiquitous for networks obtained from a variety of disciplines, including social sciences, biology, neuroscience, and ecology. However, we find several shortcomings of the currently prevalent definition and detection methods rendering the concept less powerful. First, the widely used \textit{small world coefficient} metric combines high transitivity with a low average path length in a single measure that confounds the two separate aspects. We find that the value of the metric is dominated by transitivity, and in several cases, networks get flagged as ``small world" solely because of their high transitivity. Second, the detection methods lack a formal statistical inference. Third, the comparison is typically performed against simplistic random graph models as the baseline, ignoring well-known network characteristics and risks confounding the small world property with other network properties. We decouple the properties of high transitivity and low average path length as separate events to test for. Then we define the property as a statistical test between a suitable null hypothesis and a superimposed alternative hypothesis. We propose a parametric bootstrap test with several null hypothesis models to allow a wide range of background structures in the network. In addition to the bootstrap tests, we also propose an asymptotic test under the Erdös-Renýi null model for which we provide theoretical guarantees on the asymptotic level and power. Our theoretical results include asymptotic distributions of clustering coefficient for various asymptotic growth rates on the probability of an edge. Applying the proposed methods to a large number of network datasets, we uncover new insights about their small-world property.

stat.ME↗

arXiv2024

Scalable Resampling in Massive Generalized Linear Models via Subsampled Residual Bootstrap

Residual bootstrap is a classical method for statistical inference in regression settings. With massive data sets becoming increasingly common, there is a demand for computationally efficient alternatives to residual bootstrap. We propose a simple and versatile scalable algorithm called subsampled residual bootstrap (SRB) for generalized linear models (GLMs), a large class of regression models that includes the classical linear regression model as well as other widely used models such as logistic, Poisson and probit regression. We prove consistency and distributional results that establish that the SRB has the same theoretical guarantees under the GLM framework as the classical residual bootstrap, while being computationally much faster. We demonstrate the empirical performance of SRB via simulation studies and a real data analysis of the Forest Covertype data from the UCI Machine Learning Repository.

stat.ME↗

arXiv2023

A generalized hypothesis test for community structure in networks

Researchers theorize that many real-world networks exhibit community structure where within-community edges are more likely than between-community edges. While numerous methods exist to cluster nodes into different communities, less work has addressed this question: given some network, does it exhibit statistically meaningful community structure? We answer this question in a principled manner by framing it as a statistical hypothesis test in terms of a general and model-agnostic community structure parameter. Leveraging this parameter, we propose a simple and interpretable test statistic used to formulate two separate hypothesis testing frameworks. The first is an asymptotic test against a baseline value of the parameter while the second tests against a baseline model using bootstrap-based thresholds. We prove theoretical properties of these tests and demonstrate how the proposed method yields rich insights into real-world data sets.

cs.SI↗

arXiv2023

Statistical Network Analysis: Past, Present, and Future

This article provides a brief overview of statistical network analysis, a rapidly evolving field of statistics, which encompasses statistical models, algorithms, and inferential methods for analyzing data in the form of networks. Particular emphasis is given to connecting the historical developments in network science to today's statistical network analysis, and outlining important new areas for future research. This invited article is intended as a book chapter for the volume "Frontiers of Statistics and Data Science" edited by Subhashis Ghoshal and Anindya Roy for the International Indian Statistical Association Series on Statistics and Data Science, published by Springer. This review article covers the material from the short course titled "Statistical Network Analysis: Past, Present, and Future" taught by the author at the Annual Conference of the International Indian Statistical Association, June 6-10, 2023, at Golden, Colorado.

stat.ME↗

arXiv2023

Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology

The rise of internet-based services and products in the late 1990's brought about an unprecedented opportunity for online businesses to engage in large scale data-driven decision making. Over the past two decades, organizations such as Airbnb, Alibaba, Amazon, Baidu, Booking, Alphabet's Google, LinkedIn, Lyft, Meta's Facebook, Microsoft, Netflix, Twitter, Uber, and Yandex have invested tremendous resources in online controlled experiments (OCEs) to assess the impact of innovation on their customers and businesses. Running OCEs at scale has presented a host of challenges requiring solutions from many domains. In this paper we review challenges that require new statistical methodologies to address them. In particular, we discuss the practice and culture of online experimentation, as well as its statistics literature, placing the current methodologies within their relevant statistical lineages and providing illustrative examples of OCE applications. Our goal is to raise academic statisticians' awareness of these new research opportunities to increase collaboration between academia and the online industry.

stat.AP↗

arXiv2023

Core-periphery structure in networks: a statistical exposition

Many real-world networks are theorized to have core-periphery structure consisting of a densely-connected core and a loosely-connected periphery. While this phenomenon has been extensively studied in a range of scientific disciplines, it has not received sufficient attention in the statistics community. In this expository article, our goal is to raise awareness about this topic and encourage statisticians to address the many open inference problems in this area. To this end, we first summarize the current research landscape by reviewing the metrics and models that have been used for quantitative studies on core-periphery structure. Next, we formulate and explore various inferential problems in this context, such as estimation, hypothesis testing, and Bayesian inference, and discuss related computational techniques. We also outline the multidisciplinary scientific impact of core-periphery structure in a number of real-world networks. Throughout the article, we provide our own interpretation of the literature from a statistical perspective, with the goal of prioritizing open problems where contribution from the statistics community will be most effective and important.

stat.ME↗