arXiv subjects

Robert Kohn

Publications and source records attributed to Robert Kohn.

At least 19 recordsLinked to original sources

arXiv2026

Time-Varying Multi-Seasonal ARMA Models

We propose an ARMA model that allows for multiple seasonal periods and time varying parameters in both regular and seasonal components, building upon previous work for pure AR processes and the conditional likelihood. The model is parameterized to ensure stability and invertibility at each time point. The parameter evolution is governed by dynamic shrinkage processes, enabling extended periods of essentially constant parameters, gradual changes, and abrupt shifts. The model includes a stochastic volatility component to account for potentially heterogeneous noise, also modeled by a dynamic shrinkage process. A Gibbs sampler is developed using the exact likelihood, with separate updating steps for the latent errors and the unobserved pre-sample history of the process. The time-varying AR and MA parameters are sampled jointly using a fast posterior sampler based on the extended Kalman filter. The model and the efficiency of the Gibbs sampler are evaluated using simulated and real data. A case study on monthly air passenger data in the US during 1990-2024 reveals significant changes in seasonality during the Covid-19 pandemic.

stat.ME↗

arXiv2026

Spectral subsampling MCMC for L\'evy-driven continuous-time ARMA models with expensive likelihood contributions

Subsampling-based Markov chain Monte Carlo (MCMC) algorithms aim to accelerate Bayesian inference by evaluating the likelihood using only a subset of the data at each iteration. However, in many standard tall-data applications, individual likelihood contributions are inexpensive to evaluate and the resulting reductions in actual computing time are often substantially smaller than the nominal reduction in data size due to computational overhead. We study a different computational regime arising in frequency-domain inference for continuous-time processes observed at equally spaced discrete time points. This gives rise to aliasing, whereby each contribution to the Whittle likelihood requires summation over shifted frequency components, unlike standard discrete-time spectral settings where spectral evaluations do not require such summation. We demonstrate that this structure makes subsampling MCMC, a subsampling-based MCMC approach that estimates the log-likelihood using data subsampling and efficient control variates, particularly effective for reducing computational cost. We illustrate the approach for Bayesian frequency-domain inference in discretely observed continuous-time autoregressive moving average models driven by finite second-moment L\'evy processes.

stat.CO↗

arXiv2025

Calibrated Bayesian inference for random fields on large irregular domains using the debiased spatial Whittle likelihood

Bayesian inference for stationary random fields is computationally demanding. Whittle-type likelihoods in the frequency domain based on the fast Fourier Transform (FFT) have several appealing features: i) low computational complexity of only $\mathcal{O}(n \log n)$, where $n$ is the number of spatial locations, ii) robustness to assumptions of the data-generating process, iii) ability to handle missing data and irregularly spaced domains, and iv) flexibility in modelling the covariance function via the spectral density directly in the spectral domain. It is well known, however, that the Whittle likelihood suffers from bias and low efficiency for spatial data. The debiased Whittle likelihood is a recently proposed alternative with better frequentist properties. We propose a methodology for Bayesian inference for stationary random fields using the debiased spatial Whittle likelihood, with an adjustment from the composite likelihood literature. The adjustment is shown to give a well-calibrated Bayesian posterior as measured by coverage properties of credible sets, without sacrificing the quasi-linear computation time. We apply the method to simulated data and two real datasets.

stat.ME↗

arXiv2025

Fast Variational Boosting for Latent Variable Models

We consider the problem of estimating complex statistical latent variable models using variational Bayes methods. These methods are used when exact posterior inference is either infeasible or computationally expensive, and they approximate the posterior density with a family of tractable distributions. The parameters of the approximating distribution are estimated using optimisation methods. This article develops a flexible Gaussian mixture variational approximation, where we impose sparsity in the precision matrix of each Gaussian component to reflect the appropriate conditional independence structure in the model. By introducing sparsity in the precision matrix and parameterising it using the Cholesky factor, each Gaussian mixture component becomes parsimonious (with a reduced number of non-zero parameters), while still capturing the dependence in the posterior distribution. Fast estimation methods based on global and local variational boosting moves combined with natural gradients and variance reduction methods are developed. The local boosting moves adjust an existing mixture component, and optimisation is only carried out on a subset of the variational parameters of a new component. The subset is chosen to target improvement of the current approximation in aspects where it is poor. The local boosting moves are fast because only a small number of variational parameters need to be optimised. The efficacy of the approach is illustrated by using simulated and real datasets to estimate generalised linear mixed models and state space models.

stat.ME↗

arXiv2025

A Beta Cauchy-Cauchy (BECCA) shrinkage prior for Bayesian variable selection

This paper introduces a novel Bayesian approach for variable selection in high-dimensional and potentially sparse regression settings. Our method replaces the indicator variables in the traditional spike and slab prior with continuous, Beta-distributed random variables and places half Cauchy priors over the parameters of the Beta distribution, which significantly improves the predictive and inferential performance of the technique. Similar to shrinkage methods, our continuous parameterization of the spike and slab prior enables us explore the posterior distributions of interest using fast gradient-based methods, such as Hamiltonian Monte Carlo (HMC), while at the same time explicitly allowing for variable selection in a principled framework. We study the frequentist properties of our model via simulation and show that our technique outperforms the latest Bayesian variable selection methods in both linear and logistic regression. The efficacy, applicability and performance of our approach, are further underscored through its implementation on real datasets.

stat.ME↗

arXiv2024

Spectral domain likelihoods for Bayesian inference in time-varying parameter models

Inference for locally stationary processes is often based on some local Whittle-type approximation of the likelihood function defined in the frequency domain. The main reasons for using such a likelihood approximation is that i) it has substantially lower computational cost and better scalability to long time series compared to the time domain likelihood, particularly when used for Bayesian inference via Markov Chain Monte Carlo (MCMC), ii) convenience when the model itself is specified in the frequency domain, and iii) it provides access to bootstrap and subsampling MCMC which exploits the asymptotic independence of Fourier transformed data. Most of the existing literature compares the asymptotic performance of the maximum likelihood estimator (MLE) from such frequency domain likelihood approximation with the exact time domain MLE. Our article uses three simulation studies to assess the finite-sample accuracy of several frequency domain likelihood functions when used to approximate the posterior distribution in time-varying parameter models. The methods are illustrated on an application to egg price data.

stat.ME↗

arXiv2024

Time-Varying Multi-Seasonal AR Models

We propose a seasonal AR model with time-varying parameter processes in both the regular and seasonal parameters. The model is parameterized to guarantee stability at every time point and can accommodate multiple seasonal periods. The time evolution is modeled by dynamic shrinkage processes to allow for long periods of essentially constant parameters, periods of rapid change, and abrupt jumps. A Gibbs sampler is developed with a particle Gibbs update step for the AR parameter trajectories. We show that the near-degeneracy of the model, caused by the dynamic shrinkage processes, makes it challenging to estimate the model by particle methods. To address this, a more robust, faster and accurate approximate sampler based on the extended Kalman filter is proposed. The model and the numerical effectiveness of the Gibbs sampler are investigated on simulated data. An application to more than a century of monthly US industrial production data shows interesting clear changes in seasonality over time, particularly during the Great Depression and the recent Covid-19 pandemic. Keywords: Bayesian inference; Extended Kalman filter; Particle MCMC; Seasonality.

stat.ME↗

arXiv2024

Dynamic linear regression models for forecasting time series with semi long memory errors

Dynamic linear regression models forecast the values of a time series based on a linear combination of a set of exogenous time series while incorporating a time series process for the error term. This error process is often assumed to follow a stationary autoregressive integrated moving average (ARIMA) model, or its seasonal variants, which are unable to capture a long-range dependence structure (long memory) of the error process. We propose a novel dynamic linear regression model that incorporates the long-range dependence feature of the errors and show that the proposed error process may: (i) have a significant impact on the posterior uncertainty of the estimated regression parameters and (ii) improve the model's forecasting ability. We develop a Markov chain Monte Carlo method to fit general dynamic linear regression models based on a frequency domain approach that enables fast, asymptotically exact Bayesian inference for large datasets. We demonstrate that our approximate algorithm is faster than the traditional time domain approaches, such as the Kalman filter and the multivariate Gaussian likelihood, while producing a highly accurate approximation to the posterior. The method is illustrated in simulated examples and two energy forecasting applications, showing that it outperforms approaches that do not account for semi-long memory, as well as a state-of-the-art neural-network-based forecasting procedure.

stat.ME↗

arXiv2024

Analysing symbolic data by pseudo-marginal methods

Symbolic data analysis (SDA) aggregates large individual-level datasets into a small number of distributional summaries, such as random rectangles or random histograms. The inference is carried out using these summaries in place of the original dataset, resulting in computational gains at the loss of some information. In likelihood-based SDA, the likelihood function is characterised by an integral with a large exponent, which limits the method's utility as for typical models the integral is unavailable in closed form. In addition, the likelihood function is known to produce biased parameter estimates in some circumstances. Our article develops a Bayesian framework for SDA methods in these settings that resolves the issues resulting from integral intractability and biased parameter estimation using pseudo-marginal Markov chain Monte Carlo methods. We develop an exact but computationally expensive method based on path sampling and the Poisson estimator, and a much faster, but approximate, method based on a Taylor expansion. Through simulation and real-data examples we demonstrate the performance of the developed methods, showing large reductions in computation time compared to the full-data analysis, with only a small loss of information.

stat.ME↗

arXiv2024

ProDAG: Projected Variational Inference for Directed Acyclic Graphs

Directed acyclic graph (DAG) learning is a central task in structure discovery and causal inference. Although the field has witnessed remarkable advances over the past few years, it remains statistically and computationally challenging to learn a single (point estimate) DAG from data, let alone provide uncertainty quantification. We address the difficult task of quantifying graph uncertainty by developing a Bayesian variational inference framework based on novel, provably valid distributions that have support directly on the space of sparse DAGs. These distributions, which we use to define our prior and variational posterior, are induced by a projection operation that maps an arbitrary continuous distribution onto the space of sparse weighted acyclic adjacency matrices. While this projection is combinatorial, it can be solved efficiently using recent continuous reformulations of acyclicity constraints. We empirically demonstrate that our method, ProDAG, can outperform state-of-the-art alternatives in both accuracy and uncertainty quantification.

stat.ML↗

arXiv2023

Calibrated Generalized Bayesian Inference

We propose a simple approach that provides accurate uncertainty quantification for Bayesian inference in misspecified or approximate models, and for generalized (Gibbs) posteriors. While existing solutions in this context are based on explicit Gaussian approximations or post-processing procedures, we demonstrate that correct uncertainty quantification can be achieved by substituting the usual posterior with an intuitively appealing alternative that conveys the same information. This solution applies to both likelihood-based and loss-based posteriors, and is formally demonstrated to reliably quantify uncertainty. This new approach is demonstrated through a range of examples, including generalized linear models, and doubly intractable models.

stat.ME↗

arXiv2023

Contextual Directed Acyclic Graphs

Estimating the structure of directed acyclic graphs (DAGs) from observational data remains a significant challenge in machine learning. Most research in this area concentrates on learning a single DAG for the entire population. This paper considers an alternative setting where the graph structure varies across individuals based on available "contextual" features. We tackle this contextual DAG problem via a neural network that maps the contextual features to a DAG, represented as a weighted adjacency matrix. The neural network is equipped with a novel projection layer that ensures the output matrices are sparse and satisfy a recently developed characterization of acyclicity. We devise a scalable computational framework for learning contextual DAGs and provide a convergence guarantee and an analytical gradient for backpropagating through the projection layer. Our experiments suggest that the new approach can recover the true context-specific graph where existing approaches fail.

stat.ML↗

arXiv2023

Global Neural Networks and The Data Scaling Effect in Financial Time Series Forecasting

Neural networks have revolutionized many empirical fields, yet their application to financial time series forecasting remains controversial. In this study, we demonstrate that the conventional practice of estimating models locally in data-scarce environments may underlie the mixed empirical performance observed in prior work. By focusing on volatility forecasting, we employ a dataset comprising over 10,000 global stocks and implement a global estimation strategy that pools information across cross-sections. Our econometric analysis reveals that forecasting accuracy improves markedly as the training dataset becomes larger and more heterogeneous. Notably, even with as little as 12 months of data, globally trained networks deliver robust predictions for individual stocks and portfolios that are not even in the training dataset. Furthermore, our interpretation of the model dynamics shows that these networks not only capture key stylized facts of volatility but also exhibit resilience to outliers and rapid adaptation to market regime changes. These findings underscore the importance of leveraging extensive and diverse datasets in financial forecasting and advocate for a shift from traditional local training approaches to integrated global estimation methods.

econ.EM↗

arXiv2023

Adaptively switching between a particle marginal Metropolis-Hastings and a particle Gibbs kernel in SMC$^2$

Sequential Monte Carlo squared (SMC$^2$; Chopin et al., 2012) methods can be used to sample from the exact posterior distribution of intractable likelihood state space models. These methods are the SMC analogue to particle Markov chain Monte Carlo (MCMC; Andrieu et al., 2010) and rely on particle MCMC kernels to mutate the particles at each iteration. Two options for the particle MCMC kernels are particle marginal Metropolis-Hastings (PMMH) and particle Gibbs (PG). We introduce a method to adaptively select the particle MCMC kernel at each iteration of SMC$^2$, with a particular focus on switching between a PMMH and PG kernel. The resulting method can significantly improve the efficiency of SMC$^2$ compared to using a fixed particle MCMC kernel throughout the algorithm. Code for our methods is available at https://github.com/imkebotha/kernel_switching_smc2.

stat.CO↗

arXiv2023

Particle Mean Field Variational Bayes

The Mean Field Variational Bayes (MFVB) method is one of the most computationally efficient techniques for Bayesian inference. However, its use has been restricted to models with conjugate priors or those that require analytical calculations. This paper proposes a novel particle-based MFVB approach that greatly expands the applicability of the MFVB method. We establish the theoretical basis of the new method by leveraging the connection between Wasserstein gradient flows and Langevin diffusion dynamics, and demonstrate the effectiveness of this approach using Bayesian logistic regression, stochastic volatility, and deep neural networks.

stat.CO↗

arXiv2023

Bayesian Inference for Evidence Accumulation Models with Regressors

Evidence accumulation models (EAMs) are an important class of cognitive models used to analyze both response time and response choice data recorded from decision-making tasks. Developments in estimation procedures have helped EAMs become important both in basic scientific applications and solution-focussed applied work. Hierarchical Bayesian estimation frameworks for the linear ballistic accumulator model (LBA) and the diffusion decision model (DDM) have been widely used, but still suffer from some key limitations, particularly for large sample sizes, for models with many parameters, and when linking decision-relevant covariates to model parameters. We extend upon previous work with methods for estimating the LBA and DDM in hierarchical Bayesian frameworks that include random effects which are correlated between people, and include regression-model links between decision-relevant covariates and model parameters. Our methods work equally well in cases where the covariates are measured once per person (e.g., personality traits or psychological tests) or once per decision (e.g., neural or physiological data). We provide methods for exact Bayesian inference, using particle-based MCMC, and also approximate methods based on variational Bayesian (VB) inference. The VB methods are sufficiently fast and efficient that they can address large-scale estimation problems, such as with very large data sets. We evaluate the performance of these methods in applications to data from three existing experiments. Detailed algorithmic implementations and code are freely available for all methods.

stat.ME↗

arXiv2023

Deep Learning Enhanced Realized GARCH

We propose a new approach to volatility modeling by combining deep learning (LSTM) and realized volatility measures. This LSTM-enhanced realized GARCH framework incorporates and distills modeling advances from financial econometrics, high frequency trading data and deep learning. Bayesian inference via the Sequential Monte Carlo method is employed for statistical inference and forecasting. The new framework can jointly model the returns and realized volatility measures, has an excellent in-sample fit and superior predictive performance compared to several benchmark models, while being able to adapt well to the stylized facts in volatility. The performance of the new framework is tested using a wide range of metrics, from marginal likelihood, volatility forecasting, to tail risk forecasting and option pricing. We report on a comprehensive empirical study using 31 widely traded stock indices over a time period that includes COVID-19 pandemic.

econ.EM↗

arXiv2023

Reliable Bayesian Inference in Misspecified Models

We provide a general solution to a fundamental open problem in Bayesian inference, namely poor uncertainty quantification, from a frequency standpoint, of Bayesian methods in misspecified models. While existing solutions are based on explicit Gaussian approximations of the posterior, or computationally onerous post-processing procedures, we demonstrate that correct uncertainty quantification can be achieved by replacing the usual posterior with an intuitive approximate posterior. Critically, our solution is applicable to likelihood-based, and generalized, posteriors as well as cases where the likelihood is intractable and must be estimated. We formally demonstrate the reliable uncertainty quantification of our proposed approach, and show that valid uncertainty quantification is not an asymptotic result but occurs even in small samples. We illustrate this approach through a range of examples, including linear, and generalized, mixed effects models.

stat.ME↗