Which Text Matters: Handling Model Uncertainty in Text-as-Data Regressions, with an Application to Central Bank Communication

Abstract

Social scientists routinely convert text into covariates like topic shares, ideal points or sentiment scores, and insert them into statistical models of interest. This two-step design is convenient, but it is rarely guided by theory: little tells the analyst which text-based covariates to construct or include. Each approach embodies contestable measurement choices, and competing operationalizations of the same construct can support different conclusions, so a single research question gives rise to a menu of competing regression models. Which text constructed covariates belong in a model, and in what functional form, is thus a problem of model uncertainty. We propose spike-and-slab generalized additive models (SSGAMs), which jointly select covariates and determine whether their effects are linear or nonlinear. We illustrate the approach with the communication of a powerful unelected institution, asking which of 45 competing text indicators from 224 monthly European Central Bank press conferences, spanning late 2002 to mid-2025, track professional inflation expectations. A sparse set is selected; widely used alternatives, and every tested nonlinear effect, carry no incremental information.