arXiv ScienceSearch

arXiv subjects

Christopher Small

Publications and source records attributed to Christopher Small.

At least 19 recordsLinked to original sources

AI and the Future of Digital Public Squares

Two substantial technological advances have reshaped the public square in recent decades: first with the advent of the internet and second with the recent introduction of large language models (LLMs). LLMs offer opportunities for a paradigm shift towards more decentralized, participatory online spaces that can be used to facilitate deliberative dialogues at scale, but also create risks of exacerbating societal schisms. Here, we explore four applications of LLMs to improve digital public squares: collective dialogue systems, bridging systems, community moderation, and proof-of-humanity systems. Building on the input from over 70 civil society experts and technologists, we argue that LLMs both afford promising opportunities to shift the paradigm for conversations at scale and pose distinct risks for digital public squares. We lay out an agenda for future research and investments in AI that will strengthen digital public squares and safeguard against potential misuses of AI.

cs.CY

Robust Cloud Suppression and Anomaly Detection in Time-lapse Thermography

Due to their transient nature, clouds represent anomalies relative to the underlying landscape of interest. Hence, the challenge of cloud identification can be considered a specific case in the more general problem of anomaly detection. The confounding effects of transient anomalies are particularly troublesome for spatiotemporal analysis of land surface processes. Spatiotemporal characterization provides a statistical basis to quantify the most significant temporal patterns and their spatial distributions without the need for a priori assumptions about the form, amplitude or timing of the observed changes. The objective of this study is to implement and evaluate a robust approach to distinguish clouds and other transient anomalies from diurnal and annual thermal cycles observed with time-lapse thermography. The approach uses Robust Principal Component Analysis (RPCA) to statistically distinguish low rank and sparse components of the land surface temperature image time series, followed by a spatiotemporal characterization of the L component time series to quantify the dominant diurnal and annual thermal cycles in the study area. The RPCA effectively segregated clouds, sensor anomalies, swath gaps, geospatial displacements and transient thermal anomalies into the sparse component time series. Spatiotemporal characterization of the low rank component time series clearly resolves a variety of diurnal and annual thermal cycles for different land covers and water bodies.

physics.geo-ph

Multiscale Topology of the Spectroscopic Mixing Space: Impervious Substrates

Characterization of topology and dimensionality of spectral feature spaces provides insight into information content. The objective of this study is to characterize topology and spectral dimensionality of spectral mixing spaces representing a diversity of built environments in urban areas worldwide using both spaceborne and airborne imaging spectroscopy. Comparing complementary types of dimensionality reduction to render high dimensional spectral mixing spaces allows for characterization of both spectral dimensionality and mixing space topology. Using a diverse collection of 30 decameter-resolution urban core subscenes imaged by the EMIT spaceborne imaging spectrometer and 5 sub-decameter-resolution urban gradient flight lines imaged by the AVIRIS-NG airborne imaging spectrometer, we conduct such characterizations. Global scale topology of low order principal component-derived mixing spaces resembles the Substrate, Vegetation, Dark (SVD) triangular topology of globally diverse composite mixing spaces found in numerous earlier studies - but these spaces are also characterized by multiple (2-3) distinct substrate endmembers and a wide variety of less common spectra corresponding to synthetic anthropogenic materials. Local scale topology from 2D and 3D UMAP manifold embeddings reveals the presence of multiple spectrally distinct impervious substrate mixing continua corresponding to geographically distinct built environments.

physics.geo-ph

Topological Generality and Spectral Dimensionality in the Earth Mineral Dust Source Investigation (EMIT) using Joint Characterization and the Spectral Mixture Residual

NASA's Earth Surface Mineral Dust Source Investigation (EMIT) mission seeks to use spaceborne imaging spectroscopy (hyperspectral imaging) to map the mineralogy of arid dust source regions. Here we apply recent developments in Joint Characterization (JC) and the spectral Mixture Residual (MR) to explore the information content of data from this novel mission. Specifically, for a mosaic of 20 spectrally diverse scenes we find: 1) a generalized three-endmember (Substrate, Vegetation, Dark; SVD) spectral mixture model is capable of capturing the preponderance (99% in 3 dimensions) of spectral variance with low misfit (99% of pixels with RMSE < 3.7%); 2) manifold learning (UMAP) is capable of identifying spatially coherent, physically interpretable clustering relationships in the spectral feature space; 3) UMAP yields results that are at least as informative when applied to the MR as when applied to raw reflectance; 4) SVD fraction information usefully contextualizes UMAP clustering relationships, and vice-versa (JC); and 5) when EMIT data are convolved to spectral response functions of multispectral instruments (Sentinel-2, Landsat 8/9, Planet SuperDove), SVD fractions correlate strongly across sensors but UMAP clustering relationships for the EMIT hyperspectral feature space are far more informative than for simulated multispectral sensors. Implications are discussed for both the utility of EMIT data in the near-term, and for the potential of high SNR spaceborne imaging spectroscopy more generally, to transform the future of optical remote sensing in the years and decades to come.

physics.geo-ph

"Coherent Mode" for the World's Public Square

Systems for large scale deliberation have resolved polarized issues and shifted agenda setting into the public's hands. These systems integrate bridging-based ranking algorithms - including group informed consensus implemented in Polis and the continuous matrix factorization approach implemented by Twitter Birdwatch - making it possible to highlight statements which enjoy broad support from a diversity of opinion groups. Polis has been productively employed to foster more constructive political deliberation at nation scale in law making exercises. Twitter Birdwatch is implemented with the intention of addressing misinformation in the global public square. From one perspective, Twitter Birdwatch can be viewed as an anti-misinformation system which has deliberative aspects. But it can also be viewed as a first step towards a generalized deliberative system, using Twitter's misinformation problem as a proving ground. In this paper, we propose that Twitter could adapt Birdwatch to produce maps of public opinion. We describe a system in five parts for generalizing Birdwatch: activation of a deliberative system and topic selection, population sampling and the role of expert networks, deliberation, reporting interpretable results and finally distribution of the results to the public and those in power.

cs.SI

Linking Common Multispectral Vegetation Indices to Hyperspectral Mixture Models: Results from 5 nm, 3 m Airborne Imaging Spectroscopy in a Diverse Agricultural Landscape

For decades, agronomists have used remote sensing to monitor key crop parameters like biomass, fractional cover, and plant health. Vegetation indices (VIs) are popular for this purpose, primarily leveraging the spectral red edge in multispectral imagery. In contrast, spectral mixture models use the full reflectance spectrum to simultaneously estimate area fractions of multiple endmember materials present within a mixed pixel. Here, we characterize the relationships between hyperspectral endmember fractions and 6 common multispectral VIs in crops & soils of California agriculture. Fractional area of green vegetation (Fv) was estimated directly from 64,000,000 5 nm, 3 to 5 m reflectance spectra compiled from a mosaic of 15 AVIRIS-ng flightlines. Simulated Planet SuperDove reflectance spectra were then derived from the AVIRIS-ng, and used to compute 6 popular VIs (NDVI, NIRv, EVI, EVI2, SR, DVI). Multispectral VIs were compared to hyperspectral Fv using parametric (Pearson correlation, r) and nonparametric (Mutual Information, MI) similarity metrics. 4 VIs (NIRv, DVI, EVI, EVI2) showed strong linear relationships to Fv (r > 0.94; MI > 1.2). NIRv & DVI showed strong interrelation (r > 0.99, MI > 2.4), but deviated significantly from 1:1 relative to Fv. EVI & EVI2 were also strongly interrelated (r > 0.99, MI > 2.3) and more closely followed a 1:1 relation with Fv. In contrast, NDVI & SR showed weaker, nonlinear, heteroskedastic relation to Fv (r < 0.84, MI = 0.69). NDVI showed especially severe sensitivity to substrate background reflectance (-0.05 < NDVI < +0.6 for unvegetated spectra) and saturation (0.2 < Fv < 0.8 for NDVI = 0.7). These direct observational constraints on multispectral VI and hyperspectral mixture model comparability can serve as a quantitative benchmark for agronomic applications in the coming era of increasing spatial & spectral resolution Earth observation.

physics.geo-ph

Spectrometry of the Urban Lightscape

NASA's Gateway to Astronaut Photography of Earth Contains over 30000 photos of 2500 cataloged urban lightscapes (urban night lights) taken from the International Space Station. Over 100 of these multispectral DSLR photos are of sufficient spatial resolution, sharpness and exposure to be used for broadband spectral characterization of urban lightscapes. Analysis of simulated atmospheric transmissivity from the MODTRAN radiative transfer model shows that spectral slopes of transmissivity spectra are relatively insensitive to choice of model atmopshere, with variations in atmospheric path length and aerosol optical depth primarily affecting the bias of the spectrum rather than the slope. This suggests that color tempterature-calibrated RGB channels can be corrected for relative differences in atmospheric scattering and absorption to allow for quantitative intercomparison. A mosaic of 18 intercalibrated RGB photos renders a spectral feature space with four clearly defined spectral endmembers corresponding to white, yellow and red light sources, with brightness modulated by a dark background endmember. These four spectral endmembers form the basis of a linear spectral mixture model which can be inverted to provide estimates of the areal fraction of each endmember present within every pixel instantaneous field of view. The resulting spectral feature space shows two distinct mixing trends extending from the dark endmember to near flat spectrum (white-yellow) and warm spectrum (orange) sources. The distribution of illuminated pixels is strongly skewed toward a lower luminance background of warm spectrum street lighting, with brighter lights generally corresponding to point sources and major thoroughfares. Intercomparison of 18 individual urban lightscape spectral feature spaces show consistent topology, despite variations in exposure and interior mixing trends.

physics.soc-ph

A multiscale spatiotemporal approach for smallholder irrigation detection

In presenting an irrigation detection methodology that leverages multiscale satellite imagery of vegetation abundance, this paper introduces a process to supplement limited ground-collected labels and ensure classifier applicability in an area of interest. Spatiotemporal analysis of MODIS 250m Enhanced Vegetation Index (EVI) timeseries characterizes native vegetation phenologies at regional scale to provide the basis for a continuous phenology map that guides supplementary label collection over irrigated and non-irrigated agriculture. Subsequently, validated dry season greening and senescence cycles observed in 10m Sentinel-2 imagery are used to train a suite of classifiers for automated detection of potential smallholder irrigation. Strategies to improve model robustness are demonstrated, including a method of data augmentation that randomly shifts training samples; and an assessment of classifier types that produce the best performance in withheld target regions. The methodology is applied to detect smallholder irrigation in two states in the Ethiopian highlands, Tigray and Amhara. Results show that a transformer-based neural network architecture allows for the most robust prediction performance in withheld regions, followed closely by a CatBoost random forest model. Over withheld ground-collection survey labels, the transformer-based model achieves 96.7% accuracy over non-irrigated samples and 95.9% accuracy over irrigated samples. Over a larger set of samples independently collected via the introduced method of label supplementation, non-irrigated and irrigated labels are predicted with 98.3% and 95.5% accuracy, respectively. The detection model is then deployed over Tigray and Amhara, revealing crop rotation patterns and year-over-year irrigated area change. Predictions suggest that irrigated area in these two states has decreased by approximately 40% from 2020 to 2021.

cs.CV

Joint Characterization of the Cryospheric Spectral Feature Space

Hyperspectral feature spaces are useful for many remote sensing applications ranging from spectral mixture modeling to discrete thematic classification. In such cases, characterization of the feature space dimensionality, geometry and topology can provide guidance for effective model design. The objective of this study is to compare and contrast two approaches for identifying feature space basis vectors via dimensionality reduction. These approaches can be combined to render a joint characterization that reveals spectral properties not apparent using either approach alone. We use a diverse collection of AVIRIS-NG reflectance spectra of the snow-firn-ice continuum to illustrate the utility of joint characterization and identify physical properties inferred from the spectra. Spectral feature spaces combining principal components (PCs) and t-distributed Stochastic Neighbor Embeddings (t-SNEs) provide physically interpretable dimensions representing the global (PC) structure of cryospheric reflectance properties and local (t-SNE) manifold structures revealing clustering not resolved in the global continuum. Joint characterization reveals distinct continua for snow-firn gradients on different parts of the Greenland Ice Sheet and multiple clusters of ice reflectance properties common to both glacier and sea ice in different locations. Clustering revealed in t-SNE feature spaces, and extended to the joint characterization, distinguishes differences in spectral curvature specific to location within the snow accumulation zone, and BRDF effects related to view geometry. The ability of PC+t-SNE joint characterization to produce a physically interpretable spectral feature spaces revealing global topology while preserving local manifold structures suggests that this characterization might be extended to the much higher dimensional hyperspectral feature space of all terrestrial land cover.

physics.geo-ph

Spatiotemporal Characterization of VIIRS Night Light

The VIIRS Day Night Band sensor on the Suomi NPP satellite provides almost a decade of observations of night light. The daily frequency of sampling, without the temporal averaging of annual composites, requires the distinction between apparent changes of imaged night light related to the imaging process and actual changes in the underlying sources of the light being imaged. This study characterizes night light variability over a range of spatial and temporal scales to provide a context for interpretation of changes on both subannual and interannual time scales. This analysis uses a combination of temporal moments, spatial correlation and Empirical Orthogonal Function (EOF) analysis. A key result is the pervasive heteroskedasticity of VIIRS monthly mean night light. Specifically, the monotonic decrease of temporal variability with increasing mean brightness. Anthropogenic night light is remarkably stable on subannual time scales. Overall variance partition derived from the eigenvalues of the spatiotemporal covariance matrix are 88%, 2% and 2% for spatial, seasonal and interannual variance in the most diverse geographic region on Earth (Eurasia). Heteroskedasticity is present in all areas for all months, suggesting that much, if not most, of observed month-to-month variability may result from luminance of otherwise stable sources subjected to multiple aspects of the imaging process varying in time. Given the skewed distribution of all night light arising from radial peripheral dimming of bright sources, even aggregate metrics using thresholds must be interpreted in light of the fact that much larger numbers of more variable low luminance pixels may statistically overwhelm smaller numbers of stable higher luminance pixels and cause apparent changes related to the imaging process to be interpreted as actual changes in the light sources.

physics.geo-ph

Joint Characterization of Spatiotemporal Data Manifolds

Spatiotemporal (ST) image data are increasingly common and often high-dimensional (high-D). Modeling ST data can be a challenge due to the plethora of independent and interacting processes which may or may not contribute to the measurements. Characterization can be considered the complement to modeling by helping guide assumptions about generative processes and their representation in the data. Dimensionality reduction (DR) is a frequently implemented type of characterization designed to mitigate the "curse of dimensionality" on high-D signals. For decades, Principal Component (PC) and Empirical Orthogonal Function (EOF) analysis has been used as a linear, invertible approach to DR and ST analysis. Recent years have seen the additional development of a suite of nonlinear DR algorithms, frequently categorized as "manifold learning". Here, we explore the idea of joint characterization of ST data manifolds using PCs/EOFs alongside two nonlinear DR approaches: Laplacian Eigenmaps (LE) and t-distributed stochastic neighbor embedding (t-SNE). Starting with a synthetic example and progressing to global, regional, and field scale ST datasets spanning roughly 5 orders of magnitude in space and 2 in time, we show these three DR approaches can yield complementary information about ST manifold topology. Compared to the relatively diffuse TFS produced by PCs/EOFs, the nonlinear approaches yield more compact manifolds with decreased ambiguity in temporal endmembers (LE) and/or in spatiotemporal clustering (t-SNE). These properties are compensated by the greater interpretability, significantly lower computational demand and diminished sensitivity to spatial aliasing for PCs/EOFs than LE or t-SNE. Taken together, we find joint characterization using the three complementary DR approaches capable of greater insight into generative ST processes than possible using any single approach alone.

cs.LG

Joint Characterization of Multiscale Information in High Dimensional Data

High dimensional data can contain multiple scales of variance. Analysis tools that preferentially operate at one scale can be ineffective at capturing all the information present in this cross-scale complexity. We propose a multiscale joint characterization approach designed to exploit synergies between global and local approaches to dimensionality reduction. We illustrate this approach using Principal Components Analysis (PCA) to characterize global variance structure and t-stochastic neighbor embedding (t-sne) to characterize local variance structure. Using both synthetic images and real-world imaging spectroscopy data, we show that joint characterization is capable of detecting and isolating signals which are not evident from either PCA or t-sne alone. Broadly, t-sne is effective at rendering a randomly oriented low-dimensional map of local clusters, and PCA renders this map interpretable by providing global, physically meaningful structure. This approach is illustrated using imaging spectroscopy data, and may prove particularly useful for other geospatial data given robust local variance structure due to spatial autocorrelation and physical interpretability of global variance structure due to spectral properties of Earth surface materials. However, the fundamental premise could easily be extended to other high dimensional datasets, including image time series and non-image data.

stat.ML

Spatial network connectivity of population and development in the USA; Implications for disease transmission

Zipfs Law states that rank-size distributions of city populations follow a power law with an exponent of -1. The assertion of a universal power law is controversial because the linearity and slope appear to vary over time and among countries. We compare census enumerations and night light luminance as proxies for population density and intensity of development in the contiguous United States. Treating population density and development intensity as continuous quantities allows for the definition of spatial networks based on the level of spatial connectivity. The resulting distributions of spatial network components (subsets of connected nodes) vary with degree of connectivity, but maintain consistent scaling over a wide range of network sizes. At continental scales, spatial network rank-size distributions obtained from both population density and night light brightness are well-fit by power laws with exponents near -1 for a wide range of density and luminance thresholds. However, the largest components (10,000 - 100,000 sq.km) are far larger than individual cities and represent spatially contiguous agglomerations of urban, suburban and periurban development. Projecting county-level numbers of confirmed cases of SARS-CoV-2 for the US onto spatial networks of population and development allows the spatiotemporal evolution of the epidemic to be quantified as propagation within networks of varying connectivity.. The results show an abrupt transition from slow increases in confirmed cases in a small number of network components to rapid geographic dispersion to a larger number of components before mobility reductions occurred in March 2020.

physics.soc-ph

Coupled Spatiotemporal Characterization of Monsoon Cloud Cover and Vegetation Phenology

Many biogeophysical processes continuously evolve in both space and time. Some of these processes change gradually enough across both space and time to be oversampled by the satellite image archive. In these cases, image time series of Earth observations can support robust spatiotemporal analysis. In this analysis, we leverage the spectral differences of green vegetation, soil, and clouds to perform an EOF analysis which simultaneously characterizes monsoon cloud cover and vegetation phenology in a temperate cloud forest in Dhofar, southern Oman. The Normalized Difference Vegetation Index of daily MODIS reflectance clearly distinguishes clouds and bare soil from both grassland and cloud forest vegetation. Using the temporal feature space of the low order principal components of the NDVI time series, we identify distinct annual cycles of cloud cover and vegetation phenology corresponding to geographically contiguous areas of coupled monsoon-phenological activity (phenoregions). The EOF analysis characterizes the temporal dynamics of these phenoregions. The wadi cloud forests of the Jabal Al Qara and Jabal Al Qamar can each be described with 2 distinct spatiotemporal patterns of monsoon phenology, corresponding to the eastern and western portions of each range. In addition, the rangetop grasslands of the Jabal Al Qara show 3 distinct spatiotemporal patterns, corresponding to eastern, central, and western portions of the range. While each phenoregion has a clearly repeating annual cycle, each also demonstrates substantial interannual variability in both cloud cover and phenology. This interannual variability is of a sufficient magnitude to mask any trends in cloud cover or phenology in any of the regions in the Dhofar Mountains. Finally, using the degree of asymmetry of the cumulative NDVI time series, we infer a causal link between duration of cloud cover and amplitude of phenology.

physics.geo-ph

Global cross-calibration of Landsat spectral mixture models

Data continuity for the Landsat program relies on accurate cross-calibration among sensors. The Landsat 8 OLI has been shown to exhibit superior performance to the sensors on Landsats 4-7 with respect to radiometric calibration, signal to noise, and geolocation. However, improvements to the positioning of the spectral response functions on the OLI have resulted in known biases for commonly used spectral indices because the new band responses integrate absorption features differently from previous Landsat sensors. The objective of this analysis is to quantify the impact of these changes on linear spectral mixture models that use imagery collected by different Landsat sensors. The 2013 underflight of Landsat 7 and 8 provides an opportunity to cross calibrate the spectral mixing spaces of the ETM+ and OLI sensors using near-simultaneous acquisitions from a wide variety of land cover types worldwide. We use 80,910,343 pairs of OLI and ETM+ spectra to characterize the OLI spectral mixing space and perform a cross-calibration with ETM+. This new global collection of Landsat spectra spans a greater spectral diversity than those used in prior studies and the resulting Substrate, Vegetation, and Dark (SVD) spectral endmembers (EMs) supplant prior global Landsat EMs. We find only minor (-0.01 < u < 0.01) differences between SVD fractions unmixed using sensor-specific EMs. RMS misfit fractions are also small (<98% of pixels with <5% RMSE), in accord with past studies. Finally, vegetation is used as an example to illustrate the empirical and theoretical relationship between commonly used spectral indices and subpixel fractions. SVD fractions unmixed using global EMs thus provide easily computable, linearly scalable, physically based measures of subpixel land cover which can be compared accurately across the entire Landsat 4-8 archive without introducing any additional cross-sensor corrections.

physics.geo-ph

Spatial Structure and Scaling of Agricultural Networks

Considering agricultural landscapes as networks can provide information about spatial connectivity relevant for a wide range of applications including pollination, pest management, and ecology. Global agricultural networks are well-described by power law rank-size distributions. However, regional analyses capture only a subset of the total global network. Most analyses are regional. In this paper, we seek to address the following questions: Does the globally observed scale-free property of agricultural networks hold over smaller spatial domains? Can similar properties be observed at kilometer to meter scales? We analyze 9 intensively cultivated Landsat scenes on 5 continents with a wide range of vegetation distributions. We find that networks of vegetation fraction within the domain of each of these Landsat scenes exhibit substantial variability - but still possess similar scaling properties to the global distribution of agriculture. We also find similar results using a 39 km2 IKONOS image. To illustrate an application of spatial network analysis, we show an example of network disruption. We compare two networks with similar rank-size distributions that behave differently when nodes are progressively removed. We suggest that treating agricultural land cover as spatial networks can provide a straightforward way of characterizing the connectivity of complex spatial distributions of agriculture across a wide range of landscapes and at spatial scales relevant for practical agricultural applications.

physics.soc-ph

Observations of cyclone-induced storm surge in coastal Bangladesh

Water level measurements from 15 tide gauges in the coastal zone of Bangladesh are analyzed in conjunction with cyclone tracks and wind speed data for 54 cyclones between 1977 and 2010. Storm surge magnitude is inferred from residual water levels computed by subtracting modeled astronomical tides from observed water levels at each station. Observed residual water levels are generally smaller than reported storm surge levels for cyclones where both are available, and many cyclones produce no obvious residual at all. Both maximum and minimum residual water levels are higher for west-landing cyclones producing onshore winds and generally diminish for cyclones making landfall on the Bangladesh coast or eastward producing offshore winds. Water levels observed during cyclones are generally more strongly influenced by tidal phase and amplitude than by storm surge alone. In only 7 of the 15 stations does the highest plausible observed water level coincide with a cyclone. While cyclone-coincident residual water level maxima occur at a wide range of tidal phases, very few coincide with high spring tides. Comparisons of cyclone-related casualties with maximum wind speed, hour of landfall, population density and residual water level (inferred storm surge) show no significant correlations for any single characteristic. Cyclones with high casualties are often extreme in one or more of these characteristics but there appears to be no single extreme characteristic shared by all high casualty cyclones.

physics.geo-ph

Spatial Scaling of Land Cover Networks

Spatial networks of land cover are well-described by power law rank-size distributions. Continuous field proxies for human settlements, agriculture and forest cover have similar spatial scaling properties spanning 4 to 5 orders of magnitude. Progressive segmentation of these continuous fields yields spatial networks with rank-size distributions having slopes near -1 for a wide range of thresholds. We consider a general explanation for this scaling that does not require different processes for each type of land cover. The same conditions that give rise to scale-free networks in general can produce power law distributions of component sizes for bounded spatial networks confined to a plane or surface. Progressive segmentation of a continuous field naturally results in growth of the network while the increasing perimeters of the growing components result in preferential attachment to the larger components with the longer perimeters. Progressive segmentation of two types of random continuous field results in progressions of growing spatial networks with components that achieve unity power law rank-size distributions immediately before explosive percolation occurs. This suggests that the scaling properties and uniform area distribution implied by a unity power law rank-size distribution may be a general characteristic of bounded spatial networks produced by segmenting some continuous fields. The implication is that different process-specific mechanisms may not be required to explain the rank-size distributions observed for the diversity of land cover products described here.

physics.soc-ph