arXiv subjects

David R. Thompson

Publications and source records attributed to David R. Thompson.

17 recordsLinked to original sources

arXiv2026

On-board ML for Trace Gas detection in Imaging Spectroscopy data

Data collected during aerial and spaceborne imaging spectroscopy campaigns enables the detection of transient events such as trace gas emissions. However, current processing pipelines depend on slow, on-the-ground processing, which delays the time to information of each detected event and prohibits immediate follow-up actions. During the Tokyo Field Campaign of March 2026, we explored on-board processing of Imaging Spectroscopy data from the equipped AVIRIS-5 sensor. Due to communication bottlenecks, full datacubes cannot be downlinked immediately during the flight. Instead we downlink the potential events predicted by our efficient and small machine learning model. We show the first on-board detection of methane point source emission with Imaging Spectroscopy data using Edge ML.

cs.LG↗

arXiv2026

Fully Automatic Trace Gas Plume Detection

Future imaging spectrometers will expand contemporary data volumes by orders of magnitude, requiring automated methods to upscale labor-intensive detection of trace gas point sources. Here we present a fully-automated approach that achieves operational performance for plume detection and labelling without human participation. Our method combines machine learning (ML)-based morphological analysis with physics-based spectroscopic model fitting. We deploy it on data from the EMIT imaging spectrometer, operating in two modes. First, we present a "daily digest" that runs automatically on all downlinked data, flagging the largest events for immediate response. The daily digest demonstrates that a significant fraction of the largest plumes can be detected automatically with negligible false positives. This represents a significant new high-water mark in plume detection accuracy. Second, we use it for retrospective analysis to find plumes that were missed by the existing human review process. We observe that at least 25% of large plumes may have been passed over in the existing workflow due to confirmation bias and ambiguity in the visual cues used by human reviewers. Finally, we extend detection to three understudied trace gases: NH3, NO2 and the first observations of carbon monoxide (CO) plume in EMIT imagery.

cs.LG↗

arXiv2025

Operational machine learning for remote spectroscopic detection of CH$_{4}$ point sources

Mitigating anthropogenic methane sources is one of the most cost-effective levers to slow down global warming. While satellite-based imaging spectrometers, such as EMIT, PRISMA, and EnMAP, can detect these point sources, current methane retrieval methods based on matched filters produce a high number of false detections requiring manual verification. To address this challenge, we deployed a ML system for detecting methane emissions within the Methane Alert and Response System (MARS) of UNEP's IMEO. This represents the first operational deployment of automated methane point-source detection using spaceborne imaging spectrometers, providing regular global coverage and scalability to future constellations with even higher data volumes. This task required several technical advances. First, we created one of the largest and most diverse and global ML ready datasets to date of annotated methane plumes from three imaging spectrometer missions, and quantitatively compared different deep learning model configurations. Second, we extended prior evaluation methodologies from small, tiled datasets to full granules that are more representative of operational use. This revealed that deep learning models still produce a large number of false detections, a problem we addressed with model ensembling, which reduced false detections by over 74%. During 11 months of operational deployment, our system processed more than 25,000 hyperspectral products faciliting the verification of 2,851 distinct methane leaks, which resulted in 834 stakeholder notifications. We further demonstrate the model's utility in verifying mitigation success through case studies in Libya, Argentina, Oman, and Azerbaijan. Our work represents a critical step towards a global AI-assisted methane leak detection system, which is required to process the dramatically higher data volumes expected from current and future imaging spectrometers.

cs.AI↗

arXiv2025

The Optical Design of the Carbon Investigation(Carbon-I) Imaging Spectrometer

The proposed Carbon Investigation (Carbon-I) Imaging Spectrometer is designed to measure variations of greenhouse gases in Earth's atmosphere. The instrument will survey the Earth from its own spacecraft at an altitude of approximately 610 km. It will use a coarse ground sampling distance (GSD) of <400 m in global mode for land and coastal monitoring and finer 35 m GSD in target mode to sample key regions. The identification and quantification of greenhouse gases require continuous spectral sampling over the 2040-2380 nm wavelength range with <1 nm spectral sampling. The proposed design builds upon Jet Propulsion Laboratory's (JPL) experience of spaceflight Dyson imaging spectrometers to achieve spectral sampling of 0.7 nm per pixel. This paper presents the proposed Carbon-I optical design comprised of a freeform three-mirror anastigmat telescope that couples to a F/2.2, highly uniform Dyson-inspired imaging spectrometer. The high uniformity and throughput enables Carbon-I to measure Earth's greenhouse gas concentrations with unprecedented precision and spatial sampling.

physics.optics↗

arXiv2025

SpecTf: Transformers Enable Data-Driven Imaging Spectroscopy Cloud Detection

Current and upcoming generations of visible-shortwave infrared (VSWIR) imaging spectrometers promise unprecedented capacity to quantify Earth System processes across the globe. However, reliable cloud screening remains a fundamental challenge for these instruments, where traditional spatial and temporal approaches are limited by cloud variability and limited temporal coverage. The Spectroscopic Transformer (SpecTf) addresses these challenges with a spectroscopy-specific deep learning architecture that performs cloud detection using only spectral information (no spatial or temporal data are required). By treating spectral measurements as sequences rather than image channels, SpecTf learns fundamental physical relationships without relying on spatial context. Our experiments demonstrate that SpecTf significantly outperforms the current baseline approach implemented for the EMIT instrument, and performs comparably with other machine learning methods with orders of magnitude fewer learned parameters. Critically, we demonstrate SpecTf's inherent interpretability through its attention mechanism, revealing physically meaningful spectral features the model has learned. Finally, we present SpecTf's potential for cross-instrument generalization by applying it to a different instrument on a different platform without modifications, opening the door to instrument agnostic data driven algorithms for future imaging spectroscopy tasks.

cs.LG↗

arXiv2024

Adaptive sampling with PIXL on the Mars Perseverance rover

Planetary rovers can use onboard data analysis to adapt their measurement plan on the fly, improving the science value of data collected between commands from Earth. This paper describes the implementation of an adaptive sampling algorithm used by PIXL, the X-ray fluorescence spectrometer of the Mars 2020 Perseverance rover. PIXL is deployed using the rover arm to measure X-ray spectra of rocks with a scan density of several thousand points over an area of typically 5 x 7 mm. The adaptive sampling algorithm is programmed to recognize points of interest and to increase the signal-to-noise ratio at those locations by performing longer integrations. Two approaches are used to formulate the sampling rules based on past quantification data: 1) Expressions that isolate particular regions within a ternary compositional diagram, and 2) Machine learning rules that threshold for a high weight percent of particular compounds. The design of the rulesets are outlined and the performance of the algorithm is quantified using measurements from the surface of Mars. To our knowledge, PIXL's adaptive sampling represents the first autonomous decision-making based on real-time compositional analysis by a spacecraft on the surface of another planet.

astro-ph.EP↗

arXiv2023

Evaluating the accuracy of Gaussian approximations in VSWIR imaging spectroscopy retrievals

The joint retrieval of surface reflectances and atmospheric parameters in VSWIR imaging spectroscopy is a computationally challenging high-dimensional problem. Using NASA's Surface Biology and Geology mission as the motivational context, the uncertainty associated with the retrievals is crucial for further application of the retrieved results for environmental applications. Although Markov chain Monte Carlo (MCMC) is a Bayesian method ideal for uncertainty quantification, the full-dimensional implementation of MCMC for the retrieval is computationally intractable. In this work, we developed a block Metropolis MCMC algorithm for the high-dimensional VSWIR surface reflectance retrieval that leverages the structure of the forward radiative transfer model to enable tractable fully Bayesian computation. We use the posterior distribution from this MCMC algorithm to assess the limitations of optimal estimation, the state-of-the-art Bayesian algorithm in operational retrievals which is more computationally efficient but uses a Gaussian approximation to characterize the posterior. Analyzing the differences in the posterior computed by each method, the MCMC algorithm was shown to give more physically sensible results and reveals the non-Gaussian structure of the posterior, specifically in the atmospheric aerosol optical depth parameter and the low-wavelength surface reflectances.

stat.AP↗

arXiv2021

Impact of Scene-Specific Enhancement Spectra on Matched Filter Greenhouse Gas Retrievals from Imaging Spectroscopy

Matched filter (MF) techniques have been widely used for retrieval of greenhouse gas enhancements (enh.) from imaging spectroscopy datasets. While multiple algorithmic techniques and refinements have been proposed, the greenhouse gas target spectrum used for concentration enh. estimation has remained largely unaltered since the introduction of quantitative MF retrievals. The magnitude of retrieved methane and carbon dioxide enh., and thereby integrated mass enh. (IME) and estimated flux of point-source emitters, is heavily dependent on this target spectrum. Current standard use of molecular absorption coefficients to create unit enh. target spectra does not account for absorption by background concentrations of greenhouse gases, solar and sensor geometry, or atmospheric water vapor absorption. We introduce geometric and atmospheric parameters into the generation of scene-specific (SS) unit enh. spectra to provide target spectra that are compatible with all greenhouse gas retrieval MF techniques. For methane plumes, IME resulting from use of standard, generic enh. spectra varied from -22 to +28.7% compared to SS enh. spectra. Due to differences in spectral shape between the generic and SS enh. spectra, differences in methane plume IME were linked to surface spectral characteristics in addition to geometric and atmospheric parameters. IME differences for carbon dioxide plumes, with generic enh. spectra producing integrated mass enh. -76.1 to -48.1% compared to SS enh. spectra. Fluxes calculated from these integrated enh. would vary by the same %s, assuming equivalent wind conditions. Methane and carbon dioxide IME were most sensitive to changes in solar zenith angle and ground elevation. SS target spectra can improve confidence in greenhouse gas retrievals and flux estimates across collections of scenes with diverse geometric and atmospheric conditions.

physics.ao-ph↗

arXiv2020

Fast and Accurate Retrieval of Methane Concentration from Imaging Spectrometer Data Using Sparsity Prior

The strong radiative forcing by atmospheric methane has stimulated interest in identifying natural and anthropogenic sources of this potent greenhouse gas. Point sources are important targets for quantification, and anthropogenic targets have potential for emissions reduction. Methane point source plume detection and concentration retrieval have been previously demonstrated using data from the Airborne Visible InfraRed Imaging Spectrometer Next Generation (AVIRIS-NG). Current quantitative methods have tradeoffs between computational requirements and retrieval accuracy, creating obstacles for processing real-time data or large datasets from flight campaigns. We present a new computationally efficient algorithm that applies sparsity and an albedo correction to matched filter retrieval of trace gas concentration-pathlength. The new algorithm was tested using AVIRIS-NG data acquired over several point source plumes in Ahmedabad, India. The algorithm was validated using simulated AVIRIS-NG data including synthetic plumes of known methane concentration. Sparsity and albedo correction together reduced the root mean squared error of retrieved methane concentration-pathlength enhancement by 60.7% compared with a previous robust matched filter method. Background noise was reduced by a factor of 2.64. The new algorithm was able to process the entire 300 flightline 2016 AVIRIS-NG India campaign in just over 8 hours on a desktop computer with GPU acceleration.

eess.IV↗

arXiv2019

Learning Radiative Transfer Models for Climate Change Applications in Imaging Spectroscopy

According to a recent investigation, an estimated 33-50% of the world's coral reefs have undergone degradation, believed to be as a result of climate change. A strong driver of climate change and the subsequent environmental impact are greenhouse gases such as methane. However, the exact relation climate change has to the environmental condition cannot be easily established. Remote sensing methods are increasingly being used to quantify and draw connections between rapidly changing climatic conditions and environmental impact. A crucial part of this analysis is processing spectroscopy data using radiative transfer models (RTMs) which is a computationally expensive process and limits their use with high volume imaging spectrometers. This work presents an algorithm that can efficiently emulate RTMs using neural networks leading to a multifold speedup in processing time, and yielding multiple downstream benefits.

cs.LG↗

arXiv2016

A Machine Learning Classifier for Fast Radio Burst Detection at the VLBA

Time domain radio astronomy observing campaigns frequently generate large volumes of data. Our goal is to develop automated methods that can identify events of interest buried within the larger data stream. The V-FASTR fast transient system was designed to detect rare fast radio bursts (FRBs) within data collected by the Very Long Baseline Array. The resulting event candidates constitute a significant burden in terms of subsequent human reviewing time. We have trained and deployed a machine learning classifier that marks each candidate detection as a pulse from a known pulsar, an artifact due to radio frequency interference, or a potential new discovery. The classifier maintains high reliability by restricting its predictions to those with at least 90% confidence. We have also implemented several efficiency and usability improvements to the V-FASTR web-based candidate review system. Overall, we found that time spent reviewing decreased and the fraction of interesting candidates increased. The classifier now classifies (and therefore filters) 80-90% of the candidates, with an accuracy greater than 98%, leaving only the 10-20% most promising candidates to be reviewed by humans.

astro-ph.IM↗

arXiv2016

Limits on Fast Radio Bursts from Four Years of the V-FASTR Experiment

The V-FASTR experiment on the Very Long Baseline Array was designed to detect dispersed pulses of milliseconds duration, such as fast radio bursts (FRBs). We use all V-FASTR data through February 2015 to report V-FASTR's upper limits on the rates of FRBs, and compare these with re-derived rates from Parkes FRB detection experiments. V-FASTR's operation at lambda=20 cm allows direct comparison with the 20 cm Parkes rate, and we derive a power-law limit of \gamma<-0.4 (95% confidence limit) on the index of FRB source counts, N(>S)\propto S^\gamma. Using the previously measured FRB rate and the unprecedented amount of survey time spent searching for FRBs at a large range of wavelengths (0.3 cm > \lambda > 90 cm), we also place frequency-dependent limits on the spectral distribution of FRBs. The most constraining frequencies place two-point spectral index limits of \alpha_{20cm}^{4cm} < 5.8 and \alpha_{90cm}^{20cm} > -7.6, where fluence F \propto f^\alpha if we assume true the burst rate reported by Champion et al. (2016) of R(F~0.6 Jy ms) = 7 x 10^3 sky^{-1} day^{-1} (for bursts of ~3 ms duration). This upper limit on \alpha suggests that if FRBs are extragalactic but non-cosmological, that on average they are not experiencing excessive free-free absorption due to a medium with high optical depth (assuming temperature ~8,000 K), which excessively invert their low-frequency spectrum. This in turn implies that the dispersion of FRBs arises in either or both of the intergalactic medium or the host galaxy, rather than from the source itself.

astro-ph.HE↗

arXiv2013

Real Time Event Detection in Astronomical Data Streams: Lessons from the VLBA

A new generation of observational science instruments is dramatically increasing collected data volumes in a range of fields. These instruments include the Square Kilometre Array (SKA), Large Synoptic Survey Telescope (LSST), terrestrial sensor networks, and NASA satellites participating in "decadal survey" missions. Their unprecedented coverage and sensitivity will likely reveal wholly new categories of unexpected and transient events. Commensal methods passively analyze these data streams, recognizing anomalous events of scientific interest and reacting in real time. We report on a case example: V-FASTR, an ongoing commensal experiment at the Very Long Baseline Array (VLBA) that uses online adaptive pattern recognition to search for anomalous fast radio transients. V-FASTR triages a millisecond-resolution stream of data and promotes candidate anomalies for further offline analysis. It tunes detection parameters in real time, injecting synthetic events to continually retrain itself for optimum performance. This self-tuning approach retains sensitivity to weak signals while adapting to changing instrument configurations and noise conditions. The system has operated since July 2011, making it the longest-running real time commensal radio transient experiment to date.

astro-ph.IM↗

arXiv2013

A framework for interpreting fast radio transients search experiments: application to the V-FASTR experiment

We define a framework for determining constraints on the detection rate of fast transient events from a population of underlying sources, with a view to incorporating beam shape, frequency effects, scattering effects, and detection efficiency into the metric. We then demonstrate a method for combining independent datasets into a single event rate constraint diagram, using a probabilistic approach to the limits on parameter space. We apply this new framework to present the latest results from the V-FASTR experiment, a commensal fast transients search using the Very Long Baseline Array (VLBA). In the 20 cm band, V-FASTR now has the ability to probe the regions of parameter space of importance for the observed Lorimer and Keane fast radio transient candidates, by combining the information from observations with differing bandwidths, and properly accounting for the source dispersion measure, VLBA antenna beam shape, experiment time sampling, and stochastic nature of events. We then apply the framework to combine the results of the V-FASTR and ATA Fly's Eye experiments, demonstrating their complementarity. Expectations for fast transients experiments for the SKA Phase I dish array are then computed, and the impact of large differential bandwidths is discussed.

astro-ph.IM↗

arXiv2012

Limits on the event rates of fast radio transients from the V-FASTR experiment

We present the first results from the V-FASTR experiment, a commensal search for fast transient radio bursts using the Very Long Baseline Array (VLBA). V-FASTR is unique in that the widely spaced VLBA antennas provide a discriminant against non-astronomical signals and a mechanism for the localization and identification of events that is not possible with single dishes or short baseline interferometers. Thus far V-FASTR has accumulated over 1300 hours of observation time with the VLBA, between 90 cm and 3 mm wavelength (327 MHz - 86 GHz), providing the first limits on fast transient event rates at high radio frequencies (>1.4 GHz). V-FASTR has blindly detected bright individual pulses from seven known pulsars but has not detected any single-pulse events that would indicate high redshift impulsive bursts of radio emission. At 1.4 GHz, V-FASTR puts limits on fast transient event rates comparable with the PALFA survey at the Arecibo telescope, but generally at lower sensitivities, and comparable to the "fly's eye" survey at the Allen Telescope Array, but with less sky coverage. We also illustrate the likely performance of the Phase 1 SKA dish array for an incoherent fast transient search fashioned on V-FASTR.

astro-ph.HE↗

arXiv2011

V-FASTR: The VLBA Fast Radio Transients Experiment

Recent discoveries of dispersed, non-periodic impulsive radio signals with single-dish radio telescopes have sparked significant interest in exploring the relatively uncharted space of fast transient radio signals. Here we describe V-FASTR, an experiment to perform a blind search for fast transient radio signals using the Very Long Baseline Array (VLBA). The experiment runs entirely in a commensal mode, alongside normal VLBA observations and operations. It is made possible by the features and flexibility of the DiFX software correlator that is used to process VLBA data. Using the VLBA for this type of experiment offers significant advantages over single-dish experiments, including a larger field of view, the ability to easily distinguish local radio-frequency interference from real signals and the possibility to localize detected events on the sky to milliarcsecond accuracy. We describe our software pipeline, which accepts short integration (~ms) spectrometer data from each antenna in real time during correlation and performs an incoherent dedispersion separately for each antenna, over a range of trial dispersion measures. The dedispersed data are processed by a sophisticated detector and candidate events are recorded. At the end of the correlation, small snippets of the raw data at the time of the events are stored for further analysis. We present the results of our event detection pipeline from some test observations of the pulsars B0329+54 and B0531+21 (the Crab pulsar).

astro-ph.IM↗

arXiv2011

Detection of fast radio transients with multiple stations: a case study using the Very Long Baseline Array

Recent investigations reveal an important new class of transient radio phenomena that occur on sub-millisecond timescales. Often transient surveys' data volumes are too large to archive exhaustively. Instead, an on-line automatic system must excise impulsive interference and detect candidate events in real-time. This work presents a case study using data from multiple geographically distributed stations to perform simultaneous interference excision and transient detection. We present several algorithms that incorporate dedispersed data from multiple sites, and report experiments with a commensal real-time transient detection system on the Very Long Baseline Array (VLBA). We test the system using observations of pulsar B0329+54. The multiple-station algorithms enhanced sensitivity for detection of individual pulses. These strategies could improve detection performance for a future generation of geographically distributed arrays such as the Australian Square Kilometre Array Pathfinder and the Square Kilometre Array.

astro-ph.IM↗