arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 1403.4358

Efficient Catalog Matching with Dropout Detection

Dongwei Fan·Tamás Budavári·Alexander S. Szalay·Chenzhou Cui·Yongheng Zhao
View original source ↗Read source document ↗Export metadata ↓

Abstract

Not only source catalogs are extracted from astronomy observations. Their sky coverage is always carefully recorded and used in statistical analyses, such as correlation and luminosity function studies. Here we present a novel method for catalog matching, which inherently builds on the coverage information for better performance and completeness. A modified version of the Zones Algorithm is introduced for matching partially overlapping observations, where irrelevant parts of the data are excluded up front for efficiency. Our design enables searches to focus on specific areas on the sky to further speed up the process. Another important advantage of the new method over traditional techniques is its ability to quickly detect dropouts, i.e., the missing components that are in the observed regions of the celestial sphere but did not reach the detection limit in some observations. These often provide invaluable insight into the spectral energy distribution of the matched sources but rarely available in traditional associations.

Explore related subjects

astro-ph.IM

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Dongwei Fan, Tamás Budavári, Alexander S. Szalay, Chenzhou Cui, Yongheng Zhao. 2014-03-18. Efficient Catalog Matching with Dropout Detection. https://doi.org/10.1086/669707

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Using 23 Years of ACS/SBC Data to Understand Backgrounds: Significant Reductions in Expected Background Levels

We have used 23 years of Hubble Space Telescope ACS/SBC data to study what background levels are encountered in practice and how much they vary. The backgrounds vary considerably, with F115LP, F122M, F125LP, PR110L, and PR130L all showing over an order of magnitude of variation in background between observations, apparently due to changes in airglow. The F150LP and F165LP filters, which are dominated by dark rate, not airglow, exhibit a far smaller variation in backgrounds. For the filters where the background is generally dominated by airglow, the backgrounds measured from the data are significantly lower than what the ETC predicts (as of ETC v33.2). The ETC predictions for `average' airglow are greater than the median of our measured background values by factors of 2.51, 2.64, 105, and 3.64, for F115LP, F122M, F125LP, and F140LP, respectively. A preliminary analysis suggests this could be due to certain OI airglow lines usually being fainter than expected by the ETC. With reduced reduced background levels, the shorter-wavelength SBC filters can conduct background-limited observations much more rapidly than had previously been expected. As of ETC v34.1, a new option will be included for SBC calculations, allowing users to employ empirical background percentiles to estimate required exposure times.

astro-ph.IM↗
arXiv2026

Using 23 Years of ACS/SBC Data to Understand Backgrounds: Explaining & Predicting Background Variations

Recent analysis of 23 years of Hubble Space Telescope ACS/SBC data has shown that background levels can vary considerably between observations, with most filters showing over an order of magnitude variation. For the shorter-wavelength filters, the background is understood to be dominated by airglow; however, what precisely drives background variations is not well constrained for any filter. Here, we explore the causes of the background variation. Using over 8,000 archival SBC observations, we developed a machine learning model that can accurately predict the background for an observation, based upon a set of 23 observational parameters. This model indicates that, depending on filter, the SBC background is generally dominated by Solar elevation, Solar separation angle, Earth limb angle of observation, SBC temperature, and target Galactic latitude.

astro-ph.IM↗
arXiv2026

Beyond the BLUE I: the advantage ceiling - how much can any estimator beat the matched filter in mm/submm survey data?

Papers claiming that convolutional neural networks outperform the matched filter are now common in astronomy. Since that filter is the best linear unbiased estimator (BLUE) for any noise of given covariance, and the minimum-variance unbiased estimator under Gaussian noise, such a claim requires that the noise covariance varies from image to image or the noise is non-Gaussian. We introduce the advantage ceiling eta >= 1, which quantifies both cases: the Fisher information for the amplitude in units of the matched filter's, computable from noise-only simulations before any network is trained, and a bound on the variance of any estimator unbiased at each true amplitude. We compute eta for a suite of millimeter/submillimeter survey noise models and for compact and extended source templates, using a variational score-matching ladder whose rungs are estimator classes of increasing statistical order, from linear to convolutional. With an instrumental white-noise floor, Gaussian noise returns eta = 0.94-1.06 (+-0.04); covariance mixtures give exact ceilings of 10.2 (spectral tilt, extended source), 2.1 (leaked components) and 1.1-1.8 otherwise; residual artifacts give eta <= 1.09. Source confusion gives the largest advantage, eta >= 3.8 (extended) and >= 2.4 (compact). Three unmodelled channels - a zeroed DC mode, a missing white floor and floating-point rounding - can manufacture spurious advantages of up to two orders of magnitude. A ResNet regressor is 6-15% less efficient than the matched filter on Gaussian noise once prior shrinkage is divided out, and realizes a factor 1.8 of the 10.2 available on the spectral-tilt mixture; a companion paper tests such regressors against these ceilings.

astro-ph.IM↗