arXiv subjects

Ce Yu

Publications and source records attributed to Ce Yu.

At least 19 recordsLinked to original sources

arXiv2025

TDLight: A Framework for Incremental Light-Curve Management and Continuous Classification Optimization

Time-series observations of celestial objects are fundamental to studying variable and transient phenomena. As time-domain surveys continue to scale up, timely automated classification is essential for rapid follow-up. Many existing approaches treat storage and analysis as separate stages, archiving data first and analyzing it later. Yet each source is a continuously growing stream, and early observations often contain sufficient information to assign scientifically useful labels before the full light curve is available. Motivated by this, we present TDLight, a framework for light-curve management and trigger-based classification over incremental data streams. Built on the time-series database TDengine, the system loads the Gaia DR2 archive (approximately 4.8 x 10^7 rows) in 33 s and supports low-latency retrieval. As new observations accumulate, TDLight automatically triggers a lightweight LightGBM-based classification pipeline that requires no GPU. When prediction confidence exceeds a predefined threshold, these early results are written back to the database, providing reliable classifications before offline analysis is complete. The deployed system also supports manual label confirmation and optional incremental training. Evaluated on the LEAVES dataset (approximately 1.10 million sources spanning 10 classes), the system achieves an overall accuracy of 93.6% and a weighted F1 score of 93.7%. Furthermore, it identifies 560 high-confidence catalog-disagreement candidates for follow-up verification. Notably, this includes correcting 150 RRc variables erroneously labeled as EW binaries due to inherited period-doubling artifacts. TDLight and the related data products are publicly available.

astro-ph.IM↗

arXiv2025

A Survey on Efficiency Optimization Techniques for DNN-based Video Analytics: Process Systems, Algorithms, and Applications

The explosive growth of video data in recent years has brought higher demands for video analytics, where accuracy and efficiency remain the two primary concerns. Deep neural networks (DNNs) have been widely adopted to ensure accuracy; however, improving their efficiency in video analytics remains an open challenge. Different from existing surveys that make summaries of DNN-based video mainly from the accuracy optimization aspect, in this survey, we aim to provide a thorough review of optimization techniques focusing on the improvement of the efficiency of DNNs in video analytics. We organize existing methods in a bottom-up manner, covering multiple perspectives such as hardware support, data processing, operational deployment, etc. Finally, based on the optimization framework and existing works, we analyze and discuss the problems and challenges in the performance optimization of DNN-based video analytics.

cs.CV↗

arXiv2025

Solving Online Resource-Constrained Scheduling for Follow-Up Observation in Astronomy: a Reinforcement Learning Approach

In the astronomical observation field, determining the allocation of observation resources of the telescope array and planning follow-up observations for targets of opportunity (ToOs) are indispensable components of astronomical scientific discovery. This problem is computationally challenging, given the online observation setting and the abundance of time-varying factors that can affect whether an observation can be conducted. This paper presents ROARS, a reinforcement learning approach for online astronomical resource-constrained scheduling. To capture the structure of the astronomical observation scheduling, we depict every schedule using a directed acyclic graph (DAG), illustrating the dependency of timing between different observation tasks within the schedule. Deep reinforcement learning is used to learn a policy that can improve the feasible solution by iteratively local rewriting until convergence. It can solve the challenge of obtaining a complete solution directly from scratch in astronomical observation scenarios, due to the high computational complexity resulting from numerous spatial and temporal constraints. A simulation environment is developed based on real-world scenarios for experiments, to evaluate the effectiveness of our proposed scheduling approach. The experimental results show that ROARS surpasses 5 popular heuristics, adapts to various observation scenarios and learns effective strategies with hindsight.

cs.AI↗

arXiv2025

Task Scheduling in Geo-Distributed Computing: A Survey

Geo-distributed computing, a paradigm that assigns computational tasks to globally distributed nodes, has emerged as a promising approach in cloud computing, edge computing, cloud-edge computing and supercomputer computing (HPC). It enables low-latency services, ensures data locality, and handles large-scale applications. As global computing capacity and task demands increase rapidly, scheduling tasks for efficient execution in geo-distributed computing systems has become an increasingly critical research challenge. It arises from the inherent characteristics of geographic distribution, including heterogeneous network conditions, region-specific resource pricing, and varying computational capabilities across locations. Researchers have developed diverse task scheduling methods tailored to geo-distributed scenarios, aiming to achieve objectives such as performance enhancement, fairness assurance, and fault-tolerance improvement. This survey provides a comprehensive and systematic review of task scheduling techniques across four major distributed computing environments, with an in-depth analysis of these approaches based on their core scheduling objectives. Through our analysis, we identify key research challenges and outline promising directions for advancing task scheduling in geo-distributed computing.

cs.DC↗

arXiv2024

GRRIS: a real-time intra-site observation scheduling scheme for distributed survey telescope arrays

The distributed telescope array offers promise for conducting large-sky-area, high-frequency time domain surveys. Multiple telescopes can be deployed at each observation site, so intra-site observation task scheduling is crucial for enhancing observation efficiency and quality. Efficient use of observable time and rapid response to special situations are critical to maximize scientific discovery in time domain surveys. Besides, the competing scientific priorities, time-varying observation conditions, and capabilities of observation equipment, lead to a vast search space of the scheduling. So with the increasing number of telescopes and observation fields, balancing computational time with solution quality in observation scheduling poses a significant challenge. Informed by the seminal contributions of earlier studies on a multilevel scheduling model and global scheduler for time domain telescope array, this study is devoted to further exploring the site scheduler. Formulating the observation scheduling of multiple telescopes at the site as a cooperative decision-making problem, this paper proposes GRRIS, a real-time intra-site observation scheduling scheme for telescope array using graph and reinforcement learning. It employs a graph neural network to learn node features that can embed the spatial structure of the observation scheduling. An algorithm based on multi-agent reinforcement learning is designed to efficiently learn the optimum allocation policy of telescope agents to field nodes. Through numerical simulations with real-world scenarios, GRRIS can achieve up to a 22% solution improvement over the most competitive scheme. It offers better scalability and sub-second decision speed, meeting the needs of observation scheduling control for future distributed telescope arrays.

astro-ph.IM↗

arXiv2023

Data Release of the AST3-2 Automatic Survey from Dome A, Antarctica

AST3-2 is the second of the three Antarctic Survey Telescopes, aimed at wide-field time-domain optical astronomy. It is located at Dome A, Antarctica, which is by many measures the best optical astronomy site on the Earth's surface. Here we present the data from the AST3-2 automatic survey in 2016 and the photometry results. The median 5$\sigma$ limiting magnitude in $i$-band is 17.8 mag and the light curve precision is 4 mmag for bright stars. The data release includes photometry for over 7~million stars, from which over 3,500 variable stars were detected, with 70 of them newly discovered. We classify these new variables into different types by combining their light curve features with stellar properties from surveys such as StarHorse.

astro-ph.SR↗

arXiv2023

A Multilevel Scheduling Framework for Distributed Time-domain Large-area Sky Survey Telescope Array

Telescope arrays are receiving increasing attention due to their promise of higher resource utilization, greater sky survey area, and higher frequency of full space-time monitoring than single telescopes. Compared with the ordinary coordinated operation of several telescopes, the new astronomical observation mode has an order of magnitude difference in the number of telescopes. It requires efficient coordinated observation by large-domain telescopes distributed at different sites. Coherent modeling of various abstract environmental constraints is essential for responding to multiple complex science goals. Also, due to competing science priorities and field visibility, how the telescope arrays are scheduled for observations can significantly affect observation efficiency. This paper proposes a multilevel scheduling model oriented toward the problem of telescope-array scheduling for time-domain surveys. A flexible framework is developed with basic functionality encapsulated in software components implemented on hierarchical architectures. An optimization metric is proposed to self-consistently weight contributions from time-varying observation conditions to maintain uniform coverage and efficient time utilization from a global perspective. The performance of the scheduler is evaluated through simulated instances. The experimental results show that our scheduling framework performs correctly and provides acceptable solutions considering the percentage of time allocation efficiency and sky coverage uniformity in a feasible amount of time. Using a generic version of the telescope-array scheduling framework, we also demonstrate its scalability and its potential to be applied to other astronomical applications.

astro-ph.IM↗

arXiv2023

HLC2: a highly efficient cross-matching framework for large astronomical catalogues on heterogeneous computing environments

Cross-matching operation, which is to find corresponding data for the same celestial object or region from multiple catalogues,is indispensable to astronomical data analysis and research. Due to the large amount of astronomical catalogues generated by the ongoing and next-generation large-scale sky surveys, the time complexity of the cross-matching is increasing dramatically. Heterogeneous computing environments provide a theoretical possibility to accelerate the cross-matching, but the performance advantages of heterogeneous computing resources have not been fully utilized. To meet the challenge of cross-matching for substantial increasing amount of astronomical observation data, this paper proposes Heterogeneous-computing-enabled Large Catalogue Cross-matcher (HLC2), a high-performance cross-matching framework based on spherical position deviation on CPU-GPU heterogeneous computing platforms. It supports scalable and flexible cross-matching and can be directly applied to the fusion of large astronomical cataloguesfrom survey missions and astronomical data centres. A performance estimation model is proposed to locate the performance bottlenecks and guide the optimizations. A two-level partitioning strategy is designed to generate an optimized data placement according to the positions of celestial objects to increase throughput. To make HLC2 a more adaptive solution, the architecture-aware task splitting, thread parallelization, and concurrent scheduling strategies are designed and integrated. Moreover, a novel quad-direction strategy is proposed for the boundary problem to effectively balance performance and completeness. We have experimentally evaluated HLC2 using public released catalogue data. Experiments demonstrate that HLC2 scales well on different sizes of catalogues and the cross-matching speed is significantly improved compared to the state-of-the-art cross-matchers.

astro-ph.IM↗

arXiv2022

HEGrid: A High Efficient Multi-Channel Radio Astronomical Data Gridding Framework in Heterogeneous Computing Environments

The challenge to fully exploit the potential of existing and upcoming scientific instruments like large single-dish radio telescopes is to process the collected massive data effectively and efficiently. As a "quasi 2D stencil computation" with the "Moore neighborhood pattern," gridding is the most computationally intensive step in data reduction pipeline for radio astronomy studies, enabling astronomers to create correct sky images for further analysis. However, the existing gridding frameworks can either only run on multi-core CPU architecture or do not support high-concurrency, multi-channel data gridding. Their performance is then limited, and there are emerging needs for innovative gridding frameworks to process data from large single-dish radio telescopes like the Five-hundred-meter Aperture Spherical Telescope (FAST). To address those challenges, we developed a High Efficient Gridding framework, HEGrid, by overcoming the above limitations. Specifically, we propose and construct the gridding pipeline in heterogeneous computing environments and achieve multi-pipeline concurrency for high performance multi-channel processing. Furthermore, we propose pipeline-based co-optimization to alleviate the potential negative performance impact of possible intra- and inter-pipeline low computation and I/O utilization, including component share-based redundancy elimination, thread-level data reuse and overlapping I/O and computation. Our experiments are based on both simulated datasets and actual FAST observational datasets. The results show that HEGrid outperforms other state-of-the-art gridding frameworks by up to 5.5x and has robust hardware portability, including AMD Radeon Instinct GPU and NVIDIA GPU.

cs.DC↗

arXiv2020

HCGrid: A Convolution-based Gridding Framework for RadioAstronomy in Hybrid Computing Environments

Gridding operation, which is to map non-uniform data samples onto a uniformly distributedgrid, is one of the key steps in radio astronomical data reduction process. One of the mainbottlenecks of gridding is the poor computing performance, and a typical solution for suchperformance issue is the implementation of multi-core CPU platforms. Although such amethod could usually achieve good results, in many cases, the performance of gridding is stillrestricted to an extent due to the limitations of CPU, since the main workload of gridding isa combination of a large number of single instruction, multi-data-stream operations, which ismore suitable for GPU, rather than CPU implementations. To meet the challenge of massivedata gridding for the modern large single-dish radio telescopes, e.g., the Five-hundred-meterAperture Spherical radio Telescope (FAST), inspired by existing multi-core CPU griddingalgorithms such as Cygrid, here we present an easy-to-install, high-performance, and open-source convolutional gridding framework, HCGrid,in CPU-GPU heterogeneous platforms. Itoptimises data search by employing multi-threading on CPU, and accelerates the convolutionprocess by utilising massive parallelisation of GPU. In order to make HCGrid a more adaptivesolution, we also propose the strategies of thread organisation and coarsening, as well as optimalparameter settings under various GPU architectures. A thorough analysis of computing timeand performance gain with several GPU parallel optimisation strategies show that it can leadto excellent performance in hybrid computing environments.

astro-ph.IM↗

arXiv2020

NBFTP: A Dedicated Data Transfer System for Remote Astronomical Observation at Dome A

Dome A, Antarctica has been thought to be one of the best astronomical sites on the earth since decades ago. From it was first visited by astronomers in 2008, dozens of facilities for astronomical observation and site testing were deployed. Due to its special geographical location, the data and message exchange between Dome A and the domestic control center could only depend on Iridium. Because the link bandwidth of Iridium is extremely limited, meanwhile the network traffic cost is quite expensive and the network is rather unstable, the commonly used data transfer tools, such as rsync and scp, are not suitable in this case. In this paper, we design and implement a data transfer tool called NBFTP (narrow bandwidth file transfer protocol) for the astronomical observation of Dome A. NBFTP uses a uniform interface to arrange all types of data and matches specific transmission schemes for different data types according to rules. Break-point resuming and extensibility functions are also implemented. Our experimental results show that NBFTP consumes 60% less network traffic than rsync when detecting the data pending to be transferred. And when transferring small files of 1KB, the network traffic consumption of NBFTP is 40% less than rsync. However, as the file size increases, the network traffic consumption of NBFTP tends to approach rsync, but it is still smaller than rsync.

astro-ph.IM↗

arXiv2020

A Redistribution Tool for Long-Term Archive of Astronomical Observation Data

Astronomical observation data require long-term preservation, and the rapid accumulation of observation data makes it necessary to consider the cost of long-term archive storage. In addition to low-speed disk-based online storage, optical disk or tape-based offline storage can be used to save costs. However, for astronomical research that requires historical data (particularly time-domain astronomy), the performance and energy consumption of data-accessing techniques cause problems because the requested data (which are organized according to observation time) may be located across multiple storage devices. In this study, we design and develop a tool referred to as AstroLayout to redistribute the observation data using spatial aggregation. The core algorithm uses graph partitioning to generate an optimized data placement according to the original observation data statistics and the target storage system. For the given observation data, AstroLayout can copy the long-term archive in the target storage system in accordance with this placement. An efficiency evaluation shows that AstroLayout can reduce the number of devices activated when responding to data-access requests in time-domain astronomy research. In addition to improving the performance of data-accessing techniques, AstroLayout can also reduce the storage systems power consumption. For enhanced adaptability, it supports storage systems of any media, including optical disks, tapes, and hard disks.

astro-ph.IM↗

arXiv2020

AstroCatR: a Mechanism and Tool for Efficient Time Series Reconstruction of Large-Scale Astronomical Catalogues

Time series data of celestial objects are commonly used to study valuable and unexpected objects such as extrasolar planets and supernova in time domain astronomy. Due to the rapid growth of data volume, traditional manual methods are becoming extremely hard and infeasible for continuously analyzing accumulated observation data. To meet such demands, we designed and implemented a special tool named AstroCatR that can efficiently and flexibly reconstruct time series data from large-scale astronomical catalogues. AstroCatR can load original catalogue data from Flexible Image Transport System (FITS) files or databases, match each item to determine which object it belongs to, and finally produce time series datasets. To support the high-performance parallel processing of large-scale datasets, AstroCatR uses the extract-transform-load (ETL) preprocessing module to create sky zone files and balance the workload. The matching module uses the overlapped indexing method and an in-memory reference table to improve accuracy and performance. The output of AstroCatR can be stored in CSV files or be transformed other into formats as needed. Simultaneously, the module-based software architecture ensures the flexibility and scalability of AstroCatR. We evaluated AstroCatR with actual observation data from The three Antarctic Survey Telescopes (AST3). The experiments demonstrate that AstroCatR can efficiently and flexibly reconstruct all time series data by setting relevant parameters and configuration files. Furthermore, the tool is approximately 3X faster than methods using relational database management systems at matching massive catalogues.

astro-ph.IM↗

arXiv2020

Towards an Astronomical Science Platform: Experiences and Lessons Learned from Chinese Virtual Observatory

In the era of big data astronomy, next generation telescopes and large sky surveys produce data sets at the TB or even PB level. Due to their large data volumes, these astronomical data sets are extremely difficult to transfer and analyze using personal computers or small clusters. In order to offer better access to data, data centers now generally provide online science platforms that enable analysis close to the data. The Chinese Virtual Observatory (China-VO) is one of the member projects in the International Virtual Observatory Alliance and it is dedicated to providing a research and education environment where globally distributed astronomy archives are simple to find, access, and interoperate. In this study, we summarize highlights of the work conducted at the China-VO, as well the experiences and lessons learned during the full life-cycle management of astronomical data. Finally, We discuss the challenges and future trends for astronomical science platforms.

astro-ph.IM↗

arXiv2020

Deep residual detection of radio frequency interference for FAST

Radio frequency interference (RFI) detection and excision are key steps in the data-processing pipeline of the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Because of its high sensitivity and large data rate, FAST requires more accurate and efficient RFI flagging methods than its counterparts. In the last decades, approaches based upon artificial intelligence (AI), such as codes using convolutional neural networks (CNNs), have been proposed to identify RFI more reliably and efficiently. However, RFI flagging of FAST data with such methods has often proved to be erroneous, with further manual inspections required. In addition, network construction as well as preparation of training data sets for effective RFI flagging has imposed significant additional workloads. Therefore, rapid deployment and adjustment of AI approaches for different observations is impractical to implement with existing algorithms. To overcome such problems, we propose a model called RFI-Net. With the input of raw data without any processing, RFI-Net can detect RFI automatically, producing corresponding masks without any alteration of the original data. Experiments with RFI-Net using simulated astronomical data show that our model has outperformed existing methods in terms of both precision and recall. Besides, compared with other models, our method can obtain the same relative accuracy with fewer training data, thus reducing the effort and time required to prepare the training data set. Further, the training process of RFI-Net can be accelerated, with overfittings being minimized, compared with other CNN codes. The performance of RFI-Net has also been evaluated with observing data obtained by FAST and the Bleien Observatory. Our results demonstrate the ability of RFI-Net to accurately identify RFI with fine-grained, high-precision masks that required no further modification.

astro-ph.IM↗

arXiv2018

Exoplanets in the Antarctic Sky. I. The First Data Release of AST3-II (CHESPA) and New Found Variables within the Southern CVZ of TESS

Located at Dome A, the highest point of the Antarctic plateau, the Chinese Kunlun station is considered to be one of the best ground-based photometric sites because of its extremely cold, dry, and stable atmosphere(Saunders et al. 2009). A target can be monitored from there for over 40 days without diurnal interruption during a polar winter. This makes Kunlun station a perfect site to search for short-period transiting exoplanets. Since 2008, an observatory has been built at Kunlun station and three telescopes are working there. Using these telescopes, the AST3 project has been carried out over the last six years with a search for transiting exoplanets as one of its key programs (CHESPA). In the austral winters of 2016 and 2017, a set of target fields in the Southern CVZ of TESS (Ricker et al. 2009) were monitored by the AST3-II telescope. In this paper, we introduce the CHESPA and present the first data release containing photometry of 26,578 bright stars (m_i < 15). The best photometric precision at the optimum magnitude for the survey is around 2 mmag. To demonstrate the data quality, we also present a catalog of 221 variables with a brightness variation greater than 5 mmag from the 2016 data. Among these variables, 179 are newly identified periodic variables not listed in the AAVSO databasea), and 67 are listed in the Candidate Target List(Stassun et al. 2017). These variables will require careful attention to avoid false-positive signals when searching for transiting exoplanets. Dozens of new transiting exoplanet candidates will be also released in a subsequent paper(Zhang et al. 2018b).

astro-ph.IM↗

arXiv2018

A Survey on Spark Ecosystem for Big Data Processing

With the explosive increase of big data in industry and academic fields, it is necessary to apply large-scale data processing systems to analysis Big Data. Arguably, Spark is state of the art in large-scale data computing systems nowadays, due to its good properties including generality, fault tolerance, high performance of in-memory data processing, and scalability. Spark adopts a flexible Resident Distributed Dataset (RDD) programming model with a set of provided transformation and action operators whose operating functions can be customized by users according to their applications. It is originally positioned as a fast and general data processing system. A large body of research efforts have been made to make it more efficient (faster) and general by considering various circumstances since its introduction. In this survey, we aim to have a thorough review of various kinds of optimization techniques on the generality and performance improvement of Spark. We introduce Spark programming model and computing system, discuss the pros and cons of Spark, and have an investigation and classification of various solving techniques in the literature. Moreover, we also introduce various data management and processing systems, machine learning algorithms and applications supported by Spark. Finally, we make a discussion on the open issues and challenges for large-scale in-memory data processing with Spark.

cs.DC↗

arXiv2018

Meteorological data from KLAWS-2G for an astronomical site survey of Dome A, Antarctica

We present an analysis of meteorological data from the second generation of the Kunlun Automated Weather Station (KLAWS-2G) at Dome A, Antarctica during 2015 and 2016. We find that a strong temperature inversion exists for all the elevations up to 14 m that KLAWS-2G can reach, and lasts for more than 10 hours for 50% or more of the time when temperature inversion occurs. The average wind speeds at 4 m elevation are 4.2 m/s and 3.8 m/s during 2015 and 2016, respectively. The strong temperature inversion and moderate wind speed lead to a shallow turbulent boundary layer height at Dome A. By analyzing the temperature and wind shear profiles, we note telescopes should be elevated by at least 8 m above the ice. We also find that the duration of temperature inversions, and the wind speed, vary considerably from year to year. Therefore, long-term and continuous data are still needed for the site survey at Dome A.

astro-ph.IM↗