arXiv Science⌕ Search

arXiv subjects

Liang Li

Publications and source records attributed to Liang Li.

At least 199 records · Page 11Linked to original sources
arXiv2024

Context-aware Difference Distilling for Multi-change Captioning

Multi-change captioning aims to describe complex and coupled changes within an image pair in natural language. Compared with single-change captioning, this task requires the model to have higher-level cognition ability to reason an arbitrary number of changes. In this paper, we propose a novel context-aware difference distilling (CARD) network to capture all genuine changes for yielding sentences. Given an image pair, CARD first decouples context features that aggregate all similar/dissimilar semantics, termed common/difference context features. Then, the consistency and independence constraints are designed to guarantee the alignment/discrepancy of common/difference context features. Further, the common context features guide the model to mine locally unchanged features, which are subtracted from the pair to distill locally difference features. Next, the difference context features augment the locally difference features to ensure that all changes are distilled. In this way, we obtain an omni-representation of all changes, which is translated into linguistic sentences by a transformer decoder. Extensive experiments on three public datasets show CARD performs favourably against state-of-the-art methods.The code is available at https://github.com/tuyunbin/CARD.

cs.CV↗
arXiv2024

Ten Supernova-rise in Binary Driven Gamma-ray Bursts

The observation of a gamma-ray burst (GRB) associated with a supernova (SN) coincides remarkably with the energy output from a binary system comprising a very massive carbon-oxygen (CO) core and an associated binary neutron star (NS) by the Binary-Driven Hypernova (BdHN) model. The dragging effect in the late evolution of such systems leads to co-rotation, with binary periods on the order of minutes, resulting in a very fast rotating core and a binary NS companion at a distance of $\sim 10^5$ km. Such a fast-rotating CO core, stripped of its hydrogen and helium, undergoes gravitational collapse and, within a fraction of seconds, leads to a supernova (SN) and a newly born, fast-spinning neutron star ($ν$NS), we name the emergence of the SN and the $ν$NS as the SN-rise and $ν$NS-rise. Typically, the SN energies range from $10^{51}$ to $10^{53}$ erg. We address this issue by examining 10 cases of Type-I BdHNe, the most energetic ones, in which SN accretion onto the companion NS leads to the formation of a black hole (BH). In all ten cases, the energetics of the SN events are estimated, ranging between $0.18$ and $12 \times 10^{52}$ erg. Additionally, in all 8 sources at redshift $z$ closer than $4.61$, a clear thermal blackbody component has been identified, with temperatures between $6.2$ and $39.99$ keV, as a possible signature of pair-driven SN. The triggering of the X-ray afterglow induced by the $ν$NS-rise are identified in three cases at high redshift where early X-ray observations are achievable, benefits from the interplay of cosmological effects.

astro-ph.HE↗
arXiv2024

An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training

Masked image modeling (MIM) pre-training for large-scale vision transformers (ViTs) has enabled promising downstream performance on top of the learned self-supervised ViT features. In this paper, we question if the \textit{extremely simple} lightweight ViTs' fine-tuning performance can also benefit from this pre-training paradigm, which is considerably less studied yet in contrast to the well-established lightweight architecture design methodology. We use an observation-analysis-solution flow for our study. We first systematically observe different behaviors among the evaluated pre-training methods with respect to the downstream fine-tuning data scales. Furthermore, we analyze the layer representation similarities and attention maps across the obtained models, which clearly show the inferior learning of MIM pre-training on higher layers, leading to unsatisfactory transfer performance on data-insufficient downstream tasks. This finding is naturally a guide to designing our distillation strategies during pre-training to solve the above deterioration problem. Extensive experiments have demonstrated the effectiveness of our approach. Our pre-training with distillation on pure lightweight ViTs with vanilla/hierarchical design ($5.7M$/$6.5M$) can achieve $79.4\%$/$78.9\%$ top-1 accuracy on ImageNet-1K. It also enables SOTA performance on the ADE20K segmentation task ($42.8\%$ mIoU) and LaSOT tracking task ($66.1\%$ AUC) in the lightweight regime. The latter even surpasses all the current SOTA lightweight CPU-realtime trackers.

cs.CV↗
arXiv2024

Global existence and Rayleigh-Taylor instability for the semi-dissipative Boussinesq system with Naiver boundary conditions

Considered herein is the global existence of weak, strong solutions and Rayleigh-Taylor (RT) instability for 2D semi-dissipative Boussinesq equations in an infinite strip domain $Ω_{\infty}$ subject to Navier boundary conditions with non-positive slip coefficients. We first prove the global existence of weak and strong solutions on bounded domain $Ω_{R}$ via the Galerkin method, characteristic analyzing technique and Stokes estimates etc. Based on above results, we further derive the uniform estimates, independent of the length of horizontal direction of $Ω_{R}$, ensuring the global existence of weak and strong solutions in unbounded case $Ω_{\infty}$ by utilizing the domain expansion method. Moreover, when the steady temperature is higher with decreasing height (i.e., RT steady-state) on certain region, we demonstrate that the steady-state is linear unstable through the construction of energy functional and the settlement of a family of modified variational problems. Furthermore, with the help of unstable solutions constructed in linear instability and global existence theorems, we confirm the instability of nonlinear problem in a Lipschitz structural sense. Finally, we give a series of rigorous verification (see Appendix) including the spectra of Stokes equations with Navier boundary conditions, Sobolev embedding inequalities, trace inequalities, and Stokes estimates under Navier boundary conditions etc, used in the proof of main conclusions.

math.DS↗
arXiv2024

Progressive Depth Decoupling and Modulating for Flexible Depth Completion

Image-guided depth completion aims at generating a dense depth map from sparse LiDAR data and RGB image. Recent methods have shown promising performance by reformulating it as a classification problem with two sub-tasks: depth discretization and probability prediction. They divide the depth range into several discrete depth values as depth categories, serving as priors for scene depth distributions. However, previous depth discretization methods are easy to be impacted by depth distribution variations across different scenes, resulting in suboptimal scene depth distribution priors. To address the above problem, we propose a progressive depth decoupling and modulating network, which incrementally decouples the depth range into bins and adaptively generates multi-scale dense depth maps in multiple stages. Specifically, we first design a Bins Initializing Module (BIM) to construct the seed bins by exploring the depth distribution information within a sparse depth map, adapting variations of depth distribution. Then, we devise an incremental depth decoupling branch to progressively refine the depth distribution information from global to local. Meanwhile, an adaptive depth modulating branch is developed to progressively improve the probability representation from coarse-grained to fine-grained. And the bi-directional information interactions are proposed to strengthen the information interaction between those two branches (sub-tasks) for promoting information complementation in each branch. Further, we introduce a multi-scale supervision mechanism to learn the depth distribution information in latent features and enhance the adaptation capability across different scenes. Experimental results on public datasets demonstrate that our method outperforms the state-of-the-art methods. The code will be open-sourced at [this https URL](https://github.com/Cisse-away/PDDM).

cs.CV↗
arXiv2024

Propensity Score Analysis with Guaranteed Subgroup Balance

Estimating the causal treatment effects by subgroups is important in observational studies when the treatment effect heterogeneity may be present. Existing propensity score methods rely on a correctly specified propensity score model. Model misspecification results in biased treatment effect estimation and covariate imbalance. We proposed a new algorithm, the propensity score analysis with guaranteed subgroup balance (G-SBPS), to achieve covariate mean balance in all subgroups. We further incorporated nonparametric kernel regression for the propensity scores and developed a kernelized G-SBPS (kG-SBPS) to improve the subgroup mean balance of covariate transformations in a rich functional class. This extension is more robust to propensity score model misspecification. Extensive numerical studies showed that G-SBPS and kG-SBPS improve both subgroup covariate balance and subgroup treatment effect estimation, compared to existing approaches. We applied G-SBPS and kG-SBPS to a dataset on right heart catheterization to estimate the subgroup average treatment effects on the hospital length of stay and a dataset on diabetes self-management training to estimate the subgroup average treatment effects for the treated on the hospitalization rate.

stat.ME↗
arXiv2024

Oxygen vacancies modulated VO2 for neurons and Spiking Neural Network construction

Artificial neuronal devices are the basic building blocks for neuromorphic computing systems, which have been motivated by realistic brain emulation. Aiming for these applications, various device concepts have been proposed to mimic the neuronal dynamics and functions. While till now, the artificial neuron devices with high efficiency, high stability and low power consumption are still far from practical application. Due to the special insulator-metal phase transition, Vanadium Dioxide (VO2) has been considered as an idea candidate for neuronal device fabrication. However, its intrinsic insulating state requires the VO2 neuronal device to be driven under large bias voltage, resulting in high power consumption and low frequency. Thus in the current study, we have addressed this challenge by preparing oxygen vacancies modulated VO2 film(VO2-x) and fabricating the VO2-x neuronal devices for Spiking Neural Networks (SNNs) construction. Results indicate the neuron devices can be operated under lower voltage with improved processing speed. The proposed VO2-x based back-propagation SNNs (BP-SNNs) system, trained with the MNIST dataset, demonstrates excellent accuracy in image recognition. Our study not only demonstrates the VO2-x based neurons and SNN system for practical application, but also offers an effective way to optimize the future neuromorphic computing systems by defect engineering strategy.

cs.NE↗
arXiv2024

Can AI Understand Our Universe? Test of Fine-Tuning GPT by Astrophysical Data

ChatGPT has been the most talked-about concept in recent months, captivating both professionals and the general public alike, and has sparked discussions about the changes that artificial intelligence (AI) will bring to the world. As physicists and astrophysicists, we are curious about if scientific data can be correctly analyzed by large language models (LLMs) and yield accurate physics. In this article, we fine-tune the generative pre-trained transformer (GPT) model by the astronomical data from the observations of galaxies, quasars, stars, gamma-ray bursts (GRBs), and the simulations of black holes (BHs), the fine-tuned model demonstrates its capability to classify astrophysical phenomena, distinguish between two types of GRBs, deduce the redshift of quasars, and estimate BH parameters. We regard this as a successful test, marking the LLM's proven efficacy in scientific research. With the ever-growing volume of multidisciplinary data and the advancement of AI technology, we look forward to the emergence of a more fundamental and comprehensive understanding of our universe. This article also shares some interesting thoughts on data collection and AI design. Using the approach of understanding the universe - looking outward at data and inward for fundamental building blocks - as a guideline, we propose a method of series expansion for AI, suggesting ways to train and control AI that is smarter than humans.

astro-ph.IM↗
arXiv2024

Emergency Caching: Coded Caching-based Reliable Map Transmission in Emergency Networks

Many rescue missions demand effective perception and real-time decision making, which highly rely on effective data collection and processing. In this study, we propose a three-layer architecture of emergency caching networks focusing on data collection and reliable transmission, by leveraging efficient perception and edge caching technologies. Based on this architecture, we propose a disaster map collection framework that integrates coded caching technologies. Our framework strategically caches coded fragments of maps across unmanned aerial vehicles (UAVs), fostering collaborative uploading for augmented transmission reliability. Additionally, we establish a comprehensive probability model to assess the effective recovery area of disaster maps. Towards the goal of utility maximization, we propose a deep reinforcement learning (DRL) based algorithm that jointly makes decisions about cooperative UAVs selection, bandwidth allocation and coded caching parameter adjustment, accommodating the real-time map updates in a dynamic disaster situation. Our proposed scheme is more effective than the non-coding caching scheme, as validated by simulation.

cs.NI↗
arXiv2024

Probing electromagnetic-gravitational wave emission coincidence in type I binary-driven hypernova family of long GRBs at very-high redshift

The repointing time of the XRT instrument on the Neil Gehrels Swift Observatory satellite has posed challenges in observing and studying the early X-ray emissions within $\approx40$ s after a gamma-ray burst (GRB) trigger. To address this issue, we adopt a novel approach that capitalizes on the cosmological time dilation in GRBs with redshifts ranging from $3$ to $9$. Applying this strategy to Swift/XRT data, we investigate the earliest X-ray emissions of $368$ GRBs from the Swift catalog, including short and long GRBs. We compare the time delay between the GRB trigger and the initial Swift/XRT observation, measured in the GRB observer frame (OTD) and the cosmological rest-frame (RTD). This technique is here used in the analysis of GRB 090423 at $z=8.233$ (RTD $\sim8.2$ s), GRB 090429B at $z\approx9.4$ (RTD $\sim10.1$ s), and GRB 220101A at $z=4.61$ (RTD $\sim14.4$ s). The cosmological time dilation enables us to observe the very early X-ray afterglow emission in these three GRBs. We thus validate the observation of the collapse of the carbon-oxygen (CO) core and the coeval newborn neutron star ($ν$NS) formation triggering the GRB event in the binary-driven hypernova (BdHN) scenario. We also evidence the $ν$NS spin-up due to supernova ejecta fallback and its subsequent slowing down due to the X-optical-radio synchrotron afterglow emission. A brief gravitational wave signal may separate the two stages due to a fast-spinning $ν$NS triaxial-to-axisymmetric transition. We also analyze the long GRB redshift distribution for the different BdHN types and infer that BdHNe II and III may originate the NS binary progenitors of short GRBs.

astro-ph.HE↗
arXiv2024

Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization

Diffusion-based text-to-image personalization have achieved great success in generating subjects specified by users among various contexts. Even though, existing finetuning-based methods still suffer from model overfitting, which greatly harms the generative diversity, especially when given subject images are few. To this end, we propose Pick-and-Draw, a training-free semantic guidance approach to boost identity consistency and generative diversity for personalization methods. Our approach consists of two components: appearance picking guidance and layout drawing guidance. As for the former, we construct an appearance palette with visual features from the reference image, where we pick local patterns for generating the specified subject with consistent identity. As for layout drawing, we outline the subject's contour by referring to a generative template from the vanilla diffusion model, and inherit the strong image prior to synthesize diverse contexts according to different text conditions. The proposed approach can be applied to any personalized diffusion models and requires as few as a single reference image. Qualitative and quantitative experiments show that Pick-and-Draw consistently improves identity consistency and generative diversity, pushing the trade-off between subject fidelity and image-text fidelity to a new Pareto frontier.

cs.CV↗
arXiv2024

LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detection

Due to highly constrained computing power and memory, deploying 3D lidar-based detectors on edge devices equipped in autonomous vehicles and robots poses a crucial challenge. Being a convenient and straightforward model compression approach, Post-Training Quantization (PTQ) has been widely adopted in 2D vision tasks. However, applying it directly to 3D lidar-based tasks inevitably leads to performance degradation. As a remedy, we propose an effective PTQ method called LiDAR-PTQ, which is particularly curated for 3D lidar detection (both SPConv-based and SPConv-free). Our LiDAR-PTQ features three main components, \textbf{(1)} a sparsity-based calibration method to determine the initialization of quantization parameters, \textbf{(2)} a Task-guided Global Positive Loss (TGPL) to reduce the disparity between the final predictions before and after quantization, \textbf{(3)} an adaptive rounding-to-nearest operation to minimize the layerwise reconstruction error. Extensive experiments demonstrate that our LiDAR-PTQ can achieve state-of-the-art quantization performance when applied to CenterPoint (both Pillar-based and Voxel-based). To our knowledge, for the very first time in lidar-based 3D detection tasks, the PTQ INT8 model's accuracy is almost the same as the FP32 model while enjoying $3\times$ inference speedup. Moreover, our LiDAR-PTQ is cost-effective being $30\times$ faster than the quantization-aware training method. Code will be released at \url{https://github.com/StiphyJay/LiDAR-PTQ}.

cs.CV↗
arXiv2024

Application of Graph Neural Networks in Dark Photon Search with Visible Decays at Future Beam Dump Experiment

Beam dump experiments provide a distinctive opportunity to search for dark photons, which are compelling candidates for dark matter with low mass. In this study, we propose the application of Graph Neural Networks (GNN) in tracking reconstruction with beam dump experiments to obtain high resolution in both tracking and vertex reconstruction. Our findings demonstrate that in a typical 3-track scenario with the visible decay mode, the GNN approach significantly outperforms the traditional approach, improving the 3-track reconstruction efficiency by up to 88% in the low mass region. Furthermore, we show that improving the minimal vertex detection distance significantly impacts the signal sensitivity in dark photon searches with the visible decay mode. By reducing the minimal vertex distance from 5 mm to 0.1 mm, the exclusion upper limit on the dark photon mass ($m_A\prime$) can be improved by up to a factor of 3.

hep-ex↗
arXiv2024

Neutrino Reconstruction in TRIDENT Based on Graph Neural Network

TRopIcal DEep-sea Neutrino Telescope (TRIDENT) is a next-generation neutrino telescope to be located in the South China Sea. With a large detector volume and the use of advanced hybrid digital optical modules (hDOMs), TRIDENT aims to discover multiple astrophysical neutrino sources and probe all-flavor neutrino physics. The reconstruction resolution of primary neutrinos is on the critical path to these scientific goals. We have developed a novel reconstruction method based on graph neural network (GNN) for TRIDENT. In this paper, we present the reconstruction performance of the GNN-based approach on both track- and shower-like neutrino events in TRIDENT.

hep-ex↗
arXiv2024

Optimal Stirring Strategies for Passive Scalars in a Domain with a General Shape and No-Flux Boundary Condition

Multiscale metrics such as negative Sobolev norms are effective for quantifying the degree of mixedness of a passive scalar field advected by an incompressible flow in the absence of diffusion. In this paper we introduce a mix norm that is motivated by Sobolev norm $H^{-1}$ for a general domain with a no-flux boundary. We then derive an explicit expression for the optimal flow that maximizes the instantaneous decay rate of the mix norm under fixed energy and enstrophy constraints. Numerical simulations indicate that the mix norm decays exponentially or faster for various initial conditions and geometries and the rate is closely related to the smallest non-zero eigenvalue of the Laplace operator. These results generalize previous findings restricted for a periodic domain for its analytical and numerical simplicity. Additionally, we observe that periodic boundaries tend to induce a faster decay in mix norm compared to no-flux conditions under the fixed energy constraint, while the comparison is reversed for the fixed enstrophy constraint. In the special case of even initial distributions, two types of boundary conditions yield the same optimal flow and mix norm decay.

math.OC↗
arXiv2024

Unifying Structured Data as Graph for Data-to-Text Pre-Training

Data-to-text (D2T) generation aims to transform structured data into natural language text. Data-to-text pre-training has proved to be powerful in enhancing D2T generation and yields impressive performances. However, previous pre-training methods either oversimplified structured data into a sequence without considering input structures or designed training objectives tailored for a specific data structure (e.g., table or knowledge graph). In this paper, we unify different types of structured data (i.e., table, key-value data, knowledge graph) into the graph format and cast different data-to-text generation tasks as graph-to-text generation. To effectively exploit the structural information of the input graph, we propose a structure-enhanced pre-training method for D2T generation by designing a structure-enhanced Transformer. Concretely, we devise a position matrix for the Transformer, encoding relative positional information of connected nodes in the input graph. In addition, we propose a new attention matrix to incorporate graph structures into the original Transformer by taking the available explicit connectivity structure into account. Extensive experiments on six benchmark datasets show the effectiveness of our model. Our source codes are available at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/unid2t.

cs.CL↗
arXiv2023

Context Disentangling and Prototype Inheriting for Robust Visual Grounding

Visual grounding (VG) aims to locate a specific target in an image based on a given language query. The discriminative information from context is important for distinguishing the target from other objects, particularly for the targets that have the same category as others. However, most previous methods underestimate such information. Moreover, they are usually designed for the standard scene (without any novel object), which limits their generalization to the open-vocabulary scene. In this paper, we propose a novel framework with context disentangling and prototype inheriting for robust visual grounding to handle both scenes. Specifically, the context disentangling disentangles the referent and context features, which achieves better discrimination between them. The prototype inheriting inherits the prototypes discovered from the disentangled visual features by a prototype bank to fully utilize the seen data, especially for the open-vocabulary scene. The fused features, obtained by leveraging Hadamard product on disentangled linguistic and visual features of prototypes to avoid sharp adjusting the importance between the two types of features, are then attached with a special token and feed to a vision Transformer encoder for bounding box regression. Extensive experiments are conducted on both standard and open-vocabulary scenes. The performance comparisons indicate that our method outperforms the state-of-the-art methods in both scenarios. {The code is available at https://github.com/WayneTomas/TransCP.

cs.CV↗