arXiv Science⌕ Search

arXiv subjects

Lei Ding

Publications and source records attributed to Lei Ding.

At least 55 records · Page 3Linked to original sources

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training

The effectiveness of large language models (LLMs) is often hindered by duplicated data in their extensive pre-training datasets. Current approaches primarily focus on detecting and removing duplicates, which risks the loss of valuable information and neglects the varying degrees of duplication. To address this, we propose a soft deduplication method that maintains dataset integrity while selectively reducing the sampling weight of data with high commonness. Central to our approach is the concept of "data commonness", a metric we introduce to quantify the degree of duplication by measuring the occurrence probabilities of samples using an n-gram model. Empirical analysis shows that this method significantly improves training efficiency, achieving comparable perplexity scores with at least a 26% reduction in required training steps. Additionally, it enhances average few-shot downstream accuracy by 1.77% when trained for an equivalent duration. Importantly, this approach consistently improves performance, even on rigorously deduplicated datasets, indicating its potential to complement existing methods and become a standard pre-training process for LLMs.

cs.CL↗

Multi-photon super-linear image scanning microscopy using upconversion nanoparticles

Super-resolution fluorescence microscopy is of great interest in life science studies for visualizing subcellular structures at the nanometer scale. Among various kinds of super-resolution approaches, image scanning microscopy (ISM) offers a doubled resolution enhancement in a simple and straightforward manner, based on the commonly used confocal microscopes. ISM is also suitable to be integrated with multi-photon microscopy techniques, such as two-photon excitation and second-harmonic generation imaging, for deep tissue imaging, but it remains the twofold limited resolution enhancement and requires expensive femtosecond lasers. Here, we present and experimentally demonstrate the super-linear ISM (SL-ISM) to push the resolution enhancement beyond the factor of two, with a single low-power, continuous-wave, and near-infrared laser, by harnessing the emission nonlinearity within the multiphoton excitation process of lanthanide-doped upconversion nanoparticles (UCNPs). Based on a modified confocal microscope, we achieve a resolution of about 120 nm, 1/8th of the excitation wavelength. Furthermore, we demonstrate a parallel detection strategy of SL-ISM with the multifocal structured excitation pattern, to speed up the acquisition frame rate. This method suggests a new perspective for super-resolution imaging or sensing, multi-photon imaging, and deep-tissue imaging with simple, low-cost, and straightforward implementations.

physics.optics↗

Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images

Vision Foundation Models (VFMs) such as the Segment Anything Model (SAM) allow zero-shot or interactive segmentation of visual contents, thus they are quickly applied in a variety of visual scenes. However, their direct use in many Remote Sensing (RS) applications is often unsatisfactory due to the special imaging characteristics of RS images. In this work, we aim to utilize the strong visual recognition capabilities of VFMs to improve the change detection of high-resolution Remote Sensing Images (RSIs). We employ the visual encoder of FastSAM, an efficient variant of the SAM, to extract visual representations in RS scenes. To adapt FastSAM to focus on some specific ground objects in the RS scenes, we propose a convolutional adaptor to aggregate the task-oriented change information. Moreover, to utilize the semantic representations that are inherent to SAM features, we introduce a task-agnostic semantic learning branch to model the semantic latent in bi-temporal RSIs. The resulting method, SAMCD, obtains superior accuracy compared to the SOTA methods and exhibits a sample-efficient learning ability that is comparable to semi-supervised CD methods. To the best of our knowledge, this is the first work that adapts VFMs for the CD of HR RSIs.

cs.CV↗

Fiber-based Ratiometric Optical Thermometry with Silicon-Vacancy in Microdiamonds

Fiber optic all-optical thermometry is a promising technology to track temperature at a micro-scale while designing efficient and reliable microelectronic devices and components. In this work, we demonstrate a novel real-time ratiometric fiber optic thermometry technique based on silicon-vacancy (SiV) diamond that shows the highest temperature resolution (22.91 KHz^(-1/2) Wcm^(-2)) and spatial resolution (~7.5 um) among all-optical fiber-based thermosensors reported to date. Instead of analyzing the spectral features of temperature-dependent SiV signal, coming from SiV micro-diamond fixed on the fiber tip, an alternative parallel detection method based on filtering optics and photon counters is proposed to read out the sample temperature in real-time. The signal collection efficiency of the fiber is also investigated numerically with semi-analytic ray-optical analysis and then compared with our experimental study. We finally demonstrate the performance of the thermosensor by monitoring the temperature at distinct locations in a lab-built graphite-based microheater device. Our work introduces a reconfigurable method for temperature monitoring in microelectronic, microfluidic devices, or biological environments and unlocks a new direction for fiber-based all-optical thermometry research.

physics.optics↗

Cryogenic Thermal Shock Effects on Optical Properties of Quantum Emitters in Hexagonal Boron Nitride

Solid-state quantum emitters are vital building blocks for quantum information science and quantum technology. Among various types of solid-state emitters discovered to date, color centers in hexagonal boron nitride have garnered tremendous traction in recent years thanks to their environmental robustness, high brightness and room-temperature operation. Most recently, these quantum emitters have been employed for satellite-based quantum key distribution. One of the most important requirements to qualify these emitters for space-based applications is their optical stability against cryogenic thermal shock. Such understanding has, however, remained elusive to date. Here, we report on the effects caused by such thermal shock which induces random, irreversible changes in the spectral characteristics of the quantum emitters. By employing a combination of structural characterizations and density functional calculations, we attribute the observed changes to lattice strains caused by the cryogenic temperature shock. Our study shed light on the stability of the quantum emitters under extreme conditions, similar to those countered in outer space.

physics.app-ph↗

Gaussian Differential Privacy on Riemannian Manifolds

We develop an advanced approach for extending Gaussian Differential Privacy (GDP) to general Riemannian manifolds. The concept of GDP stands out as a prominent privacy definition that strongly warrants extension to manifold settings, due to its central limit properties. By harnessing the power of the renowned Bishop-Gromov theorem in geometric analysis, we propose a Riemannian Gaussian distribution that integrates the Riemannian distance, allowing us to achieve GDP in Riemannian manifolds with bounded Ricci curvature. To the best of our knowledge, this work marks the first instance of extending the GDP framework to accommodate general Riemannian manifolds, encompassing curved spaces, and circumventing the reliance on tangent space summaries. We provide a simple algorithm to evaluate the privacy budget $μ$ on any one-dimensional manifold and introduce a versatile Markov Chain Monte Carlo (MCMC)-based algorithm to calculate $μ$ on any Riemannian manifold with constant curvature. Through simulations on one of the most prevalent manifolds in statistics, the unit sphere $S^d$, we demonstrate the superior utility of our Riemannian Gaussian mechanism in comparison to the previously proposed Riemannian Laplace mechanism for implementing GDP.

cs.CR↗

Temporal compressive edge imaging enabled by a lensless diffuser camera

Lensless imagers based on diffusers or encoding masks enable high-dimensional imaging from a single shot measurement and have been applied in various applications. However, to further extract image information such as edge detection, conventional post-processing filtering operations are needed after the reconstruction of the original object images in the diffuser imaging systems. Here, we present the concept of a temporal compressive edge detection method based on a lensless diffuser camera, which can directly recover a time sequence of edge images of a moving object from a single-shot measurement, without further post-processing steps. Our approach provides higher image quality during edge detection, compared with the conventional post-processing method. We demonstrate the effectiveness of this approach by both numerical simulation and experiments. The proof-of-concept approach can be further developed with other image post-process operations or versatile computer vision assignments toward task-oriented intelligent lensless imaging systems.

eess.IV↗

Online Local Differential Private Quantile Inference via Self-normalization

Based on binary inquiries, we developed an algorithm to estimate population quantiles under Local Differential Privacy (LDP). By self-normalizing, our algorithm provides asymptotically normal estimation with valid inference, resulting in tight confidence intervals without the need for nuisance parameters to be estimated. Our proposed method can be conducted fully online, leading to high computational efficiency and minimal storage requirements with $\mathcal{O}(1)$ space. We also proved an optimality result by an elegant application of one central limit theorem of Gaussian Differential Privacy (GDP) when targeting the frequently encountered median estimation problem. With mathematical proof and extensive numerical testing, we demonstrate the validity of our algorithm both theoretically and experimentally.

stat.ME↗

Lattice dynamics and spin excitations in the metal-organic framework [CH$_3$NH$_3$][Co(HCOO)$_3$]

In metal-organic-framework (MOF) perovskites, both magnetic and ferroelectric orderings can be readily realized by compounding spin and charge degrees of freedom. The hydrogen bonds that bridge the magnetic framework and organic molecules have long been thought of as a key in generating multiferroic properties. However, the underlying physical mechanisms remain unclear. Here, we combine neutron diffraction, quasielastic and inelastic neutron scattering, and THz spectroscopy techniques to thoroughly investigate the dynamical properties of the multiferroic MOF candidate [CH$_3$NH$_3$][Co(HCOO)$_3$] through its multiple phase transitions. The wide range of energy resolutions reachable by these techniques enables us to scrutinize the coupling between the molecules and the framework throughout the phase transitions and interrogate a possible magnetoelectric coupling. Our results also reveal a structural change around 220 K which may be associated with the activation of a nodding donkey mode of the methylammonium molecule due to the ordering of the CH$_3$ groups. Upon the occurrence of the modulated phase transition around 130 K, the methylammonium molecules undergo a freezing of its reorientational motions which is concomitant with a change of the lattice parameters and anomalies of collective lattice vibrations. No significant change has been however observed in the lattice dynamics around the magnetic ordering, which therefore indicates the absence of a substantial magneto-electric coupling in zero-field.

cond-mat.mtrl-sci↗

Joint Spatio-Temporal Modeling for the Semantic Change Detection in Remote Sensing Images

Semantic Change Detection (SCD) refers to the task of simultaneously extracting the changed areas and the semantic categories (before and after the changes) in Remote Sensing Images (RSIs). This is more meaningful than Binary Change Detection (BCD) since it enables detailed change analysis in the observed areas. Previous works established triple-branch Convolutional Neural Network (CNN) architectures as the paradigm for SCD. However, it remains challenging to exploit semantic information with a limited amount of change samples. In this work, we investigate to jointly consider the spatio-temporal dependencies to improve the accuracy of SCD. First, we propose a Semantic Change Transformer (SCanFormer) to explicitly model the 'from-to' semantic transitions between the bi-temporal RSIs. Then, we introduce a semantic learning scheme to leverage the spatio-temporal constraints, which are coherent to the SCD task, to guide the learning of semantic changes. The resulting network (SCanNet) significantly outperforms the baseline method in terms of both detection of critical semantic changes and semantic consistency in the obtained bi-temporal results. It achieves the SOTA accuracy on two benchmark datasets for the SCD.

cs.CV↗

Less is more: balancing noise reduction and data retention in fMRI with data-driven scrubbing

Artifacts in functional MRI (fMRI) data cause deviations from common distributional assumptions, introduce spatial and temporal outliers, and reduce the signal-to-noise ratio of the data -- all of which can have negative consequences for downstream statistical analysis. Scrubbing is a technique for excluding fMRI volumes thought to be contaminated by artifacts and generally comes in two flavors. Motion scrubbing based on subject head motion-derived measures is popular but suffers from a number of drawbacks, especially high rates of censoring of individual volumes and entire subjects. Alternatively, data-driven scrubbing methods like DVARS are based on observed noise in the processed fMRI timeseries and may avoid some of these issues. Here we propose "projection scrubbing", a novel data-driven scrubbing method based on a statistical outlier detection framework and strategic dimension reduction, including independent component analysis (ICA), to isolate artifactual variation. We undertake a comprehensive comparison of motion scrubbing with data-driven projection scrubbing and DVARS. We argue that an appropriate metric for the success of scrubbing is maximal data retention subject to reasonable performance on typical benchmarks of functional connectivity. We find that stringent motion scrubbing yields worsened validity, worsened reliability, and produced small improvements to fingerprinting. Meanwhile, data-driven scrubbing methods tend to yield greater improvements to fingerprinting while not generally worsening validity or reliability. Importantly, however, data-driven scrubbing excludes a fraction of the number of volumes or entire sessions compared to motion scrubbing. The ability of data-driven fMRI scrubbing to improve data retention without negatively impacting the quality of downstream analysis has major implications for sample sizes in population neuroscience research.

stat.AP↗

Deep Unsupervised Key Frame Extraction for Efficient Video Classification

Video processing and analysis have become an urgent task since a huge amount of videos (e.g., Youtube, Hulu) are uploaded online every day. The extraction of representative key frames from videos is very important in video processing and analysis since it greatly reduces computing resources and time. Although great progress has been made recently, large-scale video classification remains an open problem, as the existing methods have not well balanced the performance and efficiency simultaneously. To tackle this problem, this work presents an unsupervised method to retrieve the key frames, which combines Convolutional Neural Network (CNN) and Temporal Segment Density Peaks Clustering (TSDPC). The proposed TSDPC is a generic and powerful framework and it has two advantages compared with previous works, one is that it can calculate the number of key frames automatically. The other is that it can preserve the temporal information of the video. Thus it improves the efficiency of video classification. Furthermore, a Long Short-Term Memory network (LSTM) is added on the top of the CNN to further elevate the performance of classification. Moreover, a weight fusion strategy of different input networks is presented to boost the performance. By optimizing both video classification and key frame extraction simultaneously, we achieve better classification performance and higher efficiency. We evaluate our method on two popular datasets (i.e., HMDB51 and UCF101) and the experimental results consistently demonstrate that our strategy achieves competitive performance and efficiency compared with the state-of-the-art approaches.

cs.CV↗

Conformalized Fairness via Quantile Regression

Algorithmic fairness has received increased attention in socially sensitive domains. While rich literature on mean fairness has been established, research on quantile fairness remains sparse but vital. To fulfill great needs and advocate the significance of quantile fairness, we propose a novel framework to learn a real-valued quantile function under the fairness requirement of Demographic Parity with respect to sensitive attributes, such as race or gender, and thereby derive a reliable fair prediction interval. Using optimal transport and functional synchronization techniques, we establish theoretical guarantees of distribution-free coverage and exact fairness for the induced prediction interval constructed by fair quantiles. A hands-on pipeline is provided to incorporate flexible quantile regressions with an efficient fairness adjustment post-processing algorithm. We demonstrate the superior empirical performance of this approach on several benchmark datasets. Our results show the model's ability to uncover the mechanism underlying the fairness-accuracy trade-off in a wide range of societal and medical applications.

stat.ML↗

Distributed Robust Nash Equilibrium Seeking for Mixed-Order Games by a Neural-Network based Approach

In practical applications, decision-makers with heterogeneous dynamics may be engaged in the same decision-making process. This motivates us to study distributed Nash equilibrium seeking for games in which players are mixed-order (first- and second-order) integrators influenced by unknown dynamics and external disturbances in this paper. To solve this problem, we employ an adaptive neural network to manage unknown dynamics and disturbances, based on which a distributed Nash equilibrium seeking algorithm is developed by further adapting concepts from gradient-based optimization and multi-agent consensus. By constructing appropriate Lyapunov functions, we analytically prove convergence of the reported method. Theoretical investigations suggest that players' actions would be steered to an arbitrarily small neighborhood of the Nash equilibrium, which is also testified by simulations.

math.OC↗

Sufficient principal component regression for pattern discovery in transcriptomic data

Methods for global measurement of transcript abundance such as microarrays and RNA-Seq generate datasets in which the number of measured features far exceeds the number of observations. Extracting biologically meaningful and experimentally tractable insights from such data therefore requires high-dimensional prediction. Existing sparse linear approaches to this challenge have been stunningly successful, but some important issues remain. These methods can fail to select the correct features, predict poorly relative to non-sparse alternatives, or ignore any unknown grouping structures for the features. We propose a method called SuffPCR that yields improved predictions in high-dimensional tasks including regression and classification, especially in the typical context of omics with correlated features. SuffPCR first estimates sparse principal components and then estimates a linear model on the recovered subspace. Because the estimated subspace is sparse in the features, the resulting predictions will depend on only a small subset of genes. SuffPCR works well on a variety of simulated and experimental transcriptomic data, performing nearly optimally when the model assumptions are satisfied. We also demonstrate near-optimal theoretical guarantees.

stat.ME↗

Exploiting dynamic nonlinearity in upconversion nanoparticles for super-resolution imaging

Single-beam super-resolution microscopy, also known as superlinear microscopy, exploits the nonlinear response of fluorescent probes in confocal microscopy. The technique requires no complex purpose-built system, light field modulation, or beam shaping. Here, we present a strategy to enhance spatial resolution of superlinear microscopy by modulating excitation intensity during image acquisition. This modulation induces dynamic optical nonlinearity in upconversion nanoparticles (UCNPs), resulting in variations of higher spatial-frequency information in the obtained images. The high-order information can be extracted with a proposed weighted finite difference imaging algorithm from raw fluorescence images, to generate an image with a higher resolution than superlinear microscopy images. We apply this approach to resolve two adjacent nanoparticles within a diffraction-limited area, improving the resolution to 130 nm. This work suggests a new scope for developing dynamic nonlinear fluorescent probes in super-resolution nanoscopy.

physics.optics↗

Algorithms for Computing Wiener Indices of Acyclic and Unicyclic Graphs

Let $G=(V(G),E(G))$ be a molecular graph, where $V(G)$ and $E(G)$ are the sets of vertices (atoms) and edges (bonds). A topological index of a molecular graph is a numerical quantity which helps to predict the chemical/physical properties of the molecules. The Wiener, Wiener polarity and the terminal Wiener indices are the distance based topological indices. In this paper, we described a linear time algorithm {\bf(LTA)} that computes the Wiener index for acyclic graphs and extended this algorithm for unicyclic graphs. The same algorithms are modified to compute the terminal Wiener index and the Wiener polarity index. All these algorithms compute the indices in time $O(n)$.

math.CO↗

Looking Outside the Window: Wide-Context Transformer for the Semantic Segmentation of High-Resolution Remote Sensing Images

Long-range contextual information is crucial for the semantic segmentation of High-Resolution (HR) Remote Sensing Images (RSIs). However, image cropping operations, commonly used for training neural networks, limit the perception of long-range contexts in large RSIs. To overcome this limitation, we propose a Wide-Context Network (WiCoNet) for the semantic segmentation of HR RSIs. Apart from extracting local features with a conventional CNN, the WiCoNet has an extra context branch to aggregate information from a larger image area. Moreover, we introduce a Context Transformer to embed contextual information from the context branch and selectively project it onto the local features. The Context Transformer extends the Vision Transformer, an emerging kind of neural network, to model the dual-branch semantic correlations. It overcomes the locality limitation of CNNs and enables the WiCoNet to see the bigger picture before segmenting the land-cover/land-use (LCLU) classes. Ablation studies and comparative experiments conducted on several benchmark datasets demonstrate the effectiveness of the proposed method. In addition, we present a new Beijing Land-Use (BLU) dataset. This is a large-scale HR satellite dataset with high-quality and fine-grained reference labels, which can facilitate future studies in this field.

cs.CV↗