arXiv subjects

Cheng Li

Publications and source records attributed to Cheng Li.

At least 271 records · Page 15Linked to original sources

arXiv2023

Highly depleted alkali metals in Jupiter's deep atmosphere

Water and ammonia vapors are known to be the major sources of spectral absorption at pressure levels observed by the microwave radiometer (MWR) on Juno. However, the brightness temperatures and limb darkening observed by the MWR at its longest wavelength channel of 50 cm (600 MHz) in the first 9 perijove passes indicate the existence of an additional source of opacity in the deep atmosphere of Jupiter (pressures beyond 100 bar). The absorption properties of ammonia and water vapor, and their relative abundances in Jupiter's atmosphere do not provide sufficient opacity in deep atmosphere to explain the 600 MHz channel observation. Here we show that free electrons due to the ionization of alkali metals, i.e. sodium, and potassium, with sub-solar metallicity [M/H] (log based 10 relative concentration to solar) in the range of [M/H] = -2 to [M/H] = -5 can provide the missing source of opacity in the deep atmosphere. If the alkali metals are not the source of additional opacity in the MWR data, then their metallicity at 1000 bars can only be even lower. The upper bound of -2 on the metallicity of the alkali metals contrasts with the other heavy elements -- C, N, S, Ar, Kr, and Xe -- which are all enriched relative to their solar abundances having a metallicity of approximately +0.5.

astro-ph.EP↗

arXiv2023

Telecom-band integrated multimode photonic quantum memory

Telecom-band integrated quantum memory is an elementary building block for developing quantum networks compatible with fiber communication infrastructures. Towards such a network with large capacity, an integrated multimode photonic quantum memory at telecom band has yet been demonstrated. Here we report a fiber-integrated multimode quantum storage of single photon at telecom band on a laser-written chip. The storage device is a fiber-pigtailed Er3+:LiNbO3 waveguide and allows a storage of up to 330 temporal modes of heralded single photon with 4-GHz-wide bandwidth at 1532 nm and a 167-fold increasing of coincidence detection rate with respect to single mode. Our memory system with all-fiber addressing is performed using telecom-band fiber-integrated and on-chip devices. The results represent an important step for the future quantum networks using integrated photonics devices.

quant-ph↗

arXiv2023

Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases

Improving the deployment efficiency of transformer-based language models has been challenging given their high computation and memory cost. While INT8 quantization has recently been shown to be effective in reducing both the memory cost and latency while preserving model accuracy, it remains unclear whether we can leverage INT4 (which doubles peak hardware throughput) to achieve further latency improvement. In this study, we explore the feasibility of employing INT4 weight and activation (W4A4) quantization for language models. Our findings indicate that W4A4 quantization introduces no to negligible accuracy degradation for encoder-only and encoder-decoder models, but causes a significant accuracy drop for decoder-only models. To materialize the performance gain using W4A4, we develop a highly optimized end-to-end W4A4 encoder inference pipeline supporting different quantization strategies. Our INT4 pipeline is $8.5\times$ faster for latency-oriented scenarios and up to $3\times$ for throughput-oriented scenarios compared to the inference of FP16, and improves the SOTA BERT INT8 performance from FasterTransformer by up to $1.7\times$. We provide insights into the failure cases when applying W4A4 to decoder-only models, and further explore the compatibility of INT4 quantization with other compression methods, like pruning and layer reduction.

cs.CL↗

arXiv2023

ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation

Post-training quantization (PTQ) has emerged as a promising technique for mitigating memory consumption and computational costs in large language models (LLMs). However, a systematic examination of various quantization schemes, model families, and quantization bit precision has been absent from the literature. In this paper, we conduct a comprehensive analysis of these factors by investigating the effects of PTQ on weight-only, activation-only, and weight-and-activation quantization using diverse methods such as round-to-nearest (RTN), GPTQ, ZeroQuant, and their variants. We apply these methods to two distinct model families with parameters ranging from 125M to 176B. Our contributions include: (1) a sensitivity analysis revealing that activation quantization is generally more susceptible to weight quantization, with smaller models often outperforming larger models in terms of activation quantization; (2) an evaluation and comparison of existing PTQ methods to optimize model size reduction while minimizing the impact on accuracy, revealing that none of the current methods can achieve the original model quality for quantization with either INT4-weight or INT4-weight-and-INT8-activation; (3) based on these insights, we propose an optimized method called Low-Rank Compensation (LoRC), which employs low-rank matrices to enhance model quality recovery with a minimal increase in model size.

cs.LG↗

arXiv2023

Machine learning-based characterization of hydrochar from biomass: Implications for sustainable energy and material production

Hydrothermal carbonization (HTC) is a process that converts biomass into versatile hydrochar without the need for prior drying. The physicochemical properties of hydrochar are influenced by biomass properties and processing parameters, making it challenging to optimize for specific applications through trial-and-error experiments. To save time and money, machine learning can be used to develop a model that characterizes hydrochar produced from different biomass sources under varying reaction processing parameters. Thus, this study aims to develop an inclusive model to characterize hydrochar using a database covering a range of biomass types and reaction processing parameters. The quality and quantity of hydrochar are predicted using two models (decision tree regression and support vector regression). The decision tree regression model outperforms the support vector regression model in terms of forecast accuracy (R2 > 0.88, RMSE < 6.848, and MAE < 4.718). Using an evolutionary algorithm, optimum inputs are identified based on cost functions provided by the selected model to optimize hydrochar for energy production, soil amendment, and pollutant adsorption, resulting in hydrochar yields of 84.31%, 84.91%, and 80.40%, respectively. The feature importance analysis reveals that biomass ash/carbon content and operating temperature are the primary factors affecting hydrochar production in the HTC process.

cs.LG↗

arXiv2023

Self-Supervised Federated Learning for Fast MR Imaging

Federated learning (FL) based magnetic resonance (MR) image reconstruction can facilitate learning valuable priors from multi-site institutions without violating patient's privacy for accelerating MR imaging. However, existing methods rely on fully sampled data for collaborative training of the model. The client that only possesses undersampled data can neither participate in FL nor benefit from other clients. Furthermore, heterogeneous data distributions hinder FL from training an effective deep learning reconstruction model and thus cause performance degradation. To address these issues, we propose a Self-Supervised Federated Learning method (SSFedMRI). SSFedMRI explores the physics-based contrastive reconstruction networks in each client to realize cross-site collaborative training in the absence of fully sampled data. Furthermore, a personalized soft update scheme is designed to simultaneously capture the global shared representations among different centers and maintain the specific data distribution of each client. The proposed method is evaluated on four datasets and compared to the latest state-of-the-art approaches. Experimental results demonstrate that SSFedMRI possesses strong capability in reconstructing accurate MR images both visually and quantitatively on both in-distribution and out-of-distribution datasets.

eess.IV↗

arXiv2023

A study on the Weibull and Pareto distributions motivated by Chvátal's theorem

Let $B(n,p)$ denote a binomial random variable with parameters $n$ and $p$. Chvátal's theorem says that for any fixed $n\geq 2$, as $m$ ranges over $\{0,\ldots,n\}$, the probability $q_m:=P(B(n,m/n)\leq m)$ is the smallest when $m$ is closest to $\frac{2n}{3}$. Motivated by this theorem, we consider the minimum value problem on the probability that a random variable is at most its expectation, when its distribution is the Weibull distribution or the Pareto distribution in this note.

math.PR↗

arXiv2023

Understand Data Preprocessing for Effective End-to-End Training of Deep Neural Networks

In this paper, we primarily focus on understanding the data preprocessing pipeline for DNN Training in the public cloud. First, we run experiments to test the performance implications of the two major data preprocessing methods using either raw data or record files. The preliminary results show that data preprocessing is a clear bottleneck, even with the most efficient software and hardware configuration enabled by NVIDIA DALI, a high-optimized data preprocessing library. Second, we identify the potential causes, exercise a variety of optimization methods, and present their pros and cons. We hope this work will shed light on the new co-design of ``data storage, loading pipeline'' and ``training framework'' and flexible resource configurations between them so that the resources can be fully exploited and performance can be maximized.

cs.LG↗

arXiv2023

Deep insight into charge equilibration and the effects on producing neutron-rich isotopes around N = 126 in the multinucleon transfer reactions

The dynamics of the charge equilibration (CE) and the effects on the production of the neutron-rich isotopes around $N = 126$ in multinucleon transfer reactions are still not well understood. In this Letter, we investigate the mechanism of the CE from different viewpoints by using the extended version of the dinuclear system model (DNS-sysu) and the improved quantum molecular dynamics (ImQMD) model. From the macroscopic and microscopic dynamical viewpoints, we find incomplete CE for the mass asymmetry reaction systems even in the very deep collisions, and the behavior of "inverse CE" that the tendency of the fragments is away from the $N/Z$ value of the compound system in the reaction $^{140}$Xe + $^{198}$Pt. Unlike the slow process presented in the ImQMD model, the behavior of fast equilibration with the characteristic time $\sim$ 0.52 zs is obtained based on the DNS-sysu model, which is consistent with the experimental data. By performing the systematic calculation, the correlation between the CE and the mass asymmetry of the reaction systems is clarified, which not only accounts for the observed intriguing phenomena of the CE but also provides essential information for producing the neutron-rich isotopes around $N = 126$.

nucl-th↗

arXiv2023

Model-based Federated Learning for Accurate MR Image Reconstruction from Undersampled k-space Data

Deep learning-based methods have achieved encouraging performances in the field of magnetic resonance (MR) image reconstruction. Nevertheless, to properly learn a powerful and robust model, these methods generally require large quantities of data, the collection of which from multiple centers may cause ethical and data privacy violation issues. Lately, federated learning has served as a promising solution to exploit multi-center data while getting rid of the data transfer between institutions. However, high heterogeneity exists in the data from different centers, and existing federated learning methods tend to use average aggregation methods to combine the client's information, which limits the performance and generalization capability of the trained models. In this paper, we propose a Model-based Federated learning framework (ModFed). ModFed has three major contributions: 1) Different from the existing data-driven federated learning methods, model-driven neural networks are designed to relieve each client's dependency on large data; 2) An adaptive dynamic aggregation scheme is proposed to address the data heterogeneity issue and improve the generalization capability and robustness the trained neural network models; 3) A spatial Laplacian attention mechanism and a personalized client-side loss regularization are introduced to capture the detailed information for accurate image reconstruction. ModFed is evaluated on three in-vivo datasets. Experimental results show that ModFed has strong capability in improving image reconstruction quality and enforcing model generalization capability when compared to the other five state-of-the-art federated learning approaches. Codes will be made available at https://github.com/ternencewu123/ModFed.

eess.IV↗

arXiv2023

The optimal detection angles for producing N=126 neutron-rich isotones in the multinucleon transfer reactions

The challenge of isotopic identification over a wide angular distribution has limited the measurement of neutron-rich nuclei produced via the multinucleon transfer (MNT) process. To investigate the optimal detection angles for the N=126 isotones, we propose a method to construct the reasonable scattering angles of the MNT products in the dinuclear system (DNS-sysu) model. The reactions $^{136,144}$Xe + $^{208}$Pb are investigated. The calculated results are in rather good agreement with the available experimental data in the reaction $^{136}$Xe + $^{208}$Pb. The entrance channel effects on the scattering angle are investigated. It is found that the scattering angular distribution strongly depends on the isospin and the impact parameter of the collision system. The optimal angle ranges for detecting $N=126$ neutron-rich nuclides $^{204}$Pt, $^{203}$Ir, $^{202}$Os, and $^{201}$Re in the $^{136}$Xe + $^{208}$Pb reaction at the incident energy $E_{c.m.} = 526MeV$ are predicted. Our results suggest that the angle range $45^{\circ} \leqslant θ_{\mathrm{lab}} \leqslant 50^{\circ}$ is most favorable for detecting unknown N=126 isotones. Given the current difficulties in separating and identifying experimental MNT fragments, the results of this work could provide significant contributions to future experiments.

nucl-th↗

arXiv2023

Few-shot Class-incremental Learning for Cross-domain Disease Classification

The ability to incrementally learn new classes from limited samples is crucial to the development of artificial intelligence systems for real clinical application. Although existing incremental learning techniques have attempted to address this issue, they still struggle with only few labeled data, particularly when the samples are from varied domains. In this paper, we explore the cross-domain few-shot incremental learning (CDFSCIL) problem. CDFSCIL requires models to learn new classes from very few labeled samples incrementally, and the new classes may be vastly different from the target space. To counteract this difficulty, we propose a cross-domain enhancement constraint and cross-domain data augmentation method. Experiments on MedMNIST show that the classification performance of this method is better than other similar incremental learning methods.

cs.CV↗

arXiv2023

Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion

In recent years, denoising methods based on deep learning have achieved unparalleled performance at the cost of large computational complexity. In this work, we propose an Efficient Multi-stage Video Denoising algorithm, called EMVD, to drastically reduce the complexity while maintaining or even improving the performance. First, a fusion stage reduces the noise through a recursive combination of all past frames in the video. Then, a denoising stage removes the noise in the fused frame. Finally, a refinement stage restores the missing high frequency in the denoised frame. All stages operate on a transform-domain representation obtained by learnable and invertible linear operators which simultaneously increase accuracy and decrease complexity of the model. A single loss on the final output is sufficient for successful convergence, hence making EMVD easy to train. Experiments on real raw data demonstrate that EMVD outperforms the state of the art when complexity is constrained, and even remains competitive against methods whose complexities are several orders of magnitude higher. Further, the low complexity and memory requirements of EMVD enable real-time video denoising on commercial SoC in mobile devices.

eess.IV↗

arXiv2023

MGA: Medical generalist agent through text-guided knowledge transformation

Multi-modal representation methods have achieved advanced performance in medical applications by extracting more robust features from multi-domain data. However, existing methods usually need to train additional branches for downstream tasks, which may increase the model complexities in clinical applications as well as introduce additional human inductive bias. Besides, very few studies exploit the rich clinical knowledge embedded in clinical daily reports. To this end, we propose a novel medical generalist agent, MGA, that can address three kinds of common clinical tasks via clinical reports knowledge transformation. Unlike the existing methods, MGA can easily adapt to different tasks without specific downstream branches when their corresponding annotations are missing. More importantly, we are the first attempt to use medical professional language guidance as a transmission medium to guide the agent's behavior. The proposed method is implemented on four well-known X-ray open-source datasets, MIMIC-CXR, CheXpert, MIMIC-CXR-JPG, and MIMIC-CXR-MS. Promising results are obtained, which validate the effectiveness of our proposed MGA. Code is available at: https://github.com/SZUHvern/MGA

cs.CV↗

arXiv2023

Fixed-domain Posterior Contraction Rates for Spatial Gaussian Process Model with Nugget

Spatial Gaussian process regression models typically contain finite dimensional covariance parameters that need to be estimated from the data. We study the Bayesian estimation of covariance parameters including the nugget parameter in a general class of stationary covariance functions under fixed-domain asymptotics, which is theoretically challenging due to the increasingly strong dependence among spatial observations. We propose a novel adaptation of the Schwartz's consistency theorem for showing posterior contraction rates of the covariance parameters including the nugget. We derive a new polynomial evidence lower bound, and propose consistent higher-order quadratic variation estimators that satisfy concentration inequalities with exponentially small tails. Our Bayesian fixed-domain asymptotics theory leads to explicit posterior contraction rates for the microergodic and nugget parameters in the isotropic Matern covariance function under a general stratified sampling design. We verify our theory and the Bayesian predictive performance in simulation studies and an application to sea surface temperature data.

math.ST↗

arXiv2023

BiFeat: Supercharge GNN Training via Graph Feature Quantization

Graph Neural Networks (GNNs) is a promising approach for applications with nonEuclidean data. However, training GNNs on large scale graphs with hundreds of millions nodes is both resource and time consuming. Different from DNNs, GNNs usually have larger memory footprints, and thus the GPU memory capacity and PCIe bandwidth are the main resource bottlenecks in GNN training. To address this problem, we present BiFeat: a graph feature quantization methodology to accelerate GNN training by significantly reducing the memory footprint and PCIe bandwidth requirement so that GNNs can take full advantage of GPU computing capabilities. Our key insight is that unlike DNN, GNN is less prone to the information loss of input features caused by quantization. We identify the main accuracy impact factors in graph feature quantization and theoretically prove that BiFeat training converges to a network where the loss is within $ε$ of the optimal loss of uncompressed network. We perform extensive evaluation of BiFeat using several popular GNN models and datasets, including GraphSAGE on MAG240M, the largest public graph dataset. The results demonstrate that BiFeat achieves a compression ratio of more than 30 and improves GNN training speed by 200%-320% with marginal accuracy loss. In particular, BiFeat achieves a record by training GraphSAGE on MAG240M within one hour using only four GPUs.

cs.LG↗

arXiv2023

The need for multicomponent dust attenuation in modeling nebular emission: Constraints from SDSS-IV MaNGA

A fundamental assumption adopted in nearly every extragalactic emission-line study is that the attenuation of different emission lines can be described by a single attenuation curve. Here we show this assumption fails in many cases with important implications for derived results. We developed a new method to measure the differential nebular attenuation among three kinds of transitions: the Balmer lines of hydrogen, high-ionization transitions, and low-ionization transitions. This method bins the observed data in a multidimensional space spanned by attenuation-insensitive line ratios. Within each small bin, the variations in line ratios are mainly driven by the variations in the nebular attenuation. This allows us to measure the nebular attenuation using both forbidden lines and Balmer lines. We applied this method to a sample of 2.4 million star-forming spaxels from SDSS-IV MaNGA. We found that the attenuation of high ionization lines and Balmer lines can be well described by a single Fitzpatrick (1999) extinction curve with $R_V=3.1$. However, no single attenuation curve can simultaneously account for all three transitions. This strongly suggests that different lines have different effective attenuations, likely because spectroscopy at kiloparsec resolutions mixes multiple regions with different intrinsic line ratios and different levels of attenuation. As a result, the assumption that different lines follow the same attenuation curve breaks down. Using a single attenuation curve determined by Balmer lines to correct attenuation-sensitive forbidden line ratios could bias the nebular parameters derived by 0.06--0.25 dex at $A_V = 1$, depending on the details of the dust attenuation model. Observations of a statistically large sample of H II regions with high spatial resolutions and large spectral coverage are vital for improved modeling and deriving accurate corrections for this effect.

astro-ph.GA↗

arXiv2023

Experimental generation of polarization entanglement from spontaneous parametric down-conversion pumped by spatiotemporally highly incoherent light

The influence of pump coherence on the entanglement produced in spontaneous parametric down-conversion (SPDC) is important to understand, both from a fundamental perspective, and from a practical standpoint for controlled generation of entangled states. In this context, it is known that in the absence of postselection, the pump coherence in a given degree of freedom (DOF) imposes an upper limit on the generated entanglement in the same DOF. However, the cross-influence of the pump coherence on the generated entanglement in a different DOF is not well-understood. Here, we experimentally investigate the effect of a spatiotemporally highly-incoherent (STHI) light-emitting diode (LED) pump on the polarization entanglement generated in SPDC. Our quantum state tomography measurements using multimode collection fibers to reduce the influence of postselection yield a two-qubit state with a concurrence of 0.531+/-0.006 and a purity of 0.647+/-0.005, in excellent agreement with our theoretically predicted concurrence of 0.552 and purity of 0.652. Therefore, while the use of an STHI pump causes reduction in the entanglement and purity of the output polarization two-qubit state, the viability of SPDC with STHI pumps is nevertheless important for two reasons: (i) STHI sources are ubiquitous and available at a wider range of wavelengths than lasers, and (ii) the generated STHI polarization-entangled two-photon states could potentially be useful in long-distance quantum communication schemes due to their robustness to scattering.

quant-ph↗