arXiv subjects

Jian Li

Publications and source records attributed to Jian Li.

At least 343 records · Page 19Linked to original sources

arXiv2023

Efficient online cross-covariance monitoring with incremental SVD: An approach for the detection of emerging dependency patterns in IoT systems

The development of the manufacturing systems has made it increasingly necessary to monitor the data generated by multiple interconnected subsystems with rapid incoming of samples. Based on incremental Singular Value Decomposition (ISVD), we develop a general online monitoring approach for the relationship of data generated from two interconnected subsystems, where each subsystem produces big data streams with several variation patterns in normal working condition. When special situation happens and new associations occur, a very small amount of computation is sufficient to update the system status and compute the control statistics by using this approach. The proposed method reduces computational overhead and retains only a small number of pairs of possible dependent patterns at each step. The validation of the method through simulation studies and a case study on semiconductor manufacturing processes further supports its effectiveness.

eess.SY↗

arXiv2023

Joint cortical registration of geometry and function using semi-supervised learning

Brain surface-based image registration, an important component of brain image analysis, establishes spatial correspondence between cortical surfaces. Existing iterative and learning-based approaches focus on accurate registration of folding patterns of the cerebral cortex, and assume that geometry predicts function and thus functional areas will also be well aligned. However, structure/functional variability of anatomically corresponding areas across subjects has been widely reported. In this work, we introduce a learning-based cortical registration framework, JOSA, which jointly aligns folding patterns and functional maps while simultaneously learning an optimal atlas. We demonstrate that JOSA can substantially improve registration performance in both anatomical and functional domains over existing methods. By employing a semi-supervised training strategy, the proposed framework obviates the need for functional data during inference, enabling its use in broad neuroscientific domains where functional data may not be observed. The source code of JOSA will be released to the public at https://voxelmorph.net.

eess.IV↗

arXiv2023

The extraction of higher-order radial moments of nuclear charge density from muonic atom spectroscopy

Muonic atom transitions have been measured for almost all stable nuclei to extract nuclear structure properties, including nuclear charge radii and quadrupole moment. To investigate the possibilities of extracting higher-order radial moments of nuclear charge density %what kind of information of nuclear charge distribution can be extracted precisely from muonic atom spectroscopy, a theory-to-theory benchmark analysis based on a model-independent density distribution, i.e., the Fourier-Bessel series expansion instead of two-parameter Fermi distribution, is performed by taking $^{208}$Pb as an example, where nuclear charge density obtained from the relativistic continuum Hartree-Bogoliubov calculation is used as the benchmark. It is found that the extractions of the higher-order moments, i.e., the fourth and sixth moments in addition to the second moment are feasible with high accuracy. Moreover, the charge form factor in the low-$q$ region can also be well extracted.

nucl-th↗

arXiv2023

Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference

Latent Diffusion models (LDMs) have achieved remarkable results in synthesizing high-resolution images. However, the iterative sampling process is computationally intensive and leads to slow generation. Inspired by Consistency Models (song et al.), we propose Latent Consistency Models (LCMs), enabling swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion (rombach et al). Viewing the guided reverse diffusion process as solving an augmented probability flow ODE (PF-ODE), LCMs are designed to directly predict the solution of such ODE in latent space, mitigating the need for numerous iterations and allowing rapid, high-fidelity sampling. Efficiently distilled from pre-trained classifier-free guided diffusion models, a high-quality 768 x 768 2~4-step LCM takes only 32 A100 GPU hours for training. Furthermore, we introduce Latent Consistency Fine-tuning (LCF), a novel method that is tailored for fine-tuning LCMs on customized image datasets. Evaluation on the LAION-5B-Aesthetics dataset demonstrates that LCMs achieve state-of-the-art text-to-image generation performance with few-step inference. Project Page: https://latent-consistency-models.github.io/

cs.CV↗

arXiv2023

AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange

Graph Neural Networks (GNNs) have already been widely used in various graph mining tasks. However, recent works reveal that the learned weights (channels) in well-trained GNNs are highly redundant, which inevitably limits the performance of GNNs. Instead of removing these redundant channels for efficiency consideration, we aim to reactivate them to enlarge the representation capacity of GNNs for effective graph learning. In this paper, we propose to substitute these redundant channels with other informative channels to achieve this goal. We introduce a novel GNN learning framework named AKE-GNN, which performs the Adaptive Knowledge Exchange strategy among multiple graph views generated by graph augmentations. AKE-GNN first trains multiple GNNs each corresponding to one graph view to obtain informative channels. Then, AKE-GNN iteratively exchanges redundant channels in the weight parameter matrix of one GNN with informative channels of another GNN in a layer-wise manner. Additionally, existing GNNs can be seamlessly incorporated into our framework. AKE-GNN achieves superior performance compared with various baselines across a suite of experiments on node classification, link prediction, and graph classification. In particular, we conduct a series of experiments on 15 public benchmark datasets, 8 popular GNN models, and 3 graph tasks and show that AKE-GNN consistently outperforms existing popular GNN models and even their ensembles. Extensive ablation studies and analyses on knowledge exchange methods validate the effectiveness of AKE-GNN.

cs.LG↗

arXiv2023

Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation

Whittle index policy is a heuristic to the intractable restless multi-armed bandits (RMAB) problem. Although it is provably asymptotically optimal, finding Whittle indices remains difficult. In this paper, we present Neural-Q-Whittle, a Whittle index based Q-learning algorithm for RMAB with neural network function approximation, which is an example of nonlinear two-timescale stochastic approximation with Q-function values updated on a faster timescale and Whittle indices on a slower timescale. Despite the empirical success of deep Q-learning, the non-asymptotic convergence rate of Neural-Q-Whittle, which couples neural networks with two-timescale Q-learning largely remains unclear. This paper provides a finite-time analysis of Neural-Q-Whittle, where data are generated from a Markov chain, and Q-function is approximated by a ReLU neural network. Our analysis leverages a Lyapunov drift approach to capture the evolution of two coupled parameters, and the nonlinearity in value function approximation further requires us to characterize the approximation error. Combing these provide Neural-Q-Whittle with $\mathcal{O}(1/k^{2/3})$ convergence rate, where $k$ is the number of iterations.

cs.LG↗

arXiv2023

Native Conditional $i$SWAP Operation with Superconducting Artificial Atoms

Controlling the flow of quantum information is a fundamental task for quantum computers, which is unfeasible to realize on classical devices. Coherent devices which can process quantum states are thus required to route the quantum states that encode information. In this paper we demonstrate experimentally the smallest quantum transistor with a superconducting quantum processor which is composed of a collector qubit, an emitter qubit, and a coupler (transistor gate). The interaction strength between the collector and emitter qubits is controlled by the frequency and state of the coupler, effectively implementing a quantum switch. Through the coupler-state-dependent Heisenberg (inherent) interaction between the qubits, a single-step (native) conditional $i$SWAP operation can be applied. To this end, we find that it is important to take into consideration higher energy level for achieving a native and high-fidelity transistor operation. By reconstructing the Quantum Process Tomography, we obtain an operation fidelity of $92.36\%$ when the transistor gate is open ($i$SWAP implementation) and $95.23 \%$ in the case of closed gate (identity gate implementation). The architecture has strong potential in quantum information processing applications with superconducting qubits.

quant-ph↗

arXiv2023

Finite-nuclear-size effect in hydrogen-like ions with relativistic nuclear structure

The finite-nuclear-size (FNS) effect has a large contribution to the atomic spectral properties especially for heavy nuclei. By adopting the microscopic nuclear charge density distributions obtained from the relativistic continuum Hartree-Bogoliubov (RCHB) theory, we systematically investigate the FNS corrections to atomic energy levels and bound-electron $g$ factors of hydrogen-like ions with nuclear charge up to $118$. The comparison of the present numerical calculations with the predictions from empirical nuclear charge models, the non-relativistic Skyrme-Hartree-Fock calculations, and the results based on experimental charge densities indicate that both the nuclear charge radius and the detailed shape of charge density distribution play important roles in determining the FNS corrections. The variation of FNS corrections to energy levels and $g$ factors with respect to the nuclear charge are investigated for the lowest several bound states of hydrogen-like ions. It is shown that they both increase by orders of magnitude with increasing the nuclear charge, while the ratio between them has a relatively weak dependence on the nuclear charge. The FNS corrections to the $s_{1/2}$ and $p_{1/2}$ bound state energies from the RCHB calculations are generally in good agreement with the analytical estimations by Shabaev [J. Phys. B, 26, 1103 (1993)] based on the homogeneously charged sphere nuclear model, with the discrepancy indicating the distinct contribution of microscopic nuclear structure to the FNS effects.

nucl-th↗

arXiv2023

A Real-time Faint Space Debris Detector With Learning-based LCM

With the development of aerospace technology, the increasing population of space debris has posed a great threat to the safety of spacecraft. However, the low intensity of reflected light and high angular velocity of space debris impede the extraction. Besides, due to the limitations of the ground observation methods, small space debris can hardly be detected, making it necessary to enhance the spacecraft's capacity for space situational awareness (SSA). Considering that traditional methods have some defects in low-SNR target detection, such as low effectiveness and large time consumption, this paper proposes a method for low-SNR streak extraction based on local contrast and maximum likelihood estimation (MLE), which can detect space objects with SNR 2.0 efficiently. In the proposed algorithm, local contrast will be applied for crude classifications, which will return connected components as preliminary results, and then MLE will be performed to reconstruct the connected components of targets via orientated growth, further improving the precision. The algorithm has been verified with both simulated streaks and real star tracker images, and the average centroid error of the proposed algorithm is close to the state-of-the-art method like ODCC. At the same time, the algorithm in this paper has significant advantages in efficiency compared with ODCC. In conclusion, the algorithm in this paper is of high speed and precision, which guarantees its promising applications in the extraction of high dynamic targets.

cs.CV↗

arXiv2023

PaLM 2 Technical Report

We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report.

cs.CL↗

arXiv2023

Improving Differentiable Architecture Search via Self-Distillation

Differentiable Architecture Search (DARTS) is a simple yet efficient Neural Architecture Search (NAS) method. During the search stage, DARTS trains a supernet by jointly optimizing architecture parameters and network parameters. During the evaluation stage, DARTS discretizes the supernet to derive the optimal architecture based on architecture parameters. However, recent research has shown that during the training process, the supernet tends to converge towards sharp minima rather than flat minima. This is evidenced by the higher sharpness of the loss landscape of the supernet, which ultimately leads to a performance gap between the supernet and the optimal architecture. In this paper, we propose Self-Distillation Differentiable Neural Architecture Search (SD-DARTS) to alleviate the discretization gap. We utilize self-distillation to distill knowledge from previous steps of the supernet to guide its training in the current step, effectively reducing the sharpness of the supernet's loss and bridging the performance gap between the supernet and the optimal architecture. Furthermore, we introduce the concept of voting teachers, where multiple previous supernets are selected as teachers, and their output probabilities are aggregated through voting to obtain the final teacher prediction. Experimental results on real datasets demonstrate the advantages of our novel self-distillation-based NAS method compared to state-of-the-art alternatives.

cs.CV↗

arXiv2023

Calculation of microscopic nuclear level densities based on covariant density functional theory

A microscopic method for calculating nuclear level density (NLD) based on the covariant density functional theory (CDFT) is developed. The particle-hole state density is calculated by combinatorial method using the single-particle levels schemes obtained from the CDFT. Then the level densities are obtained by taking into account collective effects such as vibration and rotation. Our results are compared with those from other NLD models, including phenomenological, microstatistical, and non-relativistic HFB combinatorial models. The comparison suggests that the general trends among these models are basically the same, except for some deviations from different NLD models. In addition, the NLDs of the CDFT combinatorial method with normalization are compared with experimental data, including the observed cumulative number of levels at low excitation energy and the measured NLDs. Compared with the existing experimental data, the CDFT combinatorial method can give reasonable results.

nucl-th↗

arXiv2023

Characterizations of distality via weak equicontinuity

For an infinite discrete group $G$ acting on a compact metric space $X$, we introduce several weak versions of equicontinuity along subsets of $G$ and show that if a minimal system $(X,G)$ admits an invariant measure then $(X,G)$ is distal if and only if it is pairwise IP$^*$-equicontinuous; if the product system $(X\times X,G)$ of a minimal system $(X,G)$ has a dense set of minimal points, then $(X,G)$ is distal if and only if it is pairwise IP$^*$-equicontinuous if and only if it is pairwise central$^*$-equicontinuous; if $(X,G)$ is a minimal system with $G$ being abelian, then $(X,G)$ is a system of order $\infty$ if and only if it is pairwise FIP$^*$-equicontinuous.

math.DS↗

arXiv2023

Semi-supervised Vector-valued Learning: Improved Bounds and Algorithms

Vector-valued learning, where the output space admits a vector-valued structure, is an important problem that covers a broad family of important domains, e.g. multi-task learning and transfer learning. Using local Rademacher complexity and unlabeled data, we derive novel semi-supervised excess risk bounds for general vector-valued learning from both kernel perspective and linear perspective. The derived bounds are much sharper than existing ones and the convergence rates are improved from the square root of labeled sample size to the square root of total sample size or directly dependent on labeled sample size. Motivated by our theoretical analysis, we propose a general semi-supervised algorithm for efficiently learning vector-valued functions, incorporating both local Rademacher complexity and Laplacian regularization. Extensive experimental results illustrate the proposed algorithm significantly outperforms the compared methods, which coincides with our theoretical findings.

cs.LG↗

arXiv2023

Non-Hermitian Haldane-Hubbard model: Effective description of one- and two-body dissipation

Using numerically exact diagonalization, we study the correlated Haldane-Hubbard model in the presence of dissipation. Such dissipation can be modeled at short times by the dynamics governed by an effective non-Hermitian Hamiltonian, of which we present a full characterization. If the dissipation corresponds to a two-body loss, the repulsive interaction of the effective Hamiltonian acquires an imaginary component. A competition between the formation of a charge-ordered Mott insulator state and a topological insulator ensues, but with the non-Hermitian contribution aiding in stabilizing the topologically non-trivial regime, delaying the onset of the formation of a local order parameter. Lastly, we analyze the robustness of the ordered phase by following the full dissipative many-body real-time dynamics. An exponentially fast melting of the charge order occurs, whose characteristic rate is roughly independent of the interaction strength, for the case of one-body dissipation.

cond-mat.str-el↗

arXiv2023

IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making

Market making (MM) has attracted significant attention in financial trading owing to its essential function in ensuring market liquidity. With strong capabilities in sequential decision-making, Reinforcement Learning (RL) technology has achieved remarkable success in quantitative trading. Nonetheless, most existing RL-based MM methods focus on optimizing single-price level strategies which fail at frequent order cancellations and loss of queue priority. Strategies involving multiple price levels align better with actual trading scenarios. However, given the complexity that multi-price level strategies involves a comprehensive trading action space, the challenge of effectively training profitable RL agents for MM persists. Inspired by the efficient workflow of professional human market makers, we propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions to develop multi-price level MM strategies efficiently. The framework start with introducing effective state and action representations adept at encoding information about multi-price level orders. Furthermore, IMM integrates a representation learning unit capable of capturing both short- and long-term market trends to mitigate adverse selection risk. Subsequently, IMM formulates an expert strategy based on signals and trains the agent through the integration of RL and imitation learning techniques, leading to efficient learning. Extensive experimental results on four real-world market datasets demonstrate that IMM outperforms current RL-based market making strategies in terms of several financial criteria. The findings of the ablation study substantiate the effectiveness of the model components.

cs.LG↗

arXiv2023

Rethinking Mobile Block for Efficient Attention-based Models

This paper focuses on developing modern, efficient, lightweight models for dense predictions while trading off parameters, FLOPs, and performance. Inverted Residual Block (IRB) serves as the infrastructure for lightweight CNNs, but no counterpart has been recognized by attention-based studies. This work rethinks lightweight infrastructure from efficient IRB and effective components of Transformer from a unified perspective, extending CNN-based IRB to attention-based models and abstracting a one-residual Meta Mobile Block (MMB) for lightweight model design. Following simple but effective design criterion, we deduce a modern Inverted Residual Mobile Block (iRMB) and build a ResNet-like Efficient MOdel (EMO) with only iRMB for down-stream tasks. Extensive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, e.g., EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass equal-order CNN-/Attention-based models, while trading-off the parameter, efficiency, and accuracy well: running 2.8-4.0x faster than EdgeNeXt on iPhone14.

cs.CV↗

arXiv2023

A branch-cut-and-price algorithm for a dial-a-ride problem with minimum disease-transmission risk

This paper investigates a variant of the dial-a-ride problem (DARP), namely Risk-aware DARP (RDARP). Our RDARP extends the DARP by (1) minimizing a weighted sum of travel cost and disease-transmission risk exposure for onboard passengers and (2) introducing a maximum cumulative exposure risk constraint for each vehicle to ensure a trip with lower external exposure. Both extensions require that the risk exposure of each onboard passenger is propagated in a minimized fashion while satisfying other existing constraints of the DARP. To fully describe the RDARP, we first provide a three-index arc-based formulation and reformulate the RDARP into an equivalent min-max trip-based formulation, which is solved by an exact Branch-Cut-and-Price (BCP) algorithm. For each node in the branching tree, we adopt the column generation method by decomposing the problem into a master problem and a subproblem. The latter takes the form of an elementary shortest path problem with resource constraints and minimized maximum risk (ESPPRCMMR). To solve the ESPPRCMMR efficiently, we develop a new labeling algorithm and establish families of resource extension functions in compliance with the risk-related resources. We adopt a real-world paratransit trip dataset to generate the RDARP instances ranging from 17 to 55 heterogeneous passengers with 3 to 13 vehicles during the morning and afternoon periods. Computational results show that our BCP algorithm can optimally solve all small- and medium-sized instances (32 or fewer passengers) within 5 minutes. For the large instances (39 to 55 passengers), our BCP algorithm can solve 23 of 30 instances optimally within a time limit of one hour.

math.OC↗