arXiv subjects

Ying Zhang

Publications and source records attributed to Ying Zhang.

At least 379 records · Page 21Linked to original sources

arXiv2022

GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records

There is an increasing interest in developing artificial intelligence (AI) systems to process and interpret electronic health records (EHRs). Natural language processing (NLP) powered by pretrained language models is the key technology for medical AI systems utilizing clinical narratives. However, there are few clinical language models, the largest of which trained in the clinical domain is comparatively small at 110 million parameters (compared with billions of parameters in the general domain). It is not clear how large clinical language models with billions of parameters can help medical AI systems utilize unstructured EHRs. In this study, we develop from scratch a large clinical language model - GatorTron - using >90 billion words of text (including >82 billion words of de-identified clinical text) and systematically evaluate it on 5 clinical NLP tasks including clinical concept extraction, medical relation extraction, semantic textual similarity, natural language inference (NLI), and medical question answering (MQA). We examine how (1) scaling up the number of parameters and (2) scaling up the size of the training data could benefit these NLP tasks. GatorTron models scale up the clinical language model from 110 million to 8.9 billion parameters and improve 5 clinical NLP tasks (e.g., 9.6% and 9.5% improvement in accuracy for NLI and MQA), which can be applied to medical AI systems to improve healthcare delivery. The GatorTron models are publicly available at: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/gatortron_og.

cs.CL↗

arXiv2022

BadPrompt: Backdoor Attacks on Continuous Prompts

The prompt-based learning paradigm has gained much research attention recently. It has achieved state-of-the-art performance on several NLP tasks, especially in the few-shot scenarios. While steering the downstream tasks, few works have been reported to investigate the security problems of the prompt-based models. In this paper, we conduct the first study on the vulnerability of the continuous prompt learning algorithm to backdoor attacks. We observe that the few-shot scenarios have posed a great challenge to backdoor attacks on the prompt-based models, limiting the usability of existing NLP backdoor methods. To address this challenge, we propose BadPrompt, a lightweight and task-adaptive algorithm, to backdoor attack continuous prompts. Specially, BadPrompt first generates candidate triggers which are indicative for predicting the targeted label and dissimilar to the samples of the non-targeted labels. Then, it automatically selects the most effective and invisible trigger for each sample with an adaptive trigger optimization algorithm. We evaluate the performance of BadPrompt on five datasets and two continuous prompt models. The results exhibit the abilities of BadPrompt to effectively attack continuous prompts while maintaining high performance on the clean test sets, outperforming the baseline models by a large margin. The source code of BadPrompt is publicly available at https://github.com/papersPapers/BadPrompt.

cs.CL↗

arXiv2022

Joint Secure Communication and Radar Beamforming: A Secrecy-Estimation Rate-Based Design

This paper considers transmit beamforming in dual-function radar-communication (DFRC) system, where a DFRC transmitter simultaneously communicates with a communication user and detects a malicious target with the same waveform. Since the waveform is embedded with information, the information is risked to be intercepted by the target. To address this problem, physical-layer security technique is exploited. By using secrecy rate and estimation rate as performance measure for communication and radar, respectively, three secrecy rate maximization (SRM) problems are formulated, including the SRM with and without artificial noise (AN), and robust SRM. For the SRM beamforming, we prove that the optimal beamformer can be computed in closed form. For the AN-aided SRM, by leveraging alternating optimization similar closed-form solution is obtained for the beamformer and the AN covariance matrix. Finally, the imperfect CSI of the target is also considered under the premise of a moment-based random phase-error model on the direction of arrival at the target. Simulation results demonstrate the efficacy and robustness of the proposed designs.

eess.SP↗

arXiv2022

Efficiently Answering Quality Constrained Shortest Distance Queries in Large Graphs

The shortest-path distance is a fundamental concept in graph analytics and has been extensively studied in the literature. In many real-world applications, quality constraints are naturally associated with edges in the graphs and finding the shortest distance between two vertices $s$ and $t$ along only valid edges (i.e., edges that satisfy a given quality constraint) is also critical. In this paper, we investigate this novel and important problem of quality constraint shortest distance queries. We propose an efficient index structure based on 2-hop labeling approaches. Supported by a path dominance relationship incorporating both quality and length information, we demonstrate the minimal property of the new index. An efficient query processing algorithm is also developed. Extensive experimental studies over real-life datasets demonstrates efficiency and effectiveness of our techniques.

cs.DB↗

arXiv2022

Resolving the spurious-state problem in the Dirac equation with finite difference method

To solve the Dirac equation with the finite difference method, one has to face up to the spurious-state problem due to the fermion doubling problem when using the conventional central difference formula to calculate the first-order derivative on the equal interval lattices. This problem is resolved by replacing the central difference formula with the asymmetric difference formula, i.e., the backward or forward difference formula. To guarantee the hermitian of the Hamiltonian matrix, the backward and forward difference formula should be used alternatively according to the parity of the wavefunction. This provides a simple and efficient numerical prescription to solve various relativistic problems in the microscopic world.

nucl-th↗

arXiv2022

MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph Completion

Multimodal knowledge graph completion (MKGC) aims to predict missing entities in MKGs. Previous works usually share relation representation across modalities. This results in mutual interference between modalities during training, since for a pair of entities, the relation from one modality probably contradicts that from another modality. Furthermore, making a unified prediction based on the shared relation representation treats the input in different modalities equally, while their importance to the MKGC task should be different. In this paper, we propose MoSE, a Modality Split representation learning and Ensemble inference framework for MKGC. Specifically, in the training phase, we learn modality-split relation embeddings for each modality instead of a single modality-shared one, which alleviates the modality interference. Based on these embeddings, in the inference phase, we first make modality-split predictions and then exploit various ensemble methods to combine the predictions with different weights, which models the modality importance dynamically. Experimental results on three KG datasets show that MoSE outperforms state-of-the-art MKGC methods. Codes are available at https://github.com/OreOZhao/MoSE4MKGC.

cs.CL↗

arXiv2022

Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization

In this paper, we are concerned with a non-asymptotic analysis of sampling algorithms used in nonconvex optimization. In particular, we obtain non-asymptotic estimates in Wasserstein-1 and Wasserstein-2 distances for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). In addition, the aforementioned Wasserstein-2 convergence result can be applied to establish a non-asymptotic error bound for the expected excess risk. Crucially, these results are obtained under a local Lipschitz condition and a local dissipativity condition where we remove the uniform dependence in the data stream. We illustrate the importance of this relaxation by presenting examples from variational inference and from index tracking optimization.

math.ST↗

arXiv2022

Global large solution for the tropical climate model with diffusion

This paper studies the d-dimensional (d=2,3) tropical climate model with only the dissipation of the first baroclinic model of the velocity ($-ηΔv$). By choosing a class of special initial data $(u_0,v_0,θ_0)$ whose $H^s(\mathbb{R}^d)$ norm can be arbitrarily large, we obtain the global smooth solution of d-dimensional tropical climate model.

math.AP↗

arXiv2022

TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs

We propose TopoOpt, a novel direct-connect fabric for deep neural network (DNN) training workloads. TopoOpt co-optimizes the distributed training process across three dimensions: computation, communication, and network topology. We demonstrate the mutability of AllReduce traffic, and leverage this property to construct efficient network topologies for DNN training jobs. TopoOpt then uses an alternating optimization technique and a group theory-inspired algorithm called TotientPerms to find the best network topology and routing plan, together with a parallelization strategy. We build a fully functional 12-node direct-connect prototype with remote direct memory access (RDMA) forwarding at 100 Gbps. Large-scale simulations on real distributed training models show that, compared to similar-cost Fat-Tree interconnects, TopoOpt reduces DNN training time by up to 3.4x.

cs.NI↗

arXiv2022

Overcoming Language Priors in Visual Question Answering via Distinguishing Superficially Similar Instances

Despite the great progress of Visual Question Answering (VQA), current VQA models heavily rely on the superficial correlation between the question type and its corresponding frequent answers (i.e., language priors) to make predictions, without really understanding the input. In this work, we define the training instances with the same question type but different answers as \textit{superficially similar instances}, and attribute the language priors to the confusion of VQA model on such instances. To solve this problem, we propose a novel training framework that explicitly encourages the VQA model to distinguish between the superficially similar instances. Specifically, for each training instance, we first construct a set that contains its superficially similar counterparts. Then we exploit the proposed distinguishing module to increase the distance between the instance and its counterparts in the answer space. In this way, the VQA model is forced to further focus on the other parts of the input beyond the question type, which helps to overcome the language priors. Experimental results show that our method achieves the state-of-the-art performance on VQA-CP v2. Codes are available at \href{https://github.com/wyk-nku/Distinguishing-VQA.git}{Distinguishing-VQA}.

cs.CL↗

arXiv2022

Multi-grained Label Refinement Network with Dependency Structures for Joint Intent Detection and Slot Filling

Slot filling and intent detection are two fundamental tasks in the field of natural language understanding. Due to the strong correlation between these two tasks, previous studies make efforts on modeling them with multi-task learning or designing feature interaction modules to improve the performance of each task. However, none of the existing approaches consider the relevance between the structural information of sentences and the label semantics of two tasks. The intent and semantic components of a utterance are dependent on the syntactic elements of a sentence. In this paper, we investigate a multi-grained label refinement network, which utilizes dependency structures and label semantic embeddings. Considering to enhance syntactic representations, we introduce the dependency structures of sentences into our model by graph attention layer. To capture the semantic dependency between the syntactic information and task labels, we combine the task specific features with corresponding label embeddings by attention mechanism. The experimental results demonstrate that our model achieves the competitive performance on two public datasets.

cs.CL↗

arXiv2022

Marginal Regression on Transient State Occupation Probabilities with Clustered Multistate Process Data

Clustered multistate process data are commonly encountered in multicenter observational studies and clinical trials. A clinically important estimand with such data is the marginal probability of being in a particular transient state as a function of time. However, there is currently no method for nonparametric marginal regression analysis of these probabilities with clustered multistate process data. To address this problem, we propose a weighted functional generalized estimating equations approach which does not impose Markov assumptions or assumptions regarding the structure of the within-cluster dependence, and allows for informative cluster size (ICS). The asymptotic properties of the proposed estimators for the functional regression coefficients are rigorously established and a nonparametric hypothesis testing procedure for covariate effects is proposed. Simulation studies show that the proposed method performs well even with a small number of clusters, and that ignoring the within-cluster dependence and the ICS leads to invalid inferences. The proposed method is used to analyze data from a multicenter clinical trial on recurrent or metastatic squamous-cell carcinoma of the head and neck with a stratified randomization design.

stat.ME↗

arXiv2022

Towards Higher-order Topological Consistency for Unsupervised Network Alignment

Network alignment task, which aims to identify corresponding nodes in different networks, is of great significance for many subsequent applications. Without the need for labeled anchor links, unsupervised alignment methods have been attracting more and more attention. However, the topological consistency assumptions defined by existing methods are generally low-order and less accurate because only the edge-indiscriminative topological pattern is considered, which is especially risky in an unsupervised setting. To reposition the focus of the alignment process from low-order to higher-order topological consistency, in this paper, we propose a fully unsupervised network alignment framework named HTC. The proposed higher-order topological consistency is formulated based on edge orbits, which is merged into the information aggregation process of a graph convolutional network so that the alignment consistencies are transformed into the similarity of node embeddings. Furthermore, the encoder is trained to be multi-orbit-aware and then be refined to identify more trusted anchor links. Node correspondence is comprehensively evaluated by integrating all different orders of consistency. {In addition to sound theoretical analysis, the superiority of the proposed method is also empirically demonstrated through extensive experimental evaluation. On three pairs of real-world datasets and two pairs of synthetic datasets, our HTC consistently outperforms a wide variety of unsupervised and supervised methods with the least or comparable time consumption. It also exhibits robustness to structural noise as a result of our multi-orbit-aware training mechanism.

cs.LG↗

arXiv2022

Semi-supervised segmentation of tooth from 3D Scanned Dental Arches

Teeth segmentation is an important topic in dental restorations that is essential for crown generation, diagnosis, and treatment planning. In the dental field, the variability of input data is high and there are no publicly available 3D dental arch datasets. Although there has been improvement in the field provided by recent deep learning architectures on 3D data, there still exists some problems such as properly identifying missing teeth in an arch. We propose to use spectral clustering as a self-supervisory signal to joint-train neural networks for segmentation of 3D arches. Our approach is motivated by the observation that K-means clustering provides cues to capture margin lines related to human perception. The main idea is to automatically generate training data by decomposing unlabeled 3D arches into segments relying solely on geometric information. The network is then trained using a joint loss that combines a supervised loss of annotated input and a self-supervised loss of non-labeled input. Our collected data has a variety of arches including arches with missing teeth. Our experimental results show improvement over the fully supervised state-of-the-art MeshSegNet when using semi-supervised learning. Finally, we contribute code and a dataset.

cs.CV↗

arXiv2022

Farey tree and devil's staircase of frequency-locked breathers in ultrafast lasers

Nonlinear systems with two competing frequencies show locking or resonances. In lasers, the two interacting frequencies can be the cavity repetition rate and a frequency externally applied to the system. Conversely, the excitation of breather oscillations in lasers naturally triggers a second characteristic frequency in the system, therefore showing competition between the cavity repetition rate and the breathing frequency. Yet, the link between breathing solitons and frequency locking is missing. Here we demonstrate frequency locking at Farey fractions of a breather laser. The winding numbers show the hierarchy of the Farey tree and the structure of a devil's staircase. Numerical simulations of a discrete laser model confirm the experimental findings. The breather laser may therefore serve as a simple model system to explore universal synchronization dynamics of nonlinear systems. The locked breathing frequencies feature high signal-to-noise ratio and can give rise to dense radio-frequency combs, which are attractive for applications.

physics.optics↗

arXiv2022

Testing for Treatment Effect Twice Using Internal and External Controls in Clinical Trials

Leveraging external controls -- relevant individual patient data under control from external trials or real-world data -- has the potential to reduce the cost of randomized controlled trials (RCTs) while increasing the proportion of trial patients given access to novel treatments. However, due to lack of randomization, RCT patients and external controls may differ with respect to covariates that may or may not have been measured. Hence, after controlling for measured covariates, for instance by matching, testing for treatment effect using external controls may still be subject to unmeasured biases. In this paper, we propose a sensitivity analysis approach to quantify the magnitude of unmeasured bias that would be needed to alter the study conclusion that presumed no unmeasured biases are introduced by employing external controls. Whether leveraging external controls increases power or not depends on the interplay between sample sizes and the magnitude of treatment effect and unmeasured biases, which may be difficult to anticipate. This motivates a combined testing procedure that performs two highly correlated analyses, one with and one without external controls, with a small correction for multiple testing using the joint distribution of the two test statistics. The combined test provides a new method of sensitivity analysis designed for data fusion problems, which anchors at the unbiased analysis based on RCT only and spends a small proportion of the type I error to also test using the external controls. In this way, if leveraging external controls increases power, the power gain compared to the analysis based on RCT only can be substantial; if not, the power loss is small. The proposed method is evaluated in theory and power calculations, and applied to a real trial.

stat.ME↗

arXiv2022

AFD Types Sparse Representations vs. the Karhunen-Loeve Expansion for Decomposing Stochastic Processes

This article introduces adaptive Fourier decomposition (AFD) type methods, emphasizing on those that can be applied to stochastic processes and random fields, mainly including stochastic adaptive Fourier decomposition and stochastic pre-orthogonal adaptive Fourier decomposition. We establish their algorithms based on the covariant function and prove that they enjoy the same convergence rate as the Karhunen-Loève (KL) decomposition. The AFD type methods are compared with the KL decomposition. In contrast with the latter, the AFD type methods do not need to compute eigenvalues and eigenfunctions of the kernel-integral operator induced by the covariance function, and thus considerably reduce the computation complexity and computer consumes. Various kinds of dictionaries offer AFD flexibility to solve problems of a great variety, including different types of deterministic and stochastic equations. The conducted experiments show, besides the numerical convenience and fast convergence, that the AFD type decompositions outperform the KL type in describing local details, in spite of the proven global optimality of the latter.

math.ST↗

arXiv2022

Towards Real-Time Counting Shortest Cycles on Dynamic Graphs: A Hub Labeling Approach

With the ever-increasing prevalence of graph data in a wide spectrum of applications, it becomes essential to analyze structural trends in dynamic graphs on a continual basis. The shortest cycle is a fundamental pattern in graph analytics. In this paper, we investigate the problem of shortest cycle counting for a given vertex in dynamic graphs in light of its applicability to problems such as fraud detection. To address such queries efficiently, we propose a 2-hop labeling based algorithm called Counting Shortest Cycle (CSC for short). Additionally, techniques for dynamically updating the CSC index are explored. Comprehensive experiments are conducted to demonstrate the efficiency and effectiveness of our method. In particular, CSC enables query evaluation in a few hundreds of microseconds for graphs with millions of edges, and improves query efficiency by two orders of magnitude when compared to the baseline solutions. Also, the update algorithm could efficiently cope with edge insertions (deletions).

cs.SI↗