arXiv Science⌕ Search

arXiv subjects

Yuan Zhang

Publications and source records attributed to Yuan Zhang.

At least 361 records · Page 20Linked to original sources
arXiv2021

Few-shot Intent Classification and Slot Filling with Retrieved Examples

Few-shot learning arises in important practical scenarios, such as when a natural language understanding system needs to learn new semantic labels for an emerging, resource-scarce domain. In this paper, we explore retrieval-based methods for intent classification and slot filling tasks in few-shot settings. Retrieval-based methods make predictions based on labeled examples in the retrieval index that are similar to the input, and thus can adapt to new domains simply by changing the index without having to retrain the model. However, it is non-trivial to apply such methods on tasks with a complex label space like slot filling. To this end, we propose a span-level retrieval method that learns similar contextualized representations for spans with the same label via a novel batch-softmax objective. At inference time, we use the labels of the retrieved spans to construct the final structure with the highest aggregated score. Our method outperforms previous systems in various few-shot settings on the CLINC and SNIPS benchmarks.

cs.CL↗
arXiv2021

Observability Robustness under Sensor Failures: a Computational Perspective

This paper studies the robustness of observability of a linear time-invariant system under sensor failures from a computational perspective. To be precise, the problem of determining the minimum number of sensors whose removal can destroy system observability, as well as the problem of determining the minimum number of state variables that need to be prevented from being directly measured by the existing sensors to destroy observability, is investigated. The first one is closely related to the ability of unique state reconstruction of a system under adversarial sensor attacks, and the dual of both problems are in the opposite direction of the well-studied minimal controllability problems. It is proven that all these problems are NP-hard, both for a numerical system and a structured system, even restricted to some special cases. It is also shown that the first problems both for a numerical system and a structured one share a cardinality-constrained submodular minimization structure, for which there is no known constant or logarithmic factor approximation in general. On the other hand, for the first two problems, under a reasonable assumption often met by practical systems, that the eigenvalue geometric multiplicities of the numerical systems or the matching deficiencies of the structured systems are bounded by a constant, by levering the rank-one update property of the involved rank function, it is possible to obtain the corresponding optimal solutions by traversing a subset of the feasible solutions. We show such a method has polynomial time complexity in the system dimensions and the number of sensors.

math.OC↗
arXiv2021

Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System

Designing task-oriented dialogue systems is a challenging research topic, since it needs not only to generate utterances fulfilling user requests but also to guarantee the comprehensibility. Many previous works trained end-to-end (E2E) models with supervised learning (SL), however, the bias in annotated system utterances remains as a bottleneck. Reinforcement learning (RL) deals with the problem through using non-differentiable evaluation metrics (e.g., the success rate) as rewards. Nonetheless, existing works with RL showed that the comprehensibility of generated system utterances could be corrupted when improving the performance on fulfilling user requests. In our work, we (1) propose modelling the hierarchical structure between dialogue policy and natural language generator (NLG) with the option framework, called HDNO, where the latent dialogue act is applied to avoid designing specific dialogue act representations; (2) train HDNO via hierarchical reinforcement learning (HRL), as well as suggest the asynchronous updates between dialogue policy and NLG during training to theoretically guarantee their convergence to a local maximizer; and (3) propose using a discriminator modelled with language models as an additional reward to further improve the comprehensibility. We test HDNO on MultiWoz 2.0 and MultiWoz 2.1, the datasets on multi-domain dialogues, in comparison with word-level E2E model trained with RL, LaRL and HDSA, showing improvements on the performance evaluated by automatic evaluation metrics and human evaluation. Finally, we demonstrate the semantic meanings of latent dialogue acts to show the explanability for HDNO.

cs.CL↗
arXiv2021

Investigating the integrate and fire model as the limit of a random discharge model: a stochastic analysis perspective

In the mean field integrate-and-fire model, the dynamics of a typical neuron within a large network is modeled as a diffusion-jump stochastic process whose jump takes place once the voltage reaches a threshold. In this work, the main goal is to establish the convergence relationship between the regularized process and the original one where in the regularized process, the jump mechanism is replaced by a Poisson dynamic, and jump intensity within the classically forbidden domain goes to infinity as the regularization parameter vanishes. On the macroscopic level, the Fokker-Planck equation for the process with random discharges (i.e. Poisson jumps) are defined on the whole space, while the equation for the limit process is on the half space. However, with the iteration scheme, the difficulty due to the domain differences has been greatly mitigated and the convergence for the stochastic process and the firing rates can be established. Moreover, we find a polynomial-order convergence for the distribution by a re-normalization argument in probability theory. Finally, by numerical experiments, we quantitatively explore the rate and the asymptotic behavior of the convergence for both linear and nonlinear models.

math.PR↗
arXiv2021

Structural Controllability of Networked Relative Coupling Systems

This paper studies the controllability of networked relative coupling systems (NRCSs), in which subsystems are of fixed high-order linear dynamics and coupled through relative variables depending on their neighbors, from a structural perspective. The purpose is to explore conditions for subsystem dynamics and network topologies under which for almost all weights of the subsystem interaction links, the corresponding numerical NRCSs are controllable, which is called structurally controllable. Three types of subsystem interaction fashions are considered: 1) each subsystem is single-input-single-output (SISO), 2) each subsystem is multiple-input-multiple-output (MIMO), and the weights for all channels between two subsystems are identical, and 3) each subsystem is MIMO, but different channels between two subsystems can be weighted differently. {We show that all parameter-dependent modes of the NRCSs are generically controllable under some necessary connectivity conditions. We then derive necessary and/or sufficient conditions for structural controllability depending on subsystem dynamics and network topologies' connectivity in a decoupled form for all the three interaction fashions.} We also extend our results to handle certain subsystem heterogeneities and demonstrate their direct applications on some practical systems, including the mass-spring-damper system and the power network.

eess.SY↗
arXiv2021

Hölder estimates for the $\bar\partial$ problem for $(p,q)$ forms on product domains

The purpose of this paper is to study Hölder estimates for the $\bar\partial$ problem for $(p,q)$ forms on products of general planar domains. As indicated by an example of Stein and Kerzman, solutions to the $\bar\partial$ problem on product domains in $\mathbb C^n (n\ge 2)$ does not gain regularity in Hölder spaces. Making use of an integral representation of Nijenhuis and Woolf, we show that given a $\bar\partial$-closed $(p,q)$ form with $C^{k,α}$ components, $0\le p\le n, 1\le q\le n$, $k\in \mathbb Z^+\cup \{0\}, 0<α\le 1$, there is a $C^{k, α'}$ solution to the $\bar\partial$ problem on product domains for any $0<α'<α$ with the desired Hölder estimate.

math.CV↗
arXiv2021

Some Rigorous Results on the Phase Transition of Finitary Random Interlacement

In this paper, we show several rigorous results on the phase transition of Finitary Random Interlacement (FRI). For the high intensity regime, we show the existence of a critical fiber length, and give the exact asymptotic of it as intensity goes to infinity. At the same time, our result for the low intensity regime proves the global existence of a non-trivial phase transition with respect to the system intensity.

math.PR↗
arXiv2021

Edgeworth expansions for network moments

Network method of moments arXiv:1202.5101 is an important tool for nonparametric network inference. However, there has been little investigation on accurate descriptions of the sampling distributions of network moment statistics. In this paper, we present the first higher-order accurate approximation to the sampling CDF of a studentized network moment by Edgeworth expansion. In sharp contrast to classical literature on noiseless U-statistics, we show that the Edgeworth expansion of a network moment statistic as a noisy U-statistic can achieve higher-order accuracy without non-lattice or smoothness assumptions but just requiring weak regularity conditions. Behind this result is our surprising discovery that the two typically-hated factors in network analysis, namely, sparsity and edge-wise observational errors, jointly play a blessing role, contributing a crucial self-smoothing effect in the network moment statistic and making it analytically tractable. Our assumptions match the minimum requirements in related literature. For sparse networks, our theory shows a simple normal approximation achieves a gradually depreciating Berry-Esseen bound as the network becomes sparser. This result also refines the best previous theoretical result. For practitioners, our empirical Edgeworth expansion is highly accurate, fast and easy to implement. We demonstrate the clear advantage of our method by comprehensive simulation studies. We showcase three applications of our results in network inference. We prove, to our knowledge, the first theoretical guarantee of higher-order accuracy for some network bootstrap schemes, and moreover, the first theoretical guidance for selecting the sub-sample size for network sub-sampling. We also derive one-sample test and Cornish-Fisher confidence interval for a given moment with higher-order accurate controls of confidence level and type I error, respectively.

math.ST↗
arXiv2020

End-to-End Chinese Parsing Exploiting Lexicons

Chinese parsing has traditionally been solved by three pipeline systems including word-segmentation, part-of-speech tagging and dependency parsing modules. In this paper, we propose an end-to-end Chinese parsing model based on character inputs which jointly learns to output word segmentation, part-of-speech tags and dependency structures. In particular, our parsing model relies on word-char graph attention networks, which can enrich the character inputs with external word knowledge. Experiments on three Chinese parsing benchmark datasets show the effectiveness of our models, achieving the state-of-the-art results on end-to-end Chinese parsing.

cs.CL↗
arXiv2020

On (non-)monotonicity and phase diagram of finitary random interlacement

In this paper, we study the evolution of a Finitary Random Interlacement (FRI) with respect to the expected length of each fiber. In contrast to the previously proved phase transition between sufficiently large and small fiber length, we show that for $d=3,4$, FRI is NOT stochastically monotone as fiber length increasing. At the same time, numerical evidences still strongly support the existence of a unique and sharp phase transition on the existence of a unique infinite cluster, while the critical value for phase transition is estimated to be an inversely proportional function with respect to the system intensity.

math.PR↗
arXiv2020

Percolation for the Finitary Random interlacements

In this paper, we prove a phase transition in the connectivity of Finitary Random interlacements $\mathcal{FI}^{u,T}$ in $\mathbb{Z}^d$, with respect to the average stopping time. For each $u>0$, with probability one $\mathcal{FI}^{u,T}$ has no infinite connected component for all sufficiently small $T>0$, and a unique infinite connected component for all sufficiently large $T<\infty$. This answers a question of Bowen in the special case of $\mathbb{Z}^d$.

math.PR↗
arXiv2020

Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective

Graph convolutional networks (GCNs) have achieved promising performance on various graph-based tasks. However they suffer from over-smoothing when stacking more layers. In this paper, we present a quantitative study on this observation and develop novel insights towards the deeper GCN. First, we interpret the current graph convolutional operations from an optimization perspective and argue that over-smoothing is mainly caused by the naive first-order approximation of the solution to the optimization problem. Subsequently, we introduce two metrics to measure the over-smoothing on node-level tasks. Specifically, we calculate the fraction of the pairwise distance between connected and disconnected nodes to the overall distance respectively. Based on our theoretical and empirical analysis, we establish a universal theoretical framework of GCN from an optimization perspective and derive a novel convolutional kernel named GCN+ which has lower parameter amount while relieving the over-smoothing inherently. Extensive experiments on real-world datasets demonstrate the superior performance of GCN+ over state-of-the-art baseline methods on the node classification tasks.

cs.LG↗
arXiv2020

Generic Detectability and Isolability of Topology Failures in Networked Linear Systems

This paper studies the possibility of detecting and isolating topology failures (including link failures and node failures) of a networked system from subsystem measurements, in which subsystems are of fixed high-order linear dynamics, and the exact interaction weights among them are unknown. We prove that in such class of networked systems with the same network topologies, the detectability and isolability of a given topology failure (set) are generic properties, indicating that it is the network topology that dominates the property of being detectable or isolable for a failure (set). We first give algebraic conditions for detectability and isolability of arbitrary parameter perturbations for a lumped plant, and then derive graph-theoretical necessary and sufficient conditions for generic detectability and isolability of topology failures for the networked systems. On the basis of these results, we consider the problems of deploying the smallest set of sensors for generic detectability and isolability. We reduce the associated sensor placement problems to the hitting set problems, which can be effectively solved by greedy algorithms with guaranteed approximation performances.

eess.SY↗
arXiv2020

Characterization of complementing pairs of $({\mathbb Z}_{\geq 0})^n$

Let $A, B, C$ be subsets of an abelian group $G$. A pair $(A, B)$ is called a $C$-pair if $A, B\subset C$ and $C$ is the direct sum of $A$ and $B$. The $(\Z_{\geq 0})$-pairs are characterized by de Bruijn in 1950 and the $(\Z_{\geq 0})^2$-pairs are characterized by Niven in 1971. In this paper, we characterize the $(\Z_{\geq 0})^n$-pairs for all $n\geq 1$. We show that every $(\Z_{\geq 0})^n$-pair is characterized by a weighted tree if it is primitive, that is, it is not a Cartesian product of a $(\Z_{\geq 0})^p$-pair and a $(\Z_{\geq 0})^q$-pair of lower dimensions.

math.DS↗
arXiv2020

On Chemical Distance and Local Uniqueness of a Sufficiently Supercritical Finitary Random Interlacement

In this paper, we study geometric properties of the unique infinite cluster $Γ$ in a sufficiently supercritical Finitary Random Interlacements $\mathcal{FI}^{u,T}$ in $\mathbb{Z}^d, \ d\ge 3$. We prove that the chemical distance in $Γ$ is, with stretched exponentially high probability, of the same order as the Euclidean distance in $\mathbb{Z}^d$. This also implies a shape theorem parallel to those for Bernoulli percolation and random interlacements. We also prove local uniqueness of $\mathcal{FI}^{u,T}$, which says any two large clusters in $\mathcal{FI}^{u,T}$ "close to each other" will with stretched exponentially high probability be connected to each other within the same order of the distance between them.

math.PR↗
arXiv2020

Improving Monocular Depth Estimation by Leveraging Structural Awareness and Complementary Datasets

Monocular depth estimation plays a crucial role in 3D recognition and understanding. One key limitation of existing approaches lies in their lack of structural information exploitation, which leads to inaccurate spatial layout, discontinuous surface, and ambiguous boundaries. In this paper, we tackle this problem in three aspects. First, to exploit the spatial relationship of visual features, we propose a structure-aware neural network with spatial attention blocks. These blocks guide the network attention to global structures or local details across different feature layers. Second, we introduce a global focal relative loss for uniform point pairs to enhance spatial constraint in the prediction, and explicitly increase the penalty on errors in depth-wise discontinuous regions, which helps preserve the sharpness of estimation results. Finally, based on analysis of failure cases for prior methods, we collect a new Hard Case (HC) Depth dataset of challenging scenes, such as special lighting conditions, dynamic objects, and tilted camera angles. The new dataset is leveraged by an informed learning curriculum that mixes training examples incrementally to handle diverse data distributions. Experimental results show that our method outperforms state-of-the-art approaches by a large margin in terms of both prediction accuracy on NYUDv2 dataset and generalization performance on unseen datasets.

cs.CV↗