arXiv subjects

Zhen Wang

Publications and source records attributed to Zhen Wang.

At least 451 records · Page 25Linked to original sources

arXiv2022

Modern Question Answering Datasets and Benchmarks: A Survey

Question Answering (QA) is one of the most important natural language processing (NLP) tasks. It aims using NLP technologies to generate a corresponding answer to a given question based on the massive unstructured corpus. With the development of deep learning, more and more challenging QA datasets are being proposed, and lots of new methods for solving them are also emerging. In this paper, we investigate influential QA datasets that have been released in the era of deep learning. Specifically, we begin with introducing two of the most common QA tasks - textual question answer and visual question answering - separately, covering the most representative datasets, and then give some current challenges of QA research.

cs.CL↗

arXiv2022

A Benchmark for Federated Hetero-Task Learning

To investigate the heterogeneity in federated learning in real-world scenarios, we generalize the classic federated learning to federated hetero-task learning, which emphasizes the inconsistency across the participants in federated learning in terms of both data distribution and learning tasks. We also present B-FHTL, a federated hetero-task learning benchmark consisting of simulation dataset, FL protocols and a unified evaluation mechanism. B-FHTL dataset contains three well-designed federated learning tasks with increasing heterogeneity. Each task simulates the clients with different non-IID data and learning tasks. To ensure fair comparison among different FL algorithms, B-FHTL builds in a full suite of FL protocols by providing high-level APIs to avoid privacy leakage, and presets most common evaluation metrics spanning across different learning tasks, such as regression, classification, text generation and etc. Furthermore, we compare the FL algorithms in fields of federated multi-task learning, federated personalization and federated meta learning within B-FHTL, and highlight the influence of heterogeneity and difficulties of federated hetero-task learning. Our benchmark, including the federated dataset, protocols, the evaluation mechanism and the preliminary experiment, is open-sourced at https://github.com/alibaba/FederatedScope/tree/master/benchmark/B-FHTL

cs.LG↗

arXiv2022

FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimization

Hyperparameter optimization (HPO) is crucial for machine learning algorithms to achieve satisfactory performance, whose progress has been boosted by related benchmarks. Nonetheless, existing efforts in benchmarking all focus on HPO for traditional centralized learning while ignoring federated learning (FL), a promising paradigm for collaboratively learning models from dispersed data. In this paper, we first identify some uniqueness of HPO for FL algorithms from various aspects. Due to this uniqueness, existing HPO benchmarks no longer satisfy the need to compare HPO methods in the FL setting. To facilitate the research of HPO in the FL setting, we propose and implement a benchmark suite FedHPO-B that incorporates comprehensive FL tasks, enables efficient function evaluations, and eases continuing extensions. We also conduct extensive experiments based on FedHPO-B to benchmark a few HPO methods. We open-source FedHPO-B at https://github.com/alibaba/FederatedScope/tree/master/benchmark/FedHPOB.

cs.LG↗

arXiv2022

Extreme ratio between spectral and Frobenius norms of nonnegative tensors

One of the fundamental problems in multilinear algebra, the minimum ratio between the spectral and Frobenius norms of tensors, has received considerable attention in recent years. While most values are unknown for real and complex tensors, the asymptotic order of magnitude and tight lower bounds have been established. However, little is known about nonnegative tensors. In this paper, we present an almost complete picture of the ratio for nonnegative tensors. In particular, we provide a tight lower bound that can be achieved by a wide class of nonnegative tensors under a simple necessary and sufficient condition, which helps to characterize the extreme tensors and obtain results such as the asymptotic order of magnitude. We show that the ratio for symmetric tensors is no more than that for general tensors multiplied by a constant depending only on the order of tensors, hence determining the asymptotic order of magnitude for real, complex, and nonnegative symmetric tensors. We also find that the ratio is in general different to the minimum ratio between the Frobenius and nuclear norms for nonnegative tensors, a sharp contrast to the case for real tensors and complex tensors.

math.NA↗

arXiv2022

NSSIA: A New Self-Sovereign Identity Scheme with Accountability

Self-Sovereign Identity (SSI) is a new distributed method for identity management, commonly used to address the problem that users are lack of control over their identities. However, the excessive pursuit of self-sovereignty in the most existing SSI schemes hinders sanctions against attackers. To deal with the malicious behavior, a few SSI schemes introduce accountability mechanisms, but they sacrifice users' privacy. What's more, the digital identities (static strings or updatable chains) in the existing SSI schemes are as inputs to a third-party executable program (mobile app, smart contract, etc.) to achieve identity reading, storing and proving, users' self-sovereignty are weakened. To solve the above problems, we present a new self-sovereign identity scheme to strike a balance between privacy and accountability and get rid of the dependence on the third-party program. In our scheme, one and only individual-specific executable code is generated as a digital avatar-i for each human to interact with others in cyberspace without a third-party program, in which the embedding of biometrics enhances uniqueness and user control over their identity. In addition, a joint accountability mechanism, which is based on the shamir (t, n) threshold algorithm and a consortium blockchain, is designed to restrict the power of each regulatory authority and protect users' privacy. Finally, we analyze the security, SSI properties and conduct detailed experiments in term of the cost of computation, storage and blockchain gas. The analysis results indicate that our scheme resists the known attacks and fulfills all the six SSI properties. Compared with the state-of-the-art schemes, the extensive experiment results show that the cost is larger in server storage, blockchain storage and blockchain gas, but is still low enough for practical situations.

cs.CR↗

arXiv2022

N24News: A New Dataset for Multimodal News Classification

Current news datasets merely focus on text features on the news and rarely leverage the feature of images, excluding numerous essential features for news classification. In this paper, we propose a new dataset, N24News, which is generated from New York Times with 24 categories and contains both text and image information in each news. We use a multitask multimodal method and the experimental results show multimodal news classification performs better than text-only news classification. Depending on the length of the text, the classification accuracy can be increased by up to 8.11%. Our research reveals the relationship between the performance of a multimodal classifier and its sub-classifiers, and also the possible improvements when applying multimodal in news classification. N24News is shown to have great potential to prompt the multimodal news studies.

cs.CL↗

arXiv2022

Inferring Network Structures via Signal Lasso

Inferring the connectivity structure of networked systems from data is an extremely important task in many areas of science. Most of real-world networks exhibit sparsely connected topologies, with links between nodes that in some cases may be even associated to a binary state (0 or 1, denoting respectively the absence or the existence of a connection). Such un-weighted topologies are elusive to classical reconstruction methods such as Lasso or Compressed Sensing techniques. We here introduce a novel approach called signal Lasso, where the estimation of the signal parameter is subjected to 0 or 1 values. The theoretical properties and algorithm of proposed method are studied in detail. Applications of the method are illustrated to an evolutionary game and synchronization dynamics in several synthetic and empirical networks, where we show that the novel strategy is reliable and robust, and outperform the classical approaches in terms of accuracy and mean square errors.

physics.soc-ph↗

arXiv2022

PAnDR: Fast Adaptation to New Environments from Offline Experiences via Decoupling Policy and Environment Representations

Deep Reinforcement Learning (DRL) has been a promising solution to many complex decision-making problems. Nevertheless, the notorious weakness in generalization among environments prevent widespread application of DRL agents in real-world scenarios. Although advances have been made recently, most prior works assume sufficient online interaction on training environments, which can be costly in practical cases. To this end, we focus on an offline-training-online-adaptation setting, in which the agent first learns from offline experiences collected in environments with different dynamics and then performs online policy adaptation in environments with new dynamics. In this paper, we propose Policy Adaptation with Decoupled Representations (PAnDR) for fast policy adaptation. In offline training phase, the environment representation and policy representation are learned through contrastive learning and policy recovery, respectively. The representations are further refined by mutual information optimization to make them more decoupled and complete. With learned representations, a Policy-Dynamics Value Function (PDVF) [Raileanu et al., 2020] network is trained to approximate the values for different combinations of policies and environments from offline experiences. In online adaptation phase, with the environment context inferred from few experiences collected in new environments, the policy is optimized by gradient ascent with respect to the PDVF. Our experiments show that PAnDR outperforms existing algorithms in several representative policy adaptation problems.

cs.LG↗

arXiv2022

A Hybrid Josephson Transmission Line and Passive Transmission Line Routing Framework for Single Flux Quantum Logic

The Single Flux Quantum (SFQ) logic family is a novel digital logic as it provides ultra-fast and energy-efficient circuits. For large-scale SFQ circuit design, specialized electronic design automation (EDA) tools are required due to the differences in logic type, timing constraints and circuit architecture, in contrast to the CMOS logic. In order to improve the overall performance of an SFQ circuit, an efficient routing algorithm should be applied during the layout design to perform accurate timing adjustment for fixing hold violations and optimizing critical paths. Thus, a hybrid Josephson transmission line and passive transmission line routing framework is proposed. It consists of four main modules and an exploration of the potential timing performance based on the given layout placement. The proposed routing tool is demonstrated on seven testbench circuits. The obtained results demonstrate that the operating frequency is greatly improved, and all the hold violations are eliminated for each circuit.

cs.ET↗

arXiv2022

Graph-Based Similarity of Neural Network Representations

Understanding the black-box representations in Deep Neural Networks (DNN) is an essential problem in deep learning. In this work, we propose Graph-Based Similarity (GBS) to measure the similarity of layer features. Contrary to previous works that compute the similarity directly on the feature maps, GBS measures the correlation based on the graph constructed with hidden layer outputs. By treating each input sample as a node and the corresponding layer output similarity as edges, we construct the graph of DNN representations for each layer. The similarity between graphs of layers identifies the correspondences between representations of models trained in different datasets and initializations. We demonstrate and prove the invariance property of GBS, including invariance to orthogonal transformation and invariance to isotropic scaling, and compare GBS with CKA. GBS shows state-of-the-art performance in reflecting the similarity and provides insights on explaining the adversarial sample behavior on the hidden layer space.

cs.LG↗

arXiv2022

PrEF: Percolation-based Evolutionary Framework for the diffusion-source-localization problem in large networks

We assume that the state of a number of nodes in a network could be investigated if necessary, and study what configuration of those nodes could facilitate a better solution for the diffusion-source-localization (DSL) problem. In particular, we formulate a candidate set which contains the diffusion source for sure, and propose the method, Percolation-based Evolutionary Framework (PrEF), to minimize such set. Hence one could further conduct more intensive investigation on only a few nodes to target the source. To achieve that, we first demonstrate that there are some similarities between the DSL problem and the network immunization problem. We find that the minimization of the candidate set is equivalent to the minimization of the order parameter if we view the observer set as the removal node set. Hence, PrEF is developed based on the network percolation and evolutionary algorithm. The effectiveness of the proposed method is validated on both model and empirical networks in regard to varied circumstances. Our results show that the developed approach could achieve a much smaller candidate set compared to the state of the art in almost all cases. Meanwhile, our approach is also more stable, i.e., it has similar performance irrespective of varied infection probabilities, diffusion models, and outbreak ranges. More importantly, our approach might provide a new framework to tackle the DSL problem in extreme large networks.

cs.SI↗

arXiv2022

The effect of dressing on thermalization of interacting waves

We propose a more general setup for prethermalization in the system of interacting waves. The idea lies in dividing the multi-wave interactions into trivial and nontrivial ones. The trivial interactions will dress waves and lead to a less strongly interacting system which is statistically equivalent to the original one. With this in mind, we find that prethermalization occurs not only in the weakly interacting regime but also in the strongly interacting regime. The irreversible process towards equilibrium is governed by the Zakharov equation, from which double scaling of the thermalization time is expected. Finally, the theory is well confirmed in numerical experiments.

cond-mat.stat-mech↗

arXiv2022

Towards Fine-grained Causal Reasoning and QA

Understanding causality is key to the success of NLP applications, especially in high-stakes domains. Causality comes in various perspectives such as enable and prevent that, despite their importance, have been largely ignored in the literature. This paper introduces a novel fine-grained causal reasoning dataset and presents a series of novel predictive tasks in NLP, such as causality detection, event causality extraction, and Causal QA. Our dataset contains human annotations of 25K cause-effect event pairs and 24K question-answering pairs within multi-sentence samples, where each can have multiple causal relationships. Through extensive experiments and analysis, we show that the complex relations in our dataset bring unique challenges to state-of-the-art methods across all three tasks and highlight potential research opportunities, especially in developing "causal-thinking" methods.

cs.CL↗

arXiv2022

Doubly resonant photoacoustic spectroscopy: ultra-high sensitivity meets ultra-wide dynamic range

Photoacoustic spectroscopy (PAS) based gas sensors with high sensitivity, wide dynamic range, low cost, and small footprint are desirable across a broad range of applications in energy, environment, safety, and public health. However, most works have focused on either acoustic resonator to enhance acoustic wave or optical resonator to enhance optical wave. Herein, we develop a gas sensor based on doubly resonant PAS in which the acoustic and optical waves are simultaneously enhanced using combined optical and acoustic resonators in a centimeter-long configuration. Not only the lower detection limit is enhanced by the double standing waves, but also the upper detection limit is expanded due to the short resonators. As an example, we developed a sensor by detecting acetylene (C2H2), achieving a noise equivalent absorption of 5.7*10-13 cm-1 and a dynamic range of eight orders. Compared to the state-of-the-art PAS gas sensors, the developed sensor increases the sensitivity by two orders of magnitude and extends the dynamic range by three orders of magnitude. Besides, a laser-cavity-molecule locking strategy is proposed to provide additional flexibility of fast gas detection.

physics.app-ph↗

arXiv2022

Experimental quantum adversarial learning with programmable superconducting qubits

Quantum computing promises to enhance machine learning and artificial intelligence. Different quantum algorithms have been proposed to improve a wide spectrum of machine learning tasks. Yet, recent theoretical works show that, similar to traditional classifiers based on deep classical neural networks, quantum classifiers would suffer from the vulnerability problem: adding tiny carefully-crafted perturbations to the legitimate original data samples would facilitate incorrect predictions at a notably high confidence level. This will pose serious problems for future quantum machine learning applications in safety and security-critical scenarios. Here, we report the first experimental demonstration of quantum adversarial learning with programmable superconducting qubits. We train quantum classifiers, which are built upon variational quantum circuits consisting of ten transmon qubits featuring average lifetimes of 150 $μ$s, and average fidelities of simultaneous single- and two-qubit gates above 99.94% and 99.4% respectively, with both real-life images (e.g., medical magnetic resonance imaging scans) and quantum data. We demonstrate that these well-trained classifiers (with testing accuracy up to 99%) can be practically deceived by small adversarial perturbations, whereas an adversarial training process would significantly enhance their robustness to such perturbations. Our results reveal experimentally a crucial vulnerability aspect of quantum learning systems under adversarial scenarios and demonstrate an effective defense strategy against adversarial attacks, which provide a valuable guide for quantum artificial intelligence applications with both near-term and future quantum devices.

quant-ph↗

arXiv2022

Many-body Hilbert space scarring on a superconducting processor

Quantum many-body scarring (QMBS) -- a recently discovered form of weak ergodicity breaking in strongly-interacting quantum systems -- presents opportunities for mitigating thermalization-induced decoherence in quantum information processsing. However, the existing experimental realizations of QMBS are based on kinetically-constrained systems where an emergent dynamical symmetry "shields" such states from the thermalizing bulk of the spectrum. Here, we experimentally realize a distinct kind of QMBS phenomena by approximately decoupling a part of the many-body Hilbert space in the computational basis. Utilizing a programmable superconducting processor with 30 qubits and tunable couplings, we realize Hilbert space scarring in a non-constrained model in different geometries, including a linear chain as well as a quasi-one-dimensional comb geometry. By performing full quantum state tomography on 4-qubit subsystems, we provide strong evidence for QMBS states by measuring qubit population dynamics, quantum fidelity and entanglement entropy following a quench from initial product states. Our experimental findings broaden the realm of QMBS mechanisms and pave the way to exploiting correlations in QMBS states for applications in quantum information technology.

quant-ph↗

arXiv2022

Study of Electroweak Phase Transition in Exotic Higgs Decays at the CEPC

A strong first-order electroweak phase transition (EWPT) can be induced by light new physics weakly coupled to the Higgs. This study focuses on a scenario in which the first-order EWPT is driven by a light scalar $s$ with a mass between 15-60 GeV. A search for exotic decays of the Higgs boson into a pair of spin-zero particles, $h \to ss$, where the $s$-boson decays into $b$-quarks promptly is presented. The search is performed in events where the Higgs boson is produced in association with a $Z$ boson, giving rise to a signature of two charged leptons (electrons or muons) and multiple jets from $b$-quark decays. The analysis is considering a scenario of analysing 5000 fb$^{-1}$ $e^+ e^-$ collision data at $\sqrt{s} = 240 $ GeV from the Circular Electron Positron Collider (CEPC). This study with $4b$ final state conclusively tests the expected sensitivity of probing the light scalars in the CEPC experiment. The sensitivity reach is significantly larger than that can be achieved at the LHC.

hep-ex↗

arXiv2022

HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation

Discrete-continuous hybrid action space is a natural setting in many practical problems, such as robot control and game AI. However, most previous Reinforcement Learning (RL) works only demonstrate the success in controlling with either discrete or continuous action space, while seldom take into account the hybrid action space. One naive way to address hybrid action RL is to convert the hybrid action space into a unified homogeneous action space by discretization or continualization, so that conventional RL algorithms can be applied. However, this ignores the underlying structure of hybrid action space and also induces the scalability issue and additional approximation difficulties, thus leading to degenerated results. In this paper, we propose Hybrid Action Representation (HyAR) to learn a compact and decodable latent representation space for the original hybrid action space. HyAR constructs the latent space and embeds the dependence between discrete action and continuous parameter via an embedding table and conditional Variantional Auto-Encoder (VAE). To further improve the effectiveness, the action representation is trained to be semantically smooth through unsupervised environmental dynamics prediction. Finally, the agent then learns its policy with conventional DRL algorithms in the learned representation space and interacts with the environment by decoding the hybrid action embeddings to the original action space. We evaluate HyAR in a variety of environments with discrete-continuous action space. The results demonstrate the superiority of HyAR when compared with previous baselines, especially for high-dimensional action spaces.

cs.LG↗