arXiv subjects

Wei Huang

Publications and source records attributed to Wei Huang.

At least 325 records · Page 18Linked to original sources

arXiv2023

Practical Continuous-variable Quantum Key Distribution with Feasible Optimization Parameters

Continuous-variable quantum key distribution (CV-QKD) offers an approach to achieve a potential high secret key rate (SKR) in metropolitan areas. There are several challenges in developing a practical CV-QKD system from the laboratory to the real world. One of the most significant points is that it is really hard to adapt different practical optical fiber conditions for CV-QKD systems with unified hardware. Thus, how to improve the performance of practical CV-QKD systems in the field without modification of the hardware is very important. Here, a systematic optimization method, combining the modulation variance and error correction matrix optimization, is proposed to improve the performance of a practical CV-QKD system with a restricted capacity of postprocessing. The effect of restricted postprocessing capacity on the SKR is modeled as a nonlinear programming problem with modulation variance as an optimization parameter, and the selection of an optimal error correction matrix is studied under the same scheme. The results show that the SKR of a CV-QKD system can be improved by 24% and 200% compared with previous frequently used optimization methods theoretically with a transmission distance of 50 km. Furthermore, the experimental results verify the feasibility and robustness of the proposed method, where the achieved optimal SKR achieved practically deviates <1.6% from the theoretical optimal value. Our results pave the way to deploy high-performance CV-QKD in the real world.

quant-ph↗

arXiv2023

Generic-to-Specific Distillation of Masked Autoencoders

Large vision Transformers (ViTs) driven by self-supervised pre-training mechanisms achieved unprecedented progress. Lightweight ViT models limited by the model capacity, however, benefit little from those pre-training mechanisms. Knowledge distillation defines a paradigm to transfer representations from large (teacher) models to small (student) ones. However, the conventional single-stage distillation easily gets stuck on task-specific transfer, failing to retain the task-agnostic knowledge crucial for model generalization. In this study, we propose generic-to-specific distillation (G2SD), to tap the potential of small ViT models under the supervision of large models pre-trained by masked autoencoders. In generic distillation, decoder of the small model is encouraged to align feature predictions with hidden representations of the large model, so that task-agnostic knowledge can be transferred. In specific distillation, predictions of the small model are constrained to be consistent with those of the large model, to transfer task-specific features which guarantee task performance. With G2SD, the vanilla ViT-Small model respectively achieves 98.7%, 98.1% and 99.3% the performance of its teacher (ViT-Base) for image classification, object detection, and semantic segmentation, setting a solid baseline for two-stage vision distillation. Code will be available at https://github.com/pengzhiliang/G2SD.

cs.CV↗

arXiv2023

DMMG: Dual Min-Max Games for Self-Supervised Skeleton-Based Action Recognition

In this work, we propose a new Dual Min-Max Games (DMMG) based self-supervised skeleton action recognition method by augmenting unlabeled data in a contrastive learning framework. Our DMMG consists of a viewpoint variation min-max game and an edge perturbation min-max game. These two min-max games adopt an adversarial paradigm to perform data augmentation on the skeleton sequences and graph-structured body joints, respectively. Our viewpoint variation min-max game focuses on constructing various hard contrastive pairs by generating skeleton sequences from various viewpoints. These hard contrastive pairs help our model learn representative action features, thus facilitating model transfer to downstream tasks. Moreover, our edge perturbation min-max game specializes in building diverse hard contrastive samples through perturbing connectivity strength among graph-based body joints. The connectivity-strength varying contrastive pairs enable the model to capture minimal sufficient information of different actions, such as representative gestures for an action while preventing the model from overfitting. By fully exploiting the proposed DMMG, we can generate sufficient challenging contrastive pairs and thus achieve discriminative action feature representations from unlabeled skeleton data in a self-supervised manner. Extensive experiments demonstrate that our method achieves superior results under various evaluation protocols on widely-used NTU-RGB+D and NTU120-RGB+D datasets.

cs.CV↗

arXiv2023

Oxidized organic molecules in the tropical free troposphere over Amazonia

New particle formation (NPF) in the tropical free troposphere (FT) is a globally important source of cloud condensation nuclei, affecting cloud properties and climate. Oxidized organic molecules (OOMs) produced from biogenic volatile organic compounds are believed to contribute to aerosol formation in the tropical FT, but without direct chemical observations. We performed in-situ molecular-level OOMs measurements at the Bolivian station Chacaltaya at 5240 meters above sea level, on the western edge of Amazonia. For the first time, we demonstrate the presence of OOMs, mainly with 4-5 carbon atoms, simultaneously in both gas and particulate phases in tropical FT air from Amazonia. These observations, combined with air mass history analyses, indicate that the observed OOMs are linked to isoprene emitted from the rainforests hundreds of kilometers away. Based on particle-phase measurements, we find that these compounds can contribute to the growth of newly formed particles, and are potentially crucial for new particle formation in the tropical free troposphere on a continental scale. Our study will thus improve the understanding of aerosol formation process in the tropics.

physics.ao-ph↗

arXiv2023

On the Sparse DAG Structure Learning Based on Adaptive Lasso

Learning the underlying Bayesian Networks (BNs), represented by directed acyclic graphs (DAGs), of the concerned events from purely-observational data is a crucial part of evidential reasoning. This task remains challenging due to the large and discrete search space. A recent flurry of developments followed NOTEARS[1] recast this combinatorial problem into a continuous optimization problem by leveraging an algebraic equality characterization of acyclicity. However, the continuous optimization methods suffer from obtaining non-spare graphs after the numerical optimization, which leads to the inflexibility to rule out the potentially cycle-inducing edges or false discovery edges with small values. To address this issue, in this paper, we develop a completely data-driven DAG structure learning method without a predefined value to post-threshold small values. We name our method NOTEARS with adaptive Lasso (NOTEARS-AL), which is achieved by applying the adaptive penalty method to ensure the sparsity of the estimated DAG. Moreover, we show that NOTEARS-AL also inherits the oracle properties under some specific conditions. Extensive experiments on both synthetic and a real-world dataset demonstrate that our method consistently outperforms NOTEARS.

stat.ML↗

arXiv2023

Communication under Mixed Gaussian-Impulsive Channel: An End-to-End Framework

In many communication scenarios, the communication signals simultaneously suffer from white Gaussian noise (WGN) and non-Gaussian impulsive noise (IN), i.e., mixed Gaussian-impulsive noise (MGIN). Under MGIN channel, classical communication signal schemes and corresponding detection methods usually can not achieve desirable performance as they are optimized with respect to WGN. Moreover, as the widely adopted IN model has no analytical and general closed-form expression of probability density function (PDF), it is extremely hard to obtain optimal communication signal and corresponding detection schemes based on classical stochastic signal processing theory. To circumvent these difficulties, we propose a data-driven end-to-end framework to address the communication signal design and detection under MGIN channel in this paper. In this proposed framework, a channel noise simulator (CNS) is elaborately designed based on an improved generative adversarial net (GAN) to simulate the MGIN without requirement of any analytical PDF. Meanwhile, a multi-level wavelet convolutional neural network (MWCNN) based preprocessing network is used to mitigate the negative effect of outliers due to the IN. Compared with conventional approaches and existing end-to-end systems, extensive simulation results verify that our proposed novel end-to-end communication system can achieve better performance in terms of bit-error rate (BER) under MGIN environments.

eess.SP↗

arXiv2023

MissDAG: Causal Discovery in the Presence of Missing Data with Continuous Additive Noise Models

State-of-the-art causal discovery methods usually assume that the observational data is complete. However, the missing data problem is pervasive in many practical scenarios such as clinical trials, economics, and biology. One straightforward way to address the missing data problem is first to impute the data using off-the-shelf imputation methods and then apply existing causal discovery methods. However, such a two-step method may suffer from suboptimality, as the imputation algorithm may introduce bias for modeling the underlying data distribution. In this paper, we develop a general method, which we call MissDAG, to perform causal discovery from data with incomplete observations. Focusing mainly on the assumptions of ignorable missingness and the identifiable additive noise models (ANMs), MissDAG maximizes the expected likelihood of the visible part of observations under the expectation-maximization (EM) framework. In the E-step, in cases where computing the posterior distributions of parameters in closed-form is not feasible, Monte Carlo EM is leveraged to approximate the likelihood. In the M-step, MissDAG leverages the density transformation to model the noise distributions with simpler and specific formulations by virtue of the ANMs and uses a likelihood-based causal discovery algorithm with directed acyclic graph constraint. We demonstrate the flexibility of MissDAG for incorporating various causal discovery algorithms and its efficacy through extensive simulations and real data experiments.

cs.LG↗

arXiv2022

Performance analysis for OFDM-based multi-carrier continuous-variable quantum key distribution with arbitrary modulation protocol

Multi-carrier continuous-variable quantum key distribution (CV-QKD) is considered to be a promising way to boost the secret key rate (SKR) over the existing single-carrier CV-QKD scheme. However, the extra excess noise induced in the imperfect multi-carrier quantum state preparation process of N subcarriers will limit the performance of the system. Here, a systematic modulation noise model is proposed for the multi-carrier CV-QKD based on the orthogonal frequency division multiplexing (OFDM). Subsequently, the performance of multi-carrier CV-QKD with arbitrary modulation protocol (e.g. QPSK, 256QAM and Gaussian modulation protocol) can be quantitatively evaluated by combining the security analysis method of the single-carrier CV-QKD. Under practical system parameters, the simulation results show that the SKR of the multi-carrier CV-QKD can still be significantly improved by increasing the carrier number N even with imperfect practical modulations. Specifically, the total SKR of multi-carrier CV-QKD can be optimized by carefully choosing N. The proposed model provides a feasible theoretical framework for the future multi-carrier CV-QKD experimental implementation.

quant-ph↗

arXiv2022

Sub-Mbps key-rate continuous-variable quantum key distribution with local-local-oscillator over 100 km fiber

We experimentally demonstrated a sub-Mbps key rate Gaussian-modulated coherent-state continuous-variable quantum key distribution (CV-QKD) over 100 km transmission distance. To efficiently control the excess noise, the quantum signal and the pilot tone are co-transmitted in fiber channel based on wide-band frequency and polarization multiplexing methods. Furthermore, a high-accuracy data-assisted time domain equalization algorithm is carefully designed to compensate the phase noise and polarization variation in low signal-to-noise ratio. The asymptotic secure key rate (SKR) of the demonstrated CV-QKD is experimentally evaluated to be 10.36 Mbps, 2.59 Mbps, and 0.69 Mbps over transmission distance of 50 km, 75 km, and 100 km, respectively. The experimental demonstrated CV-QKD system significantly improves transmission distance and SKR compared to the state-of-art GMCS CV-QKD experimental results, and shows the potential for long-distance and high-speed secure quantum key distribution.

quant-ph↗

arXiv2022

TCFimt: Temporal Counterfactual Forecasting from Individual Multiple Treatment Perspective

Determining causal effects of temporal multi-intervention assists decision-making. Restricted by time-varying bias, selection bias, and interactions of multiple interventions, the disentanglement and estimation of multiple treatment effects from individual temporal data is still rare. To tackle these challenges, we propose a comprehensive framework of temporal counterfactual forecasting from an individual multiple treatment perspective (TCFimt). TCFimt constructs adversarial tasks in a seq2seq framework to alleviate selection and time-varying bias and designs a contrastive learning-based block to decouple a mixed treatment effect into separated main treatment effects and causal interactions which further improves estimation accuracy. Through implementing experiments on two real-world datasets from distinct fields, the proposed method shows satisfactory performance in predicting future outcomes with specific treatments and in choosing optimal treatment type and timing than state-of-the-art methods.

cs.LG↗

arXiv2022

Nonparametric Estimation of the Continuous Treatment Effect with Measurement Error

We identify the average dose-response function (ADRF) for a continuously valued error-contaminated treatment by a weighted conditional expectation. We then estimate the weights nonparametrically by maximising a local generalised empirical likelihood subject to an expanding set of conditional moment equations incorporated into the deconvolution kernels. Thereafter, we construct a deconvolution kernel estimator of ADRF. We derive the asymptotic bias and variance of our ADRF estimator and provide its asymptotic linear expansion, which helps conduct statistical inference. To select our smoothing parameters, we adopt the simulation-extrapolation method and propose a new extrapolation procedure to stabilise the computation. Monte Carlo simulations and a real data study illustrate our method's practical performance.

math.ST↗

arXiv2022

Deep Active Learning by Leveraging Training Dynamics

Active learning theories and methods have been extensively studied in classical statistical learning settings. However, deep active learning, i.e., active learning with deep learning models, is usually based on empirical criteria without solid theoretical justification, thus suffering from heavy doubts when some of those fail to provide benefits in real applications. In this paper, by exploring the connection between the generalization performance and the training dynamics, we propose a theory-driven deep active learning method (dynamicAL) which selects samples to maximize training dynamics. In particular, we prove that the convergence speed of training and the generalization performance are positively correlated under the ultra-wide condition and show that maximizing the training dynamics leads to better generalization performance. Furthermore, to scale up to large deep neural networks and data sets, we introduce two relaxations for the subset selection problem and reduce the time complexity from polynomial to constant. Empirical results show that dynamicAL not only outperforms the other baselines consistently but also scales well on large deep learning models. We hope our work would inspire more attempts on bridging the theoretical findings of deep networks and practical impacts of deep active learning in real applications.

cs.LG↗

arXiv2022

Synthetic Topological Vacua of Yang-Mills Fields in Bose-Einstein Condensates

Topological vacua are a family of degenerate ground states of Yang-Mills fields with zero field strength but nontrivial topological structures. They play a fundamental role in particle physics and quantum field theory, but have not yet been experimentally observed. Here we report the first theoretical proposal and experimental realization of synthetic topological vacua with a cloud of atomic Bose-Einstein condensates. Our setup provides a promising platform to demonstrate the fundamental concept that a vacuum, rather than being empty, has rich spatial structures. The Hamiltonian for the vacuum of topological number n = 1 is synthesized and the related Hopf index is measured. The vacuum of topological number n = 2 is also realized, and we find that vacua with different topological numbers have distinctive spin textures and Hopf links. Our work opens up opportunities for exploring topological vacua and related long-sought-after instantons in tabletop experiments.

cond-mat.quant-gas↗

arXiv2022

Anisotropic multiresolution analyses for deepfake detection

Generative Adversarial Networks (GANs) have paved the path towards entirely new media generation capabilities at the forefront of image, video, and audio synthesis. However, they can also be misused and abused to fabricate elaborate lies, capable of stirring up the public debate. The threat posed by GANs has sparked the need to discern between genuine content and fabricated one. Previous studies have tackled this task by using classical machine learning techniques, such as k-nearest neighbours and eigenfaces, which unfortunately did not prove very effective. Subsequent methods have focused on leveraging on frequency decompositions, i.e., discrete cosine transform, wavelets, and wavelet packets, to preprocess the input features for classifiers. However, existing approaches only rely on isotropic transformations. We argue that, since GANs primarily utilize isotropic convolutions to generate their output, they leave clear traces, their fingerprint, in the coefficient distribution on sub-bands extracted by anisotropic transformations. We employ the fully separable wavelet transform and multiwavelets to obtain the anisotropic features to feed to standard CNN classifiers. Lastly, we find the fully separable transform capable of improving the state-of-the-art.

cs.CV↗

arXiv2022

ViT-LSLA: Vision Transformer with Light Self-Limited-Attention

Transformers have demonstrated a competitive performance across a wide range of vision tasks, while it is very expensive to compute the global self-attention. Many methods limit the range of attention within a local window to reduce computation complexity. However, their approaches cannot save the number of parameters; meanwhile, the self-attention and inner position bias (inside the softmax function) cause each query to focus on similar and close patches. Consequently, this paper presents a light self-limited-attention (LSLA) consisting of a light self-attention mechanism (LSA) to save the computation cost and the number of parameters, and a self-limited-attention mechanism (SLA) to improve the performance. Firstly, the LSA replaces the K (Key) and V (Value) of self-attention with the X(origin input). Applying it in vision Transformers which have encoder architecture and self-attention mechanism, can simplify the computation. Secondly, the SLA has a positional information module and a limited-attention module. The former contains a dynamic scale and an inner position bias to adjust the distribution of the self-attention scores and enhance the positional information. The latter uses an outer position bias after the softmax function to limit some large values of attention weights. Finally, a hierarchical Vision Transformer with Light self-Limited-attention (ViT-LSLA) is presented. The experiments show that ViT-LSLA achieves 71.6% top-1 accuracy on IP102 (2.4% absolute improvement of Swin-T); 87.2% top-1 accuracy on Mini-ImageNet (3.7% absolute improvement of Swin-T). Furthermore, it greatly reduces FLOPs (3.5GFLOPs vs. 4.5GFLOPs of Swin-T) and parameters (18.9M vs. 27.6M of Swin-T).

cs.CV↗

arXiv2022

Joint Microstrip Selection and Beamforming Design for MmWave Systems with Dynamic Metasurface Antennas

Dynamic metasurface antennas (DMAs) provide a new paradigm to realize large-scale antenna arrays for future wireless systems. In this paper, we study the downlink millimeter wave (mmWave) DMA systems with limited number of radio frequency (RF) chains. By using a specific DMA structure, an equivalent mmWave channel model is first explicitly characterized. Based on that, we propose an effective joint microstrip selection and beamforming scheme to accommodate for the limited number of RF chains. A low-complexity digital beamforming solution with channel gain-based microstrip selection is developed, while the analog beamformer is obtained via a coordinate ascent method. The proposed scheme is numerically shown to approach the performance of DMAs without RF chain reduction, verifying the effectiveness of the proposed schemes.

eess.SP↗

arXiv2022

Reliability Assessment and Safety Arguments for Machine Learning Components in System Assurance

The increasing use of Machine Learning (ML) components embedded in autonomous systems -- so-called Learning-Enabled Systems (LESs) -- has resulted in the pressing need to assure their functional safety. As for traditional functional safety, the emerging consensus within both, industry and academia, is to use assurance cases for this purpose. Typically assurance cases support claims of reliability in support of safety, and can be viewed as a structured way of organising arguments and evidence generated from safety analysis and reliability modelling activities. While such assurance activities are traditionally guided by consensus-based standards developed from vast engineering experience, LESs pose new challenges in safety-critical application due to the characteristics and design of ML models. In this article, we first present an overall assurance framework for LESs with an emphasis on quantitative aspects, e.g., breaking down system-level safety targets to component-level requirements and supporting claims stated in reliability metrics. We then introduce a novel model-agnostic Reliability Assessment Model (RAM) for ML classifiers that utilises the operational profile and robustness verification evidence. We discuss the model assumptions and the inherent challenges of assessing ML reliability uncovered by our RAM and propose solutions to practical use. Probabilistic safety argument templates at the lower ML component-level are also developed based on the RAM. Finally, to evaluate and demonstrate our methods, we not only conduct experiments on synthetic/benchmark datasets but also scope our methods with case studies on simulated Autonomous Underwater Vehicles and physical Unmanned Ground Vehicles.

cs.SE↗

arXiv2022

Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis

Advanced deep neural networks (DNNs), designed by either human or AutoML algorithms, are growing increasingly complex. Diverse operations are connected by complicated connectivity patterns, e.g., various types of skip connections. Those topological compositions are empirically effective and observed to smooth the loss landscape and facilitate the gradient flow in general. However, it remains elusive to derive any principled understanding of their effects on the DNN capacity or trainability, and to understand why or in which aspect one specific connectivity pattern is better than another. In this work, we theoretically characterize the impact of connectivity patterns on the convergence of DNNs under gradient descent training in fine granularity. By analyzing a wide network's Neural Network Gaussian Process (NNGP), we are able to depict how the spectrum of an NNGP kernel propagates through a particular connectivity pattern, and how that affects the bound of convergence rates. As one practical implication of our results, we show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate, and significantly accelerate the large-scale neural architecture search without any overhead. Code is available at: https://github.com/VITA-Group/architecture_convergence.

cs.LG↗