arXiv subjects

Gang Huang

Publications and source records attributed to Gang Huang.

At least 91 records · Page 5Linked to original sources

arXiv2023

Masked Collaborative Contrast for Weakly Supervised Semantic Segmentation

This study introduces an efficacious approach, Masked Collaborative Contrast (MCC), to highlight semantic regions in weakly supervised semantic segmentation. MCC adroitly draws inspiration from masked image modeling and contrastive learning to devise a novel framework that induces keys to contract toward semantic regions. Unlike prevalent techniques that directly eradicate patch regions in the input image when generating masks, we scrutinize the neighborhood relations of patch tokens by exploring masks considering keys on the affinity matrix. Moreover, we generate positive and negative samples in contrastive learning by utilizing the masked local output and contrasting it with the global output. Elaborate experiments on commonly employed datasets evidences that the proposed MCC mechanism effectively aligns global and local perspectives within the image, attaining impressive performance. The source code is available at \url{https://github.com/fwu11/MCC}.

cs.CV↗

arXiv2023

Stability results for a hierarchical size-structured population model with distributed delay

In this paper we investigate a structured population model with distributed delay. Our model incorporates two different types of nonlinearities. Specifically we assume that individual growth and mortality are affected by scramble competition, while fertility is affected by contest competition. In particular, we assume that there is a hierarchical structure in the population, which affects mating success. The dynamical behavior of the model is analysed via linearisation by means of semigroup and spectral methods. In particular, we introduce a reproduction function and use it to derive linear stability criteria for our model. Further we present numerical simulations to underpin the stability results we obtained.

math.AP↗

arXiv2023

Energy-Efficient GPU Clusters Scheduling for Deep Learning

Training deep neural networks (DNNs) is a major workload in datacenters today, resulting in a tremendously fast growth of energy consumption. It is important to reduce the energy consumption while completing the DL training jobs early in data centers. In this paper, we propose PowerFlow, a GPU clusters scheduler that reduces the average Job Completion Time (JCT) under an energy budget. We first present performance models for DL training jobs to predict the throughput and energy consumption performance with different configurations. Based on the performance models, PowerFlow dynamically allocates GPUs and adjusts the GPU-level or job-level configurations of DL training jobs. PowerFlow applies network packing and buddy allocation to job placement, thus avoiding extra energy consumed by cluster fragmentations. Evaluation results show that under the same energy consumption, PowerFlow improves the average JCT by 1.57 - 3.39 x at most, compared to competitive baselines.

cs.DC↗

arXiv2023

A Soft Coordination Method of Heterogeneous Devices in Distribution System Voltage Control

With the continuous increase of photovoltaic (PV) penetration, the voltage control interactions between newly installed PV inverters and previously deployed on-load tap-changer (OLTC) transformers become ever more significant. To achieve coordinated voltage regulation, current methods often rely on a decision-making algorithm to fully take over the control of all devices, requiring OLTC to give up its existing tap switching logic and execute corresponding upgrades. Aiming at bridging this gap, a soft coordination framework is proposed in this paper. Specifically, the decision-making commands are only applied on inverters, and OLTC that retains its own operation rule will be indirectly controlled by the changed system voltage, which is a result of appropriately adjusting inverters' Var output. The proposed method achieves the soft coordination by establishing a modified actor-critic algorithm to train a proxy model of inverters. The well-trained proxy model can properly adjust inverters' Var output to "softly" coordinate OLTC's tap operations, which finally attains coordinated voltage regulation and line loss minimization. Simulation results verify the superiority of our proposed method over traditional ones in coordinating heterogeneous devices for voltage control.

eess.SY↗

arXiv2023

Personalized Federated Learning on Long-Tailed Data via Adversarial Feature Augmentation

Personalized Federated Learning (PFL) aims to learn personalized models for each client based on the knowledge across all clients in a privacy-preserving manner. Existing PFL methods generally assume that the underlying global data across all clients are uniformly distributed without considering the long-tail distribution. The joint problem of data heterogeneity and long-tail distribution in the FL environment is more challenging and severely affects the performance of personalized models. In this paper, we propose a PFL method called Federated Learning with Adversarial Feature Augmentation (FedAFA) to address this joint problem in PFL. FedAFA optimizes the personalized model for each client by producing a balanced feature set to enhance the local minority classes. The local minority class features are generated by transferring the knowledge from the local majority class features extracted by the global model in an adversarial example learning manner. The experimental results on benchmarks under different settings of data heterogeneity and long-tail distribution demonstrate that FedAFA significantly improves the personalized performance of each client compared with the state-of-the-art PFL algorithm. The code is available at https://github.com/pxqian/FedAFA.

cs.LG↗

arXiv2023

MuxFlow: Efficient and Safe GPU Sharing in Large-Scale Production Deep Learning Clusters

Large-scale GPU clusters are widely-used to speed up both latency-critical (online) and best-effort (offline) deep learning (DL) workloads. However, most DL clusters either dedicate each GPU to one workload or share workloads in time, leading to very low GPU resource utilization. We present MuxFlow, the first production cluster system that supports efficient and safe space-sharing for DL workloads. NVIDIA MPS provides an opportunity to share multiple workloads in space on widely-deployed NVIDIA GPUs, but it cannot guarantee the performance and safety of online workloads. MuxFlow introduces a two-level protection mechanism for memory and computation to guarantee the performance of online workloads. Based on our practical error analysis, we design a mixed error-handling mechanism to guarantee the safety of online workloads. MuxFlow further proposes dynamic streaming multiprocessor (SM) allocation and matching-based scheduling to improve the efficiency of offline workloads. MuxFlow has been deployed at CompanyX's clusters with more than 20,000 GPUs. The deployment results indicate that MuxFlow substantially improves the GPU utilization from 26$\%$ to 76$\%$, SM activity from 16$\%$ to 33$\%$, and GPU memory from 42$\%$ to 48$\%$.

cs.DC↗

arXiv2023

Federated Semi-Supervised Learning with Annotation Heterogeneity

Federated Semi-Supervised Learning (FSSL) aims to learn a global model from different clients in an environment with both labeled and unlabeled data. Most of the existing FSSL work generally assumes that both types of data are available on each client. In this paper, we study a more general problem setup of FSSL with annotation heterogeneity, where each client can hold an arbitrary percentage (0%-100%) of labeled data. To this end, we propose a novel FSSL framework called Heterogeneously Annotated Semi-Supervised LEarning (HASSLE). Specifically, it is a dual-model framework with two models trained separately on labeled and unlabeled data such that it can be simply applied to a client with an arbitrary labeling percentage. Furthermore, a mutual learning strategy called Supervised-Unsupervised Mutual Alignment (SUMA) is proposed for the dual models within HASSLE with global residual alignment and model proximity alignment. Subsequently, the dual models can implicitly learn from both types of data across different clients, although each dual model is only trained locally on a single type of data. Experiments verify that the dual models in HASSLE learned by SUMA can mutually learn from each other, thereby effectively utilizing the information of both types of data across different clients.

cs.LG↗

arXiv2023

PriorLane: A Prior Knowledge Enhanced Lane Detection Approach Based on Transformer

Lane detection is one of the fundamental modules in self-driving. In this paper we employ a transformer-only method for lane detection, thus it could benefit from the blooming development of fully vision transformer and achieve the state-of-the-art (SOTA) performance on both CULane and TuSimple benchmarks, by fine-tuning the weight fully pre-trained on large datasets. More importantly, this paper proposes a novel and general framework called PriorLane, which is used to enhance the segmentation performance of the fully vision transformer by introducing the low-cost local prior knowledge. Specifically, PriorLane utilizes an encoder-only transformer to fuse the feature extracted by a pre-trained segmentation model with prior knowledge embeddings. Note that a Knowledge Embedding Alignment (KEA) module is adapted to enhance the fusion performance by aligning the knowledge embedding. Extensive experiments on our Zjlab dataset show that PriorLane outperforms SOTA lane detection methods by a 2.82% mIoU when prior knowledge is employed, and the code will be released at: https://github.com/vincentqqb/PriorLane.

cs.CV↗

arXiv2023

Characterizing and Detecting WebAssembly Runtime Bugs

WebAssembly (abbreviated WASM) has emerged as a promising language of the Web and also been used for a wide spectrum of software applications such as mobile applications and desktop applications. These applications, named as WASM applications, commonly run in WASM runtimes. Bugs in WASM runtimes are frequently reported by developers and cause the crash of WASM applications. However, these bugs have not been well studied. To fill in the knowledge gap, we present a systematic study to characterize and detect bugs in WASM runtimes. We first harvest a dataset of 311 real-world bugs from hundreds of related posts on GitHub. Based on the collected high-quality bug reports, we distill 31 bug categories of WASM runtimes and summarize their common fix strategies. Furthermore, we develop a pattern-based bug detection framework to automatically detect bugs in WASM runtimes. We apply the detection framework to five popular WASM runtimes and successfully uncover 53 bugs that have never been reported previously, among which 14 have been confirmed and 6 have been fixed by runtime developers.

cs.SE↗

arXiv2022

Compound Batch Normalization for Long-tailed Image Classification

Significant progress has been made in learning image classification neural networks under long-tail data distribution using robust training algorithms such as data re-sampling, re-weighting, and margin adjustment. Those methods, however, ignore the impact of data imbalance on feature normalization. The dominance of majority classes (head classes) in estimating statistics and affine parameters causes internal covariate shifts within less-frequent categories to be overlooked. To alleviate this challenge, we propose a compound batch normalization method based on a Gaussian mixture. It can model the feature space more comprehensively and reduce the dominance of head classes. In addition, a moving average-based expectation maximization (EM) algorithm is employed to estimate the statistical parameters of multiple Gaussian distributions. However, the EM algorithm is sensitive to initialization and can easily become stuck in local minima where the multiple Gaussian components continue to focus on majority classes. To tackle this issue, we developed a dual-path learning framework that employs class-aware split feature normalization to diversify the estimated Gaussian distributions, allowing the Gaussian components to fit with training samples of less-frequent classes more comprehensively. Extensive experiments on commonly used datasets demonstrated that the proposed method outperforms existing methods on long-tailed image classification.

cs.CV↗

arXiv2022

The LCLS-II Gun & Buncher LLRF Controller Upgrade

LCLS-II is currently in its commissioning phase at SLAC. It is an X-ray FEL driven by a CW superconducting LINAC. The beam injector plays a crucial role in the overall performance of the accelerator, and is critical to the final electron beam performance parameters. The LCLS-II injector comprises of a 185.7 MHz VHF copper gun cavity, and a 1.3 GHz two-cell L-band copper buncher cavity. The FPGA-based controller employs feedback and Self-Excited Loop logic in order to regulate the cavity fields. It also features several other functionalities, such as live detune computation, active frequency tracking, and waveform recording. The LLRF system drives the cavities via two 60 kW SSAs through two power couplers, and thus stabilizes the fields inside the plant. This paper provides an outline of the general functionalities of the system, alongside a description of its hardware, firmware and software architecture, before finalizing with the current status of the project and its future goals.

physics.acc-ph↗

arXiv2022

DeFL: Decentralized Weight Aggregation for Cross-silo Federated Learning

Federated learning (FL) is an emerging promising paradigm of privacy-preserving machine learning (ML). An important type of FL is cross-silo FL, which enables a small scale of organizations to cooperatively train a shared model by keeping confidential data locally and aggregating weights on a central parameter server. However, the central server may be vulnerable to malicious attacks or software failures in practice. To address this issue, in this paper, we propose DeFL, a novel decentralized weight aggregation framework for cross-silo FL. DeFL eliminates the central server by aggregating weights on each participating node and weights of only the current training round are maintained and synchronized among all nodes. We use Multi-Krum to enable aggregating correct weights from honest nodes and use HotStuff to ensure the consistency of the training round number and weights among all nodes. Besides, we theoretically analyze the Byzantine fault tolerance, convergence, and complexity of DeFL. We conduct extensive experiments over two widely-adopted public datasets, i.e. CIFAR-10 and Sentiment140, to evaluate the performance of DeFL. Results show that DeFL defends against common threat models with minimal accuracy loss, and achieves up to 100x reduction in storage overhead and up to 12x reduction in network overhead, compared to state-of-the-art decentralized FL approaches.

cs.LG↗

arXiv2022

Federated Neural Architecture Search

To preserve user privacy while enabling mobile intelligence, techniques have been proposed to train deep neural networks on decentralized data. However, training over decentralized data makes the design of neural architecture quite difficult as it already was. Such difficulty is further amplified when designing and deploying different neural architectures for heterogeneous mobile platforms. In this work, we propose an automatic neural architecture search into the decentralized training, as a new DNN training paradigm called Federated Neural Architecture Search, namely federated NAS. To deal with the primary challenge of limited on-client computational and communication resources, we present FedNAS, a highly optimized framework for efficient federated NAS. FedNAS fully exploits the key opportunity of insufficient model candidate re-training during the architecture search process, and incorporates three key optimizations: parallel candidates training on partial clients, early dropping candidates with inferior performance, and dynamic round numbers. Tested on large-scale datasets and typical CNN architectures, FedNAS achieves comparable model accuracy as state-of-the-art NAS algorithm that trains models with centralized data, and also reduces the client cost by up to two orders of magnitude compared to a straightforward design of federated NAS.

cs.LG↗

arXiv2022

Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features

Federated learning (FL) provides a privacy-preserving solution for distributed machine learning tasks. One challenging problem that severely damages the performance of FL models is the co-occurrence of data heterogeneity and long-tail distribution, which frequently appears in real FL applications. In this paper, we reveal an intriguing fact that the biased classifier is the primary factor leading to the poor performance of the global model. Motivated by the above finding, we propose a novel and privacy-preserving FL method for heterogeneous and long-tailed data via Classifier Re-training with Federated Features (CReFF). The classifier re-trained on federated features can produce comparable performance as the one re-trained on real data in a privacy-preserving manner without information leakage of local data or class distribution. Experiments on several benchmark datasets show that the proposed CReFF is an effective solution to obtain a promising FL model under heterogeneous and long-tailed data. Comparative results with the state-of-the-art FL methods also validate the superiority of CReFF. Our code is available at https://github.com/shangxinyi/CReFF-FL.

cs.LG↗

arXiv2021

AliCG: Fine-grained and Evolvable Conceptual Graph Construction for Semantic Search at Alibaba

Conceptual graphs, which is a particular type of Knowledge Graphs, play an essential role in semantic search. Prior conceptual graph construction approaches typically extract high-frequent, coarse-grained, and time-invariant concepts from formal texts. In real applications, however, it is necessary to extract less-frequent, fine-grained, and time-varying conceptual knowledge and build taxonomy in an evolving manner. In this paper, we introduce an approach to implementing and deploying the conceptual graph at Alibaba. Specifically, We propose a framework called AliCG which is capable of a) extracting fine-grained concepts by a novel bootstrapping with alignment consensus approach, b) mining long-tail concepts with a novel low-resource phrase mining approach, c) updating the graph dynamically via a concept distribution estimation method based on implicit and explicit user behaviors. We have deployed the framework at Alibaba UC Browser. Extensive offline evaluation as well as online A/B testing demonstrate the efficacy of our approach.

cs.AI↗

arXiv2021

QubiC: An open source FPGA-based control and measurement system for superconducting quantum information processors

As quantum information processors grow in quantum bit (qubit) count and functionality, the control and measurement system becomes a limiting factor to large scale extensibility. To tackle this challenge and keep pace with rapidly evolving classical control requirements, full control stack access is essential to system level optimization. We design a modular FPGA (field-programmable gate array) based system called QubiC to control and measure a superconducting quantum processing unit. The system includes room temperature electronics hardware, FPGA gateware, and engineering software. A prototype hardware module is assembled from several commercial off-the-shelf evaluation boards and in-house developed circuit boards. Gateware and software are designed to implement basic qubit control and measurement protocols. System functionality and performance are demonstrated by performing qubit chip characterization, gate optimization, and randomized benchmarking sequences on a superconducting quantum processor operating at the Advanced Quantum Testbed at Lawrence Berkeley National Laboratory. The single-qubit and two-qubit process fidelities are measured to be 0.9980$\pm$0.0001 and 0.948$\pm$0.004 by randomized benchmarking. With fast circuit sequence loading capability, the QubiC performs randomized compiling experiments efficiently and improves the feasibility of executing more complex algorithms.

quant-ph↗

arXiv2021

A machine learning model to classify dynamic processes in liquid water

The dynamics of water molecules plays a vital role in understanding water. We combined computer simulation and deep learning to study the dynamics of H-bonds between water molecules. Based on ab initio molecular dynamics simulations and a newly defined directed Hydrogen (H-) bond population operator, we studied a typical dynamic process in bulk water: interchange, in which the H-bond donor reverses roles with the acceptor. By designing a recurrent neural network-based model, we have successfully classified the interchange and breakage processes in water. We have found that the ratio between them is approximately 1:4, and it hardly depends on temperatures from 280 to 360 K. This work implies that deep learning has the great potential to help distinguish complex dynamic processes containing H-bonds in other systems.

cond-mat.dis-nn↗

arXiv2021

Radio frequency mixing modules for superconducting qubit room temperature control systems

As the number of qubits in nascent quantum processing units increases, the connectorized RF (radio frequency) analog circuits used in first generation experiments become exceedingly complex. The physical size, cost and electrical failure rate all become limiting factors in the extensibility of control systems. We have developed a series of compact RF mixing boards to address this challenge by integrating I/Q quadrature mixing, IF(intermediate frequency)/LO(local oscillator)/RF power level adjustments, and DC (direct current) bias fine tuning on a 40 mm $\times $ 80 mm 4-layer PCB (printed circuit board) board with EMI (electromagnetic interference) shielding. The RF mixing module is designed to work with RF and LO frequencies between 2.5 and 8.5 GHz. The typical image rejection and adjacent channel isolation are measured to be $\sim$27 dBc and $\sim$50 dB. By scanning the drive phase in a loopback test, the module short-term amplitude and phase linearity are typically measured to be 5$\times$10$^{-4}$ (V$_{\mathrm{pp}}$/V$_{\mathrm{mean}}$) and 1$\times$10$^{-3}$ radian (pk-pk). The operation of RF mixing board was validated by integrating it into the room temperature control system of a superconducting quantum processor and executing randomized benchmarking characterization of single and two qubit gates. We measured a single-qubit process infidelity of $9.3(3) \times 10^{-4}$ and a two-qubit process infidelity of $2.7(1) \times 10^{-2}$.

quant-ph↗