arXiv subjects

Cong Guo

Publications and source records attributed to Cong Guo.

At least 73 records · Page 4Linked to original sources

arXiv2023

Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design

Graph neural networks (GNNs) have shown significant accuracy improvements in a variety of graph learning domains, sparking considerable research interest. To translate these accuracy improvements into practical applications, it is essential to develop high-performance and efficient hardware acceleration for GNN models. However, designing GNN accelerators faces two fundamental challenges: the high bandwidth requirement of GNN models and the diversity of GNN models. Previous works have addressed the first challenge by using more expensive memory interfaces to achieve higher bandwidth. For the second challenge, existing works either support specific GNN models or have generic designs with poor hardware utilization. In this work, we tackle both challenges simultaneously. First, we identify a new type of partition-level operator fusion, which we utilize to internally reduce the high bandwidth requirement of GNNs. Next, we introduce partition-level multi-threading to schedule the concurrent processing of graph partitions, utilizing different hardware resources. To further reduce the extra on-chip memory required by multi-threading, we propose fine-grained graph partitioning to generate denser graph partitions. Importantly, these three methods make no assumptions about the targeted GNN models, addressing the challenge of model variety. We implement these methods in a framework called SwitchBlade, consisting of a compiler, a graph partitioner, and a hardware accelerator. Our evaluation demonstrates that SwitchBlade achieves an average speedup of $1.85\times$ and energy savings of $19.03\times$ compared to the NVIDIA V100 GPU. Additionally, SwitchBlade delivers performance comparable to state-of-the-art specialized accelerators.

cs.AR↗

arXiv2023

A Class of Smoothing Modulus-Based Iterative Method for Solving Implicit Complementarity Problems

In this paper, a class of smoothing modulus-based iterative method was presented for solving implicit complementarity problems. The main idea was to transform the implicit complementarity problem into an equivalent implicit fixed-point equation, then introduces a smoothing function to obtain its approximation solutions. The convergence analysis of the algorithm was given, and the efficiency of the algorithms was verified by numerical experiment

math.NA↗

arXiv2023

AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs

Graph neural networks (GNNs) are powerful tools for exploring and learning from graph structures and features. As such, achieving high-performance execution for GNNs becomes crucially important. Prior works have proposed to explore the sparsity (i.e., low density) in the input graph to accelerate GNNs, which uses the full-graph-level or block-level sparsity format. We show that they fail to balance the sparsity benefit and kernel execution efficiency. In this paper, we propose a novel system, referred to as AdaptGear, that addresses the challenge of optimizing GNNs performance by leveraging kernels tailored to the density characteristics at the subgraph level. Meanwhile, we also propose a method that dynamically chooses the optimal set of kernels for a given input graph. Our evaluation shows that AdaptGear can achieve a significant performance improvement, up to $6.49 \times$ ($1.87 \times$ on average), over the state-of-the-art works on two mainstream NVIDIA GPUs across various datasets.

cs.DC↗

arXiv2023

Pure Quantum Gradient Descent Algorithm and Full Quantum Variational Eigensolver

Optimization problems are prevalent in various fields, and the gradient-based gradient descent algorithm is a widely adopted optimization method. However, in classical computing, computing the numerical gradient for a function with $d$ variables necessitates at least $d+1$ function evaluations, resulting in a computational complexity of $O(d)$. As the number of variables increases, the classical gradient estimation methods require substantial resources, ultimately surpassing the capabilities of classical computers. Fortunately, leveraging the principles of superposition and entanglement in quantum mechanics, quantum computers can achieve genuine parallel computing, leading to exponential acceleration over classical algorithms in some cases.In this paper, we propose a novel quantum-based gradient calculation method that requires only a single oracle calculation to obtain the numerical gradient result for a multivariate function. The complexity of this algorithm is just $O(1)$. Building upon this approach, we successfully implemented the quantum gradient descent algorithm and applied it to the Variational Quantum Eigensolver (VQE), creating a pure quantum variational optimization algorithm. Compared with classical gradient-based optimization algorithm, this quantum optimization algorithm has remarkable complexity advantages, providing an efficient solution to optimization problems.The proposed quantum-based method shows promise in enhancing the performance of optimization algorithms, highlighting the potential of quantum computing in this field.

quant-ph↗

arXiv2023

OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization

Transformer-based large language models (LLMs) have achieved great success with the growing model size. LLMs' size grows by $240\times$ every two years, which outpaces the hardware progress and makes model inference increasingly costly. Model quantization is a promising approach to mitigate the widening gap between LLM size and hardware capacity. However, the existence of outliers, values with significant magnitudes, in LLMs makes existing quantization methods less effective. Prior outlier-aware quantization schemes adopt sparsity encoding techniques to separate outliers from normal values where the process requires global coordination (e.g., a global sparsity coordination list). This incurs complex encoding/decoding hardware logics and an extra orchestration controller for the computation between outlier and normal values. As such, it is not hardware-efficient and hence only achieves sub-optimal quantization benefits. We propose OliVe, an algorithm/architecture co-designed solution that adopts an outlier-victim pair (OVP) quantization and handles outlier values locally with low hardware overheads and high performance gains. The key insight of OliVe is that outliers are important while the normal values next to them are not. Thus those normal values (called victims) can be sacrificed to accommodate outliers. This enables a memory-aligned OVP encoding scheme, which can be efficiently integrated to the existing hardware accelerators like systolic array and tensor core. As a result, OliVe-based accelerator surpasses the existing outlier-aware accelerator, GOBO, by 4.5$\times$ speedup and 4.0$\times$ energy reduction, respectively, with a superior model accuracy.

cs.AR↗

arXiv2022

Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training

An activation function is an element-wise mathematical function and plays a crucial role in deep neural networks (DNN). Many novel and sophisticated activation functions have been proposed to improve the DNN accuracy but also consume massive memory in the training process with back-propagation. In this study, we propose the nested forward automatic differentiation (Forward-AD), specifically for the element-wise activation function for memory-efficient DNN training. We deploy nested Forward-AD in two widely-used deep learning frameworks, TensorFlow and PyTorch, which support the static and dynamic computation graph, respectively. Our evaluation shows that nested Forward-AD reduces the memory footprint by up to 1.97x than the baseline model and outperforms the recomputation by 20% under the same memory reduction ratio.

cs.LG↗

arXiv2022

ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization

Quantization is a technique to reduce the computation and memory cost of DNN models, which are getting increasingly large. Existing quantization solutions use fixed-point integer or floating-point types, which have limited benefits, as both require more bits to maintain the accuracy of original models. On the other hand, variable-length quantization uses low-bit quantization for normal values and high-precision for a fraction of outlier values. Even though this line of work brings algorithmic benefits, it also introduces significant hardware overheads due to variable-length encoding and decoding. In this work, we propose a fixed-length adaptive numerical data type called ANT to achieve low-bit quantization with tiny hardware overheads. Our data type ANT leverages two key innovations to exploit the intra-tensor and inter-tensor adaptive opportunities in DNN models. First, we propose a particular data type, flint, that combines the advantages of float and int for adapting to the importance of different values within a tensor. Second, we propose an adaptive framework that selects the best type for each tensor according to its distribution characteristics. We design a unified processing element architecture for ANT and show its ease of integration with existing DNN accelerators. Our design results in 2.8$\times$ speedup and 2.5$\times$ energy efficiency improvement over the state-of-the-art quantization accelerators.

cs.LG↗

arXiv2022

SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation

Quantization of deep neural networks (DNN) has been proven effective for compressing and accelerating DNN models. Data-free quantization (DFQ) is a promising approach without the original datasets under privacy-sensitive and confidential scenarios. However, current DFQ solutions degrade accuracy, need synthetic data to calibrate networks, and are time-consuming and costly. This paper proposes an on-the-fly DFQ framework with sub-second quantization time, called SQuant, which can quantize networks on inference-only devices with low computation and memory requirements. With the theoretical analysis of the second-order information of DNN task loss, we decompose and approximate the Hessian-based optimization objective into three diagonal sub-items, which have different areas corresponding to three dimensions of weight tensor: element-wise, kernel-wise, and output channel-wise. Then, we progressively compose sub-items and propose a novel data-free optimization objective in the discrete domain, minimizing Constrained Absolute Sum of Error (or CASE in short), which surprisingly does not need any dataset and is even not aware of network architecture. We also design an efficient algorithm without back-propagation to further reduce the computation complexity of the objective solver. Finally, without fine-tuning and synthetic datasets, SQuant accelerates the data-free quantization process to a sub-second level with >30% accuracy improvement over the existing data-free post-training quantization works, with the evaluated models under 4-bit quantization. We have open-sourced the SQuant framework at https://github.com/clevercool/SQuant.

cs.LG↗

arXiv2021

Characterizing and Demystifying the Implicit Convolution Algorithm on Commercial Matrix-Multiplication Accelerators

Many of today's deep neural network accelerators, e.g., Google's TPU and NVIDIA's tensor core, are built around accelerating the general matrix multiplication (i.e., GEMM). However, supporting convolution on GEMM-based accelerators is not trivial. The naive method explicitly lowers the convolution to GEMM, commonly known as im2col, which introduces significant performance and memory overhead. Existing implicit im2col algorithms require unscalable hardware and are inefficient in supporting important convolution variants such as strided convolution. In this paper, we propose a memory-efficient and hardware-friendly implicit im2col algorithm used by Google's TPU, which dynamically converts a convolution into a GEMM with practically zero performance and memory overhead, fully unleashing the power of GEMM engines. Through comprehensive experimental results, we quantitatively argue that this algorithm has been adopted in commercial closed-source platforms, and we are the first to describe its high-level idea and implementation details. Finally, we show that our algorithm can also be generally applied to Nvidia's Tensor Cores (TC), matching and out-performing the measured performance on TCs.

cs.DC↗

arXiv2021

Dual-side Sparse Tensor Core

Leveraging sparsity in deep neural network (DNN) models is promising for accelerating model inference. Yet existing GPUs can only leverage the sparsity from weights but not activations, which are dynamic, unpredictable, and hence challenging to exploit. In this work, we propose a novel architecture to efficiently harness the dual-side sparsity (i.e., weight and activation sparsity). We take a systematic approach to understand the (dis)advantages of previous sparsity-related architectures and propose a novel, unexplored paradigm that combines outer-product computation primitive and bitmap-based encoding format. We demonstrate the feasibility of our design with minimal changes to the existing production-scale inner-product-based Tensor Core. We propose a set of novel ISA extensions and co-design the matrix-matrix multiplication and convolution algorithms, which are the two dominant computation patterns in today's DNN models, to exploit our new dual-side sparse Tensor Core. Our evaluation shows that our design can fully unleash the dual-side DNN sparsity and improve the performance by up to one order of magnitude with \hl{small} hardware overhead.

cs.AR↗

arXiv2021

Prospects of detecting the reactor $\bar{ν_e}$-Ar coherent elastic scattering with a low threshold dual-phase argon time projection chamber at Taishan

We propose to measure the coherent elastic neutrino nucleus scattering (CE$ν$NS) using a dual-phase liquid argon time projection chamber (TPC) with 200kg fiducial mass. The detector is expected to be adjacent to the JUNO-TAO experiment and to be about 35m from a reactor core with 4.6GW thermal power at Taishan. The antineutrino flux is approximately 6$\times10^{12}$cm$^{-1}$s$^{-1}$ at this location, leading to more than 11,000 coherent scattering events per day in the fiducial mass. However, the nuclear recoil energies concentrate in the sub-keV region, corresponding to less than ten ionisation electrons in the liquid argon. The detection of several ionisation electrons can be achieved in the dual-phase TPC due to the large amplification in the gas region. With a feasible detection threshold of four ionisation electrons, the signal rate is 955 per day. The detector is designed to be shielded well from cosmogenic backgrounds and ambient radioactivities to reach a 16% background-to-signal ratio in the energy region of interest. With the large CE$ν$NS sample, the expected sensitivity of measuring the weak mixing angle $\sin^{2}θ_{W}$, and of limiting the neutrino magnetic moment are discussed. In addition, a synergy between the reactor antineutrino CE$ν$NS experiment and the dark matter experiment is foreseen.

hep-ex↗

arXiv2020

Accelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsity

Network pruning can reduce the high computation cost of deep neural network (DNN) models. However, to maintain their accuracies, sparse models often carry randomly-distributed weights, leading to irregular computations. Consequently, sparse models cannot achieve meaningful speedup on commodity hardware (e.g., GPU) built for dense matrix computations. As such, prior works usually modify or design completely new sparsity-optimized architectures for exploiting sparsity. We propose an algorithm-software co-designed pruning method that achieves latency speedups on existing dense architectures. Our work builds upon the insight that the matrix multiplication generally breaks the large matrix into multiple smaller tiles for parallel execution. We propose a tiling-friendly "tile-wise" sparsity pattern, which maintains a regular pattern at the tile level for efficient execution but allows for irregular, arbitrary pruning at the global scale to maintain the high accuracy. We implement and evaluate the sparsity pattern on GPU tensor core, achieving a 1.95x speedup over the dense model.

cs.DC↗

arXiv2020

Balancing Efficiency and Flexibility for DNN Acceleration via Temporal GPU-Systolic Array Integration

The research interest in specialized hardware accelerators for deep neural networks (DNN) spikes recently owing to their superior performance and efficiency. However, today's DNN accelerators primarily focus on accelerating specific "kernels" such as convolution and matrix multiplication, which are vital but only part of an end-to-end DNN-enabled application. Meaningful speedups over the entire application often require supporting computations that are, while massively parallel, ill-suited to DNN accelerators. Integrating a general-purpose processor such as a CPU or a GPU incurs significant data movement overhead and leads to resource under-utilization on the DNN accelerators. We propose Simultaneous Multi-mode Architecture (SMA), a novel architecture design and execution model that offers general-purpose programmability on DNN accelerators in order to accelerate end-to-end applications. The key to SMA is the temporal integration of the systolic execution model with the GPU-like SIMD execution model. The SMA exploits the common components shared between the systolic-array accelerator and the GPU, and provides lightweight reconfiguration capability to switch between the two modes in-situ. The SMA achieves up to 63% performance improvement while consuming 23% less energy than the baseline Volta architecture with TensorCore.

cs.DC↗

arXiv2020

The liquid argon detector and measurement of SiPM array at liquid argon temperature

Particle detectors based on liquid argon (LAr) have recently become recognized as an extremely attractive technology for the direct detection of dark matter as well as the measurement of coherent elastic neutrino-nucleus scattering (CE$ν$NS). The Chinese argon group at Institute of High Energy Physics has been studying the LAr detector technology and a LAr detector has been operating steadily. A program of using a dual phase LAr detector to measure the CE$ν$NS at Taishang Nuclear Power Plant has been proposed and the R\&D work is ongoing. Considering the requirements of ultra-low radio-purity and high photon collection efficiency, SiPMs will be a good choice and will be used in the detector. In this proceeding, an introduction of the LAr detector and the measurement results of SiPM array at LAr temperature will be presented.

physics.ins-det↗

arXiv2020

Developing the radium measurement system for the water Cherenkov detector of the Jiangmen Underground Neutrino Observatory

The Jiangmen Underground Neutrino Observatory is proposed to determine neutrino mass hierarchy using a 20~ktonne liquid scintillator detector. Strict radio-purity requirements have been put forward for all the components of the detector. According to the MC simulation results, the radon dissolved in the water Cherenkov detector should be below 200~mBq/m$^3$. Radium, the progenitor of radon, should also be taken seriously into account. In order to measure the radium concentration in water, a radium measurement system, which consists of a radium extraction system, a radon emanation chamber and a radon concentration measurement system, has been developed. In this paper, the updated radon concentration in gas measurement system with a one-day-measurement sensitivity of $\sim$5~mBq/m$^3$, the detail of the development of the radium concentration in water measurement system with a sensitivity of $\sim$23~mBq/m$^3$ as well as the measurement results of Daya Bay water samples will be presented.

astro-ph.IM↗

arXiv2019

Status of the Jiangmen Underground Neutrino Observatory

The Jiangmen Underground Neutrino Observatory is a multipurpose neutrino experiment designed to determine neutrino mass hierarchy and precisely measure oscillation parameters by detecting reactor neutrinos from the Yangjiang and Taishan Nuclear Power Plants, observe supernova neutrinos, study the atmospheric, solar neutrinos and geo-neutrinos, and perform exotic searches, with a 20-thousand-ton liquid scintillator detector of unprecedented 3\% energy resolution (at 1 MeV) at 700-meter deep underground. In this proceeding, the subsystems of the experiment, including the cental detector, the online scintillator internal radioactivity investigation system, the PMT, the veto detector, the calibration system and the taishan antineutrino observatory, will be described. The construction is expected to be completed in 2021.

physics.ins-det↗

arXiv2019

Calibration of liquid argon detector with $^{83m}Kr$ and $^{22}Na$ in different drift field

$^{83m}Kr$ and $^{22}Na$ have been used in calibrating a liquid argon (LAr) detector.$^{83m}Kr$ atoms are produced through the decay of $^{83}Rb$ and introduced into the LAr detector through the circulating purification system. The light yield reaches 7.26$\pm$0.02 photonelectrons/keV for 41.5keV from $^{83m}Kr$ and 7.66$\pm$0.01 photonelectrons/keV for the 511keV from $^{22}Na$, as a comparison. The light yield varies with the drift electric field from 50 to 200V/cm have been also reported. After stopping fill, the decay rate of $^{83m}Kr$ with a fitted half-life of 1.83$\pm$0.11 h, which is consistent with the reported value of 1.83$\pm$0.02 h.

physics.ins-det↗

arXiv2019

Adversarial Defense Through Network Profiling Based Path Extraction

Recently, researchers have started decomposing deep neural network models according to their semantics or functions. Recent work has shown the effectiveness of decomposed functional blocks for defending adversarial attacks, which add small input perturbation to the input image to fool the DNN models. This work proposes a profiling-based method to decompose the DNN models to different functional blocks, which lead to the effective path as a new approach to exploring DNNs' internal organization. Specifically, the per-image effective path can be aggregated to the class-level effective path, through which we observe that adversarial images activate effective path different from normal images. We propose an effective path similarity-based method to detect adversarial images with an interpretable model, which achieve better accuracy and broader applicability than the state-of-the-art technique.

cs.LG↗