arXiv subjects

Song Han

Publications and source records attributed to Song Han.

At least 181 records · Page 10Linked to original sources

arXiv2023

A Fast Algorithm for Source-wise Round-trip Spanners

In this paper, we study the problem of fast constructions of source-wise round-trip spanners in weighted directed graphs. For a source vertex set $S\subseteq V$ in a graph $G(V,E)$, an $S$-sourcewise round-trip spanner of $G$ of stretch $k$ is a subgraph $H$ of $G$ such that for every pair of vertices $u,v\in S\times V$, their round-trip distance in $H$ is at most $k$ times of their round-trip distance in $G$. We show that for a graph $G(V,E)$ with $n$ vertices and $m$ edges, an $s$-sized source vertex set $S\subseteq V$ and an integer $k>1$, there exists an algorithm that in time $O(ms^{1/k}\log^5n)$ constructs an $S$-sourcewise round-trip spanner of stretch $O(k\log n)$ and $O(ns^{1/k}\log^2n)$ edges with high probability. Compared to the fast algorithms for constructing all-pairs round-trip spanners \cite{PRS+18,CLR+20}, our algorithm improve the running time and the number of edges in the spanner when $k$ is super-constant. Compared with the existing algorithm for constructing source-wise round-trip spanners \cite{ZL17}, our algorithm significantly improves their construction time $Ω(\min\{ms,n^ω\})$ (where $ω\in [2,2.373)$ and 2.373 is the matrix multiplication exponent) to nearly linear $O(ms^{1/k}\log^5n)$, at the expense of paying an extra $O(\log n)$ in the stretch. As an important building block of the algorithm, we develop a graph partitioning algorithm to partition $G$ into clusters of bounded radius and prove that for every $u,v\in S\times V$ at small round-trip distance, the probability of separating them in different clusters is small. The algorithm takes the size of $S$ as input and does not need the knowledge of $S$. With the algorithm and a reachability vertex size estimation algorithm, we show that the recursive algorithm for constructing standard round-trip spanners \cite{PRS+18} can be adapted to the source-wise setting.

cs.DS↗

arXiv2023

Offsite-Tuning: Transfer Learning without Full Model

Transfer learning is important for foundation models to adapt to downstream tasks. However, many foundation models are proprietary, so users must share their data with model owners to fine-tune the models, which is costly and raise privacy concerns. Moreover, fine-tuning large foundation models is computation-intensive and impractical for most downstream users. In this paper, we propose Offsite-Tuning, a privacy-preserving and efficient transfer learning framework that can adapt billion-parameter foundation models to downstream data without access to the full model. In offsite-tuning, the model owner sends a light-weight adapter and a lossy compressed emulator to the data owner, who then fine-tunes the adapter on the downstream data with the emulator's assistance. The fine-tuned adapter is then returned to the model owner, who plugs it into the full model to create an adapted foundation model. Offsite-tuning preserves both parties' privacy and is computationally more efficient than the existing fine-tuning methods that require access to the full model weights. We demonstrate the effectiveness of offsite-tuning on various large language and vision foundation models. Offsite-tuning can achieve comparable accuracy as full model fine-tuning while being privacy-preserving and efficient, achieving 6.5x speedup and 5.6x memory reduction. Code is available at https://github.com/mit-han-lab/offsite-tuning.

cs.CL↗

arXiv2022

Hybrid Gate-Pulse Model for Variational Quantum Algorithms

Current quantum programs are mostly synthesized and compiled on the gate-level, where quantum circuits are composed of quantum gates. The gate-level workflow, however, introduces significant redundancy when quantum gates are eventually transformed into control signals and applied on quantum devices. For superconducting quantum computers, the control signals are microwave pulses. Therefore, pulse-level optimization has gained more attention from researchers due to their advantages in terms of circuit duration. Recent works, however, are limited by their poor scalability brought by the large parameter space of control signals. In addition, the lack of gate-level "knowledge" also affects the performance of pure pulse-level frameworks. We present a hybrid gate-pulse model that can mitigate these problems. We propose to use gate-level compilation and optimization for "fixed" part of the quantum circuits and to use pulse-level methods for problem-agnostic parts. Experimental results demonstrate the efficiency of the proposed framework in discrete optimization tasks. We achieve a performance boost at most 8% with 60% shorter pulse duration in the problem-agnostic layer.

quant-ph↗

arXiv2022

Composite Resource Scheduling for Networked Control Systems

Real-time end-to-end task scheduling in networked control systems (NCSs) requires the joint consideration of both network and computing resources to guarantee the desired quality of service (QoS). This paper introduces a new model for composite resource scheduling (CRS) in real-time networked control systems, which considers a strict execution order of sensing, computing, and actuating segments based on the control loop of the target NCS. We prove that the general CRS problem is NP-hard and study two special cases of the CRS problem. The first case restricts the computing and actuating segments to have unit-size execution time while the second case assumes that both sensing and actuating segments have unit-size execution time. We propose an optimal algorithm to solve the first case by checking the intervals with 100% network resource utilization and modify the deadlines of the tasks within those intervals to prune the search. For the second case, we propose another optimal algorithm based on a novel backtracking strategy to check the time intervals with the network resource utilization larger than 100% and modify the timing parameters of tasks based on these intervals. For the general case, we design a greedy strategy to modify the timing parameters of both network segments and computing segments within the time intervals that have network and computing resource utilization larger than 100%, respectively. The correctness and effectiveness of the proposed algorithms are verified through extensive experiments.

cs.NI↗

arXiv2022

TopGen: Topology-Aware Bottom-Up Generator for Variational Quantum Circuits

Variational Quantum Algorithms (VQA) are promising to demonstrate quantum advantages on near-term devices. Designing ansatz, a variational circuit with parameterized gates, is of paramount importance for VQA as it lays the foundation for parameter optimizations. Due to the large noise on Noisy-Intermediate Scale Quantum (NISQ) machines, considering circuit size and real device noise in the ansatz design process is necessary. Unfortunately, recent works on ansatz design either consider no noise impact or only treat the real device as a black box with no specific noise information. In this work, we propose to open the black box by designing specific ansatz tailored for the qubit topology on target machines. Specifically, we propose a bottom-up approach to generate topology-specific ansatz. Firstly, we generate topology-compatible sub-circuits with desirable properties such as high expressibility and entangling capability. Then, the sub-circuits are combined together to form an initial ansatz. We further propose circuits stitching to solve the sparse connectivity issue between sub-circuits, and dynamic circuit growing to improve the accuracy. The ansatz constructed with this method is highly flexible and thus we can explore a much larger design space than previous state-of-the-art method in which all ansatz candidates are strict subsets of a pre-defined large ansatz. We use a popular VQA algorithm - Quantum Neural Networks (QNN) for Machine Learning (ML) task as the benchmarks. Experiments on 14 ML tasks show that under the same performance, the TopGen-searched ansatz can reduce the circuit depth and the number of CNOT gates by up to 2 * and 4 * respectively. Experiments on three real quantum machines demonstrate on average 17% accuracy improvements over baselines.

quant-ph↗

arXiv2022

Demonstration of broadband topological slow light

Slow-light devices are able to significantly enhance light-matter interaction due to the reduced group velocity of light, but a very low group velocity is usually achieved in a narrow bandwidth, accompanied by extreme sensitivity to imperfections that causes increased disorder-induced attenuation. Recent theories have suggested an ideal solution to this problem - unidirectional chiral photonic states, previously discovered in structures known as photonic topological insulators, not only resist backscattering from imperfections but can also be slowed down in the entire topological bandgap with multiple windings in the Brillouin zone. Here, we report on the experimental demonstration of broadband topological slow light in a photonic topological insulator. When coupled with periodic resonators that form flat bands, the chiral photonic states can wind many times around the Brillouin zone, achieving an ultra-low group velocity in the entire topological bandgap. This demonstration extends the scope of topological photonics into slow light engineering and opens a unique avenue in the dispersion manipulation of chiral photonic states.

physics.app-ph↗

arXiv2022

Variational Quantum Pulse Learning

Quantum computing is among the most promising emerging techniques to solve problems that are computationally intractable on classical hardware. A large body of existing works focus on using variational quantum algorithms on the gate level for machine learning tasks, such as the variational quantum circuit (VQC). However, VQC has limited flexibility and expressibility due to limited number of parameters, e.g. only one parameter can be trained in one rotation gate. On the other hand, we observe that quantum pulses are lower than quantum gates in the stack of quantum computing and offers more control parameters. Inspired by the promising performance of VQC, in this paper we propose variational quantum pulses (VQP), a novel paradigm to directly train quantum pulses for learning tasks. The proposed method manipulates variational quantum pulses by pulling and pushing the amplitudes of pulses in an optimization framework. Similar to variational quantum algorithms, our framework to train pulses maintains the robustness to noise on Noisy Intermediate-Scale Quantum (NISQ) computers. In an example task of binary classification, VQP learning achieves up to 11% and 9% higher accuracy compared with VQC learning on the qiskit noise simulators (with noise model from real machine) and ibmq-jarkata, respectively, demonstrating its effectiveness and feasibility. Stability for VQP to obtain reliable results has also been verified in the presence of noise.

quant-ph↗

arXiv2022

RobustAnalog: Fast Variation-Aware Analog Circuit Design Via Multi-task RL

Analog/mixed-signal circuit design is one of the most complex and time-consuming stages in the whole chip design process. Due to various process, voltage, and temperature (PVT) variations from chip manufacturing, analog circuits inevitably suffer from performance degradation. Although there has been plenty of work on automating analog circuit design under the typical condition, limited research has been done on exploring robust designs under real and unpredictable silicon variations. Automatic analog design against variations requires prohibitive computation and time costs. To address the challenge, we present RobustAnalog, a robust circuit design framework that involves the variation information in the optimization process. Specifically, circuit optimizations under different variations are considered as a set of tasks. Similarities among tasks are leveraged and competitions are alleviated to realize a sample-efficient multi-task training. Moreover, RobustAnalog prunes the task space according to the current performance in each iteration, leading to a further simulation cost reduction. In this way, RobustAnalog can rapidly produce a set of circuit parameters that satisfies diverse constraints (e.g. gain, bandwidth, noise...) across variations. We compare RobustAnalog with Bayesian optimization, Evolutionary algorithm, and Deep Deterministic Policy Gradient (DDPG) and demonstrate that RobustAnalog can significantly reduce required optimization time by 14-30 times. Therefore, our study provides a feasible method to handle various real silicon conditions.

cs.ET↗

arXiv2022

Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation

Pose estimation plays a critical role in human-centered vision applications. However, it is difficult to deploy state-of-the-art HRNet-based pose estimation models on resource-constrained edge devices due to the high computational cost (more than 150 GMACs per frame). In this paper, we study efficient architecture design for real-time multi-person pose estimation on edge. We reveal that HRNet's high-resolution branches are redundant for models at the low-computation region via our gradual shrinking experiments. Removing them improves both efficiency and performance. Inspired by this finding, we design LitePose, an efficient single-branch architecture for pose estimation, and introduce two simple approaches to enhance the capacity of LitePose, including Fusion Deconv Head and Large Kernel Convs. Fusion Deconv Head removes the redundancy in high-resolution branches, allowing scale-aware feature fusion with low overhead. Large Kernel Convs significantly improve the model's capacity and receptive field while maintaining a low computational cost. With only 25% computation increment, 7x7 kernels achieve +14.0 mAP better than 3x3 kernels on the CrowdPose dataset. On mobile platforms, LitePose reduces the latency by up to 5.0x without sacrificing performance, compared with prior state-of-the-art efficient pose estimation models, pushing the frontier of real-time multi-person pose estimation on edge. Our code and pre-trained models are released at https://github.com/mit-han-lab/litepose.

cs.CV↗

arXiv2022

MME-CRS: Multi-Metric Evaluation Based on Correlation Re-Scaling for Evaluating Open-Domain Dialogue

Automatic open-domain dialogue evaluation is a crucial component of dialogue systems. Recently, learning-based evaluation metrics have achieved state-of-the-art performance in open-domain dialogue evaluation. However, these metrics, which only focus on a few qualities, are hard to evaluate dialogue comprehensively. Furthermore, these metrics lack an effective score composition approach for diverse evaluation qualities. To address the above problems, we propose a Multi-Metric Evaluation based on Correlation Re-Scaling (MME-CRS) for evaluating open-domain dialogue. Firstly, we build an evaluation metric composed of 5 groups of parallel sub-metrics called Multi-Metric Evaluation (MME) to evaluate the quality of dialogue comprehensively. Furthermore, we propose a novel score composition method called Correlation Re-Scaling (CRS) to model the relationship between sub-metrics and diverse qualities. Our approach MME-CRS ranks first on the final test data of DSTC10 track5 subtask1 Automatic Open-domain Dialogue Evaluation Challenge with a large margin, which proved the effectiveness of our proposed approach.

cs.CL↗

arXiv2022

PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution

3D neural networks are widely used in real-world applications (e.g., AR/VR headsets, self-driving cars). They are required to be fast and accurate; however, limited hardware resources on edge devices make these requirements rather challenging. Previous work processes 3D data using either voxel-based or point-based neural networks, but both types of 3D models are not hardware-efficient due to the large memory footprint and random memory access. In this paper, we study 3D deep learning from the efficiency perspective. We first systematically analyze the bottlenecks of previous 3D methods. We then combine the best from point-based and voxel-based models together and propose a novel hardware-efficient 3D primitive, Point-Voxel Convolution (PVConv). We further enhance this primitive with the sparse convolution to make it more effective in processing large (outdoor) scenes. Based on our designed 3D primitive, we introduce 3D Neural Architecture Search (3D-NAS) to explore the best 3D network architecture given a resource constraint. We evaluate our proposed method on six representative benchmark datasets, achieving state-of-the-art performance with 1.8-23.7x measured speedup. Furthermore, our method has been deployed to the autonomous racing vehicle of MIT Driverless, achieving larger detection range, higher accuracy and lower latency.

cs.CV↗

arXiv2022

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech recognition. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that are able to lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand in order to enable numerous edge AI applications. This paper provides an overview of efficient deep learning methods, systems and applications. We start from introducing popular model compression methods, including pruning, factorization, quantization as well as compact model design. To reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization. We then cover efficient on-device training to enable user customization based on the local data on mobile devices. Apart from general acceleration techniques, we also showcase several task-specific accelerations for point cloud, video and natural language processing by exploiting their spatial sparsity and temporal/token redundancy. Finally, to support all these algorithmic advancements, we introduce the efficient deep learning system design from both software and hardware perspectives.

cs.LG↗

arXiv2022

Network Augmentation for Tiny Deep Learning

We introduce Network Augmentation (NetAug), a new training method for improving the performance of tiny neural networks. Existing regularization techniques (e.g., data augmentation, dropout) have shown much success on large neural networks by adding noise to overcome over-fitting. However, we found these techniques hurt the performance of tiny neural networks. We argue that training tiny models are different from large models: rather than augmenting the data, we should augment the model, since tiny models tend to suffer from under-fitting rather than over-fitting due to limited capacity. To alleviate this issue, NetAug augments the network (reverse dropout) instead of inserting noise into the dataset or the network. It puts the tiny model into larger models and encourages it to work as a sub-model of larger models to get extra supervision, in addition to functioning as an independent model. At test time, only the tiny model is used for inference, incurring zero inference overhead. We demonstrate the effectiveness of NetAug on image classification and object detection. NetAug consistently improves the performance of tiny models, achieving up to 2.2% accuracy improvement on ImageNet. On object detection, achieving the same level of performance, NetAug requires 41% fewer MACs on Pascal VOC and 38% fewer MACs on COCO than the baseline.

cs.CV↗

arXiv2022

TorchSparse: Efficient Point Cloud Inference Engine

Deep learning on point clouds has received increased attention thanks to its wide applications in AR/VR and autonomous driving. These applications require low latency and high accuracy to provide real-time user experience and ensure user safety. Unlike conventional dense workloads, the sparse and irregular nature of point clouds poses severe challenges to running sparse CNNs efficiently on the general-purpose hardware. Furthermore, existing sparse acceleration techniques for 2D images do not translate to 3D point clouds. In this paper, we introduce TorchSparse, a high-performance point cloud inference engine that accelerates the sparse convolution computation on GPUs. TorchSparse directly optimizes the two bottlenecks of sparse convolution: irregular computation and data movement. It applies adaptive matrix multiplication grouping to trade computation for better regularity, achieving 1.4-1.5x speedup for matrix multiplication. It also optimizes the data movement by adopting vectorized, quantized and fused locality-aware memory access, reducing the memory movement cost by 2.7x. Evaluated on seven representative models across three benchmark datasets, TorchSparse achieves 1.6x and 1.5x measured end-to-end speedup over the state-of-the-art MinkowskiEngine and SpConv, respectively.

cs.LG↗

arXiv2022

RT-WiFi on Software-Defined Radio: Design and Implementation

Applying high-speed real-time wireless technologies in industrial applications has the great potential to reduce the deployment and maintenance costs compared to their wired counterparts. Wireless technologies enhance the mobility and reduce the communication jitter and delay for mobile industrial equipment, such as mobile collaborative robots. Unfortunately, most existing wireless solutions employed in industrial fields either cannot support the desired high-speed communications or cannot guarantee deterministic, real-time performance. A more recent wireless technology, RT-WiFi, achieves a good balance between high-speed data rates and deterministic communication performance. It is however developed on commercial-of-the-shelf (COTS) hardware, and takes considerable effort and hardware expertise to maintain and upgrade. To address these problems, this paper introduces the software-defined radio (SDR)-based RT-WiFi solution which we call SRT-WiFi. SRT-WiFi provides full-stack configurability for high-speed real-time wireless communications. We present the overall system architecture of SRT-WiFi and discuss its key functions which achieve better timing performance and solve the queue management and rate adaptation issues compared to COTS hardware-based RT-WiFi. To achieve effective network management with rate adaptation in multi-cluster SRT-WiFi, a novel scheduling problem is formulated and an effective algorithm is proposed to solve the problem. A multi-cluster SRT-WiFi testbed is developed to validate the design, and extensive experiments are performed to evaluate the performance at both device and system levels.

cs.NI↗

arXiv2022

QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits

Quantum noise is the key challenge in Noisy Intermediate-Scale Quantum (NISQ) computers. Previous work for mitigating noise has primarily focused on gate-level or pulse-level noise-adaptive compilation. However, limited research efforts have explored a higher level of optimization by making the quantum circuits themselves resilient to noise. We propose QuantumNAS, a comprehensive framework for noise-adaptive co-search of the variational circuit and qubit mapping. Variational quantum circuits are a promising approach for constructing QML and quantum simulation. However, finding the best variational circuit and its optimal parameters is challenging due to the large design space and parameter training cost. We propose to decouple the circuit search and parameter training by introducing a novel SuperCircuit. The SuperCircuit is constructed with multiple layers of pre-defined parameterized gates and trained by iteratively sampling and updating the parameter subsets (SubCircuits) of it. It provides an accurate estimation of SubCircuits performance trained from scratch. Then we perform an evolutionary co-search of SubCircuit and its qubit mapping. The SubCircuit performance is estimated with parameters inherited from SuperCircuit and simulated with real device noise models. Finally, we perform iterative gate pruning and finetuning to remove redundant gates. Extensively evaluated with 12 QML and VQE benchmarks on 14 quantum computers, QuantumNAS significantly outperforms baselines. For QML, QuantumNAS is the first to demonstrate over 95% 2-class, 85% 4-class, and 32% 10-class classification accuracy on real QC. It also achieves the lowest eigenvalue for VQE tasks on H2, H2O, LiH, CH4, BeH2 compared with UCCSD. We also open-source TorchQuantum (https://github.com/mit-han-lab/torchquantum) for fast training of parameterized quantum circuits to facilitate future research.

quant-ph↗

arXiv2021

AET-SGD: Asynchronous Event-triggered Stochastic Gradient Descent

Communication cost is the main bottleneck for the design of effective distributed learning algorithms. Recently, event-triggered techniques have been proposed to reduce the exchanged information among compute nodes and thus alleviate the communication cost. However, most existing event-triggered approaches only consider heuristic event-triggered thresholds. They also ignore the impact of computation and network delay, which play an important role on the training performance. In this paper, we propose an Asynchronous Event-triggered Stochastic Gradient Descent (SGD) framework, called AET-SGD, to i) reduce the communication cost among the compute nodes, and ii) mitigate the impact of the delay. Compared with baseline event-triggered methods, AET-SGD employs a linear increasing sample size event-triggered threshold, and can significantly reduce the communication cost while keeping good convergence performance. We implement AET-SGD and evaluate its performance on multiple representative data sets, including MNIST, FashionMNIST, KMNIST and CIFAR10. The experimental results validate the correctness of the design and show a significant communication cost reduction from 44x to 120x, compared to the state of the art. Our results also show that AET-SGD can resist large delay from the straggler nodes while obtaining a decent performance and a desired speedup ratio.

cs.LG↗

arXiv2021

VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles

Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios. However, the poor photorealism and lack of diverse sensor modalities of existing simulation engines remain key hurdles towards realizing this potential. Here, we present VISTA, an open source, data-driven simulator that integrates multiple types of sensors for autonomous vehicles. Using high fidelity, real-world datasets, VISTA represents and simulates RGB cameras, 3D LiDAR, and event-based cameras, enabling the rapid generation of novel viewpoints in simulation and thereby enriching the data available for policy learning with corner cases that are difficult to capture in the physical world. Using VISTA, we demonstrate the ability to train and test perception-to-control policies across each of the sensor types and showcase the power of this approach via deployment on a full scale autonomous vehicle. The policies learned in VISTA exhibit sim-to-real transfer without modification and greater robustness than those trained exclusively on real-world data.

cs.RO↗