arXiv subjects

Peng Zhao

Publications and source records attributed to Peng Zhao.

At least 109 records · Page 6Linked to original sources

arXiv2024

High Performance Im2win and Direct Convolutions using Three Tensor Layouts on SIMD Architectures

Convolution is the core component within deep neural networks and it is computationally intensive and time consuming. Tensor data layouts significantly impact convolution operations in terms of memory access and computational efficiency. Yet, there is still a lack of comprehensive performance characterization on data layouts on SIMD architectures concerning convolution methods. This paper proposes three novel data layouts for im2win convolution: NHWC, CHWN, and CHWN8, and introduces a set of general optimization techniques for both direct and im2win convolutions. We compare the optimized im2win convolution with the direct convolution and PyTorch's im2col-based convolution across the aforementioned layouts on SIMD machines. The experiments demonstrated that the im2win convolution with the new NHWC layout achieved up to 355% performance speedup over NCHW layout. Our optimizations also significantly improve the performance of both im2win and direct convolutions. Our optimized im2win and direct convolutions achieved up to 95% and 94% of machine's theoretical peak performance, respectively.

cs.LG↗

arXiv2024

Bayesian Nowcasting Data Breach IBNR Incidents

The reporting delay in data breach incidents poses a formidable challenge for Incurred But Not Reported (IBNR) studies, complicating reserve estimation for actuarial professionals. This work presents a novel Bayesian nowcasting model designed to accurately model and predict the number of IBNR data breach incidents. Leveraging a Bayesian modeling framework, the model integrates time and heterogeneous effects to enhance predictive accuracy. Synthetic and empirical studies demonstrate the superior performance of the proposed model, highlighting its efficacy in addressing the complexities of IBNR estimation. Furthermore, we examine reserve estimation for IBNR incidents using the proposed model, shedding light on its implications for actuarial practice.

stat.AP↗

arXiv2024

Notes on sums over horizons

For a large class of black holes, we show that the sum of thermodynamic quantities over all the horizons is determined by the asymptotic data at infinity. For the Kerr-Newman metric, this proves a recent numerical observation by Hristov. We propose a new method to compute the sum of the entropies using a generalized Smarr formula. The higher-curvature corrections in Gauss-Bonnet gravity are discussed.

gr-qc↗

arXiv2024

Factorized Fusion Shrinkage for Dynamic Relational Data

Modern data science applications often involve complex relational data with dynamic structures. An abrupt change in such dynamic relational data is typically observed in systems that undergo regime changes due to interventions. In such a case, we consider a factorized fusion shrinkage model in which all decomposed factors are dynamically shrunk towards group-wise fusion structures, where the shrinkage is obtained by applying global-local shrinkage priors to the successive differences of the row vectors of the factorized matrices. The proposed priors enjoy many favorable properties in comparison and clustering of the estimated dynamic latent factors. Comparing estimated latent factors involves both adjacent and long-term comparisons, with the time range of comparison considered as a variable. Under certain conditions, we demonstrate that the posterior distribution attains the minimax optimal rate up to logarithmic factors. In terms of computation, we present a structured mean-field variational inference framework that balances optimal posterior inference with computational scalability, exploiting both the dependence among components and across time. The framework can accommodate a wide variety of models, including dynamic matrix factorization, latent space models for networks and low-rank tensors. The effectiveness of our methodology is demonstrated through extensive simulations and real-world data analysis.

stat.ME↗

arXiv2024

Efficient LLM inference solution on Intel GPU

Transformer based Large Language Models (LLMs) have been widely used in many fields, and the efficiency of LLM inference becomes hot topic in real applications. However, LLMs are usually complicatedly designed in model structure with massive operations and perform inference in the auto-regressive mode, making it a challenging task to design a system with high efficiency. In this paper, we propose an efficient LLM inference solution with low latency and high throughput. Firstly, we simplify the LLM decoder layer by fusing data movement and element-wise operations to reduce the memory access frequency and lower system latency. We also propose a segment KV cache policy to keep key/value of the request and response tokens in separate physical memory for effective device memory management, helping enlarge the runtime batch size and improve system throughput. A customized Scaled-Dot-Product-Attention kernel is designed to match our fusion policy based on the segment KV cache solution. We implement our LLM inference solution on Intel GPU and publish it publicly. Compared with the standard HuggingFace implementation, the proposed solution achieves up to 7x lower token latency and 27x higher throughput for some popular LLMs on Intel GPU.

cs.AR↗

arXiv2024

Additive Manufacturing of functionalised atomic vapour cells for next-generation quantum technologies

Atomic vapour cells are an indispensable tool for quantum technologies (QT), but potential improvements are limited by the capacities of conventional manufacturing methods. Using an additive manufacturing (AM) technique - vat polymerisation by digital light processing - we demonstrate, for the first time, a 3D-printed glass vapour cell. The exploitation of AM capacities allows intricate internal architectures, overprinting of 2D optoelectronical materials to create integrated sensors and surface functionalisation, while also showing the ability to tailor the optical properties of the AM glass by in-situ growth of gold nanoparticles. The produced cells achieve ultra-high vacuum of $2 \times 10^{-9}$ mbar and enable Doppler-free spectroscopy; we demonstrate laser frequency stabilisation as a QT application. These results highlight the transformative role that AM can play for QT in enabling compact, optimised and integrated multi-material components and devices.

physics.atom-ph↗

arXiv2024

Direct generation of multi-photon hyperentanglement

Multi-photon hyperentangement is of fundamental importance in optical quantum information processing. Existing theory and experiment producing multi-photon hyperentangled states have until now relied on the outcome post-selection, a procedure where only the measurement results corresponding to the desired state are considered. Such approach severely limits the usefulness of the resulting hyperentangled states. We present the protocols of direct production of three- and four-photon hyperentanglement and extend the approach to an arbitrary number of photons through a straightforward cascade of spontaneous parametric down-conversion (SPDC) sources. The generated multi-photon hyperentangled states are encoded in polarization-spatial modes and polarization-time bin degrees of freedom, respectively. Numerical calculation shows that if the average photon number $μ$ is set to 1, the down conversion efficiency is $7.6*10^{-6}$ and the repetition frequency of the laser is $10^9$ Hz, the number of the generation of three-photon and four-photon hyperentanglement after cascading can reach about $5.78*10^{-2}$ and $4.44*10^{-7}$ pairs per second, respectively. By eliminating the constraints of outcome post-selection, our protocols may represent important progresses for multi-photon hyperentangement generation and providing a pivotal role in future multi-party and high-capacity communication networks.

quant-ph↗

arXiv2024

EdgeSync: Faster Edge-model Updating via Adaptive Continuous Learning for Video Data Drift

Real-time video analytics systems typically place models with fewer weights on edge devices to reduce latency. The distribution of video content features may change over time for various reasons (i.e. light and weather change) , leading to accuracy degradation of existing models, to solve this problem, recent work proposes a framework that uses a remote server to continually train and adapt the lightweight model at edge with the help of complex model. However, existing analytics approaches leave two challenges untouched: firstly, retraining task is compute-intensive, resulting in large model update delays; secondly, new model may not fit well enough with the data distribution of the current video stream. To address these challenges, in this paper, we present EdgeSync, EdgeSync filters the samples by considering both timeliness and inference results to make training samples more relevant to the current video content as well as reduce the update delay, to improve the quality of training, EdgeSync also designs a training management module that can efficiently adjusts the model training time and training order on the runtime. By evaluating real datasets with complex scenes, our method improves about 3.4% compared to existing methods and about 10% compared to traditional means.

cs.CV↗

arXiv2024

Exploratory Machine Learning with Unknown Unknowns

In conventional supervised learning, a training dataset is given with ground-truth labels from a known label set, and the learned model will classify unseen instances to known labels. This paper studies a new problem setting in which there are unknown classes in the training data misperceived as other labels, and thus their existence appears unknown from the given supervision. We attribute the unknown unknowns to the fact that the training dataset is badly advised by the incompletely perceived label space due to the insufficient feature information. To this end, we propose the exploratory machine learning, which examines and investigates training data by actively augmenting the feature space to discover potentially hidden classes. Our method consists of three ingredients including rejection model, feature exploration, and model cascade. We provide theoretical analysis to justify its superiority, and validate the effectiveness on both synthetic and real datasets.

cs.LG↗

arXiv2024

Universal Online Convex Optimization with $1$ Projection per Round

To address the uncertainty in function types, recent progress in online convex optimization (OCO) has spurred the development of universal algorithms that simultaneously attain minimax rates for multiple types of convex functions. However, for a $T$-round online problem, state-of-the-art methods typically conduct $O(\log T)$ projections onto the domain in each round, a process potentially time-consuming with complicated feasible sets. In this paper, inspired by the black-box reduction of Cutkosky and Orabona (2018), we employ a surrogate loss defined over simpler domains to develop universal OCO algorithms that only require $1$ projection. Embracing the framework of prediction with expert advice, we maintain a set of experts for each type of functions and aggregate their predictions via a meta-algorithm. The crux of our approach lies in a uniquely designed expert-loss for strongly convex functions, stemming from an innovative decomposition of the regret into the meta-regret and the expert-regret. Our analysis sheds new light on the surrogate loss, facilitating a rigorous examination of the discrepancy between the regret of the original loss and that of the surrogate loss, and carefully controlling meta-regret under the strong convexity condition. In this way, with only $1$ projection per round, we establish optimal regret bounds for general convex, exponentially concave, and strongly convex functions simultaneously. Furthermore, we enhance the expert-loss to exploit the smoothness property, and demonstrate that our algorithm can attain small-loss regret for multiple types of convex and smooth functions.

cs.LG↗

arXiv2024

Adapting to Continuous Covariate Shift via Online Density Ratio Estimation

Dealing with distribution shifts is one of the central challenges for modern machine learning. One fundamental situation is the covariate shift, where the input distributions of data change from training to testing stages while the input-conditional output distribution remains unchanged. In this paper, we initiate the study of a more challenging scenario -- continuous covariate shift -- in which the test data appear sequentially, and their distributions can shift continuously. Our goal is to adaptively train the predictor such that its prediction risk accumulated over time can be minimized. Starting with the importance-weighted learning, we show the method works effectively if the time-varying density ratios of test and train inputs can be accurately estimated. However, existing density ratio estimation methods would fail due to data scarcity at each time step. To this end, we propose an online method that can appropriately reuse historical information. Our density ratio estimation method is proven to perform well by enjoying a dynamic regret bound, which finally leads to an excess risk guarantee for the predictor. Empirical results also validate the effectiveness.

cs.LG↗

arXiv2024

A Space/Time Interchange Symmetry of Rotating AdS Black Holes in General Dimensions

We revisit the previously known local inversion symmetry of the five-dimensional Kerr-AdS metric that relates the over-rotating black hole to the under-rotating one and reinterpret it as an interchanging symmetry between time and the longitudinal angular coordinates. We generalize this to all $D$ dimensions, including $D=4$, thereby enlarging the trivial linear $\mathbb Z_N$ symmetry of the $N=\lfloor(D-1)/2\rfloor$ longitudinal angular coordinates to the nonlinearly realized $\mathbb Z_{N+1}$ symmetry that involves time.

hep-th↗

arXiv2024

Cluster Alphabets from Generalized Worldsheets: A Geometric Approach to Finite Types

We provide a systematic derivation of cluster alphabets of finite types. The construction is based on a geometric realization of the generalized worldsheets by gluing and folding a pair of polygons. The cross ratios of the worldsheet z variables are evolved using the Y-system equations. By a new gauge choice, we obtain a simpler set of cluster alphabets than the known ones.

hep-th↗

arXiv2024

Utilizing Deep Learning to Optimize Software Development Processes

This study explores the application of deep learning technologies in software development processes, particularly in automating code reviews, error prediction, and test generation to enhance code quality and development efficiency. Through a series of empirical studies, experimental groups using deep learning tools and control groups using traditional methods were compared in terms of code error rates and project completion times. The results demonstrated significant improvements in the experimental group, validating the effectiveness of deep learning technologies. The research also discusses potential optimization points, methodologies, and technical challenges of deep learning in software development, as well as how to integrate these technologies into existing software development workflows.

cs.SE↗

arXiv2024

Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classification

With wearing masks becoming a new cultural norm, facial expression recognition (FER) while taking masks into account has become a significant challenge. In this paper, we propose a unified multi-branch vision transformer for facial expression recognition and mask wearing classification tasks. Our approach extracts shared features for both tasks using a dual-branch architecture that obtains multi-scale feature representations. Furthermore, we propose a cross-task fusion phase that processes tokens for each task with separate branches, while exchanging information using a cross attention module. Our proposed framework reduces the overall complexity compared with using separate networks for both tasks by the simple yet effective cross-task fusion phase. Extensive experiments demonstrate that our proposed model performs better than or on par with different state-of-the-art methods on both facial expression recognition and facial mask wearing classification task.

cs.CV↗

arXiv2024

Universal Online Learning with Gradient Variations: A Multi-layer Online Ensemble Approach

In this paper, we propose an online convex optimization approach with two different levels of adaptivity. On a higher level, our approach is agnostic to the unknown types and curvatures of the online functions, while at a lower level, it can exploit the unknown niceness of the environments and attain problem-dependent guarantees. Specifically, we obtain $\mathcal{O}(\log V_T)$, $\mathcal{O}(d \log V_T)$ and $\hat{\mathcal{O}}(\sqrt{V_T})$ regret bounds for strongly convex, exp-concave and convex loss functions, respectively, where $d$ is the dimension, $V_T$ denotes problem-dependent gradient variations and the $\hat{\mathcal{O}}(\cdot)$-notation omits $\log V_T$ factors. Our result not only safeguards the worst-case guarantees but also directly implies the small-loss bounds in analysis. Moreover, when applied to adversarial/stochastic convex optimization and game theory problems, our result enhances the existing universal guarantees. Our approach is based on a multi-layer online ensemble framework incorporating novel ingredients, including a carefully designed optimism for unifying diverse function types and cascaded corrections for algorithmic stability. Notably, despite its multi-layer structure, our algorithm necessitates only one gradient query per round, making it favorable when the gradient evaluation is time-consuming. This is facilitated by a novel regret decomposition equipped with carefully designed surrogate losses.

cs.LG↗

arXiv2024

Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimization

We investigate online convex optimization in non-stationary environments and choose dynamic regret as the performance measure, defined as the difference between cumulative loss incurred by the online algorithm and that of any feasible comparator sequence. Let $T$ be the time horizon and $P_T$ be the path length that essentially reflects the non-stationarity of environments, the state-of-the-art dynamic regret is $\mathcal{O}(\sqrt{T(1+P_T)})$. Although this bound is proved to be minimax optimal for convex functions, in this paper, we demonstrate that it is possible to further enhance the guarantee for some easy problem instances, particularly when online functions are smooth. Specifically, we introduce novel online algorithms that can exploit smoothness and replace the dependence on $T$ in dynamic regret with problem-dependent quantities: the variation in gradients of loss functions, the cumulative loss of the comparator sequence, and the minimum of these two terms. These quantities are at most $\mathcal{O}(T)$ while could be much smaller in benign environments. Therefore, our results are adaptive to the intrinsic difficulty of the problem, since the bounds are tighter than existing results for easy problems and meanwhile safeguard the same rate in the worst case. Notably, our proposed algorithms can achieve favorable dynamic regret with only one gradient per iteration, sharing the same gradient query complexity as the static regret minimization methods. To accomplish this, we introduce the collaborative online ensemble framework. The proposed framework employs a two-layer online ensemble to handle non-stationarity, and uses optimistic online learning and further introduces crucial correction terms to enable effective collaboration within the meta-base two layers, thereby attaining adaptivity. We believe the framework can be useful for broader problems.

cs.LG↗

arXiv2024

A multiplexed control architecture for superconducting qubits with row-column addressing

In state-of-the-art superconducting quantum processors, each qubit is controlled by at least one control line that delivers control pulses generated at room temperature to qubits operating at millikelvin temperatures. While this strategy has been successfully applied to control hundreds of qubits, it is unlikely to be scalable to control thousands of qubits, let alone millions or even billions of qubits needed in fault-tolerance quantum computing. The primary obstacle lies in the wiring challenge, wherein the number of accommodated control lines is limited by factors, such as the cooling power, physical space of the cryogenic system, the control footprint area at the qubit chip level, and so on. Here, we introduce a multiplexed control architecture for superconducting qubits with two types of shared control lines, row and column lines, providing an efficient approach for parallel controlling $N$ qubits with $O(\sqrt{N})$ control lines. With the combination of the two-type shared lines, unique pairs of control pulses are delivered to qubits at each row-column intersection, enabling parallel qubit addressing. Of particular concern here is that, unlike traditional gate schemes, both single- and two-qubit gates are implemented with pairs of control pulses. Considering the inherent parallelism and the control limitations, the integration of the architecture into quantum computing systems should be tailored as much as possible to the specific properties of the quantum circuits to be executed. As such, the architecture could be scalable for executing structured quantum circuits, such as quantum error correction circuits.

quant-ph↗