arXiv Science⌕ Search

arXiv subjects

Zhe Wang

Publications and source records attributed to Zhe Wang.

At least 361 records · Page 20Linked to original sources
arXiv2023

Conditional Perceptual Quality Preserving Image Compression

We propose conditional perceptual quality, an extension of the perceptual quality defined in \citet{blau2018perception}, by conditioning it on user defined information. Specifically, we extend the original perceptual quality $d(p_{X},p_{\hat{X}})$ to the conditional perceptual quality $d(p_{X|Y},p_{\hat{X}|Y})$, where $X$ is the original image, $\hat{X}$ is the reconstructed, $Y$ is side information defined by user and $d(.,.)$ is divergence. We show that conditional perceptual quality has similar theoretical properties as rate-distortion-perception trade-off \citep{blau2019rethinking}. Based on these theoretical results, we propose an optimal framework for conditional perceptual quality preserving compression. Experimental results show that our codec successfully maintains high perceptual quality and semantic quality at all bitrate. Besides, by providing a lowerbound of common randomness required, we settle the previous arguments on whether randomness should be incorporated into generator for (conditional) perceptual quality compression. The source code is provided in supplementary material.

eess.IV↗
arXiv2023

Multiple antiferromagnetic phases and magnetic anisotropy in exfoliated CrBr$_3$ multilayers

In twisted two-dimensional (2D) magnets, the stacking dependence of the magnetic exchange interaction can lead to regions of ferromagnetic and antiferromagnetic interlayer order, separated by non-collinear, skyrmion-like spin textures. Recent experimental searches for these textures have focused on CrI$_3$, known to exhibit either ferromagnetic or antiferromagnetic interlayer order, depending on layer stacking. However, the very strong uniaxial anisotropy of CrI$_3$ disfavors smooth non-collinear phases in twisted bilayers. Here, we report the experimental observation of three distinct magnetic phases -- one ferromagnetic and two antiferromagnetic -- in exfoliated CrBr$_3$ multilayers, and reveal that the uniaxial anisotropy is significantly smaller than in CrI$_3$. These results are obtained by magnetoconductance measurements on CrBr$_3$ tunnel barriers and Raman spectroscopy, in conjunction with density functional theory calculations, which enable us to identify the stackings responsible for the different interlayer magnetic couplings. The detection of all locally stable magnetic states predicted to exist in CrBr$_3$ and the excellent agreement found between theory and experiments, provide complete information on the stacking-dependent interlayer exchange energy and establish twisted bilayer CrBr$_3$ as an ideal system to deterministically create non-collinear magnetic phases.

cond-mat.mtrl-sci↗
arXiv2023

Deep Context Interest Network for Click-Through Rate Prediction

Click-Through Rate (CTR) prediction, estimating the probability of a user clicking on an item, is essential in industrial applications, such as online advertising. Many works focus on user behavior modeling to improve CTR prediction performance. However, most of those methods only model users' positive interests from users' click items while ignoring the context information, which is the display items around the clicks, resulting in inferior performance. In this paper, we highlight the importance of context information on user behavior modeling and propose a novel model named Deep Context Interest Network (DCIN), which integrally models the click and its display context to learn users' context-aware interests. DCIN consists of three key modules: 1) Position-aware Context Aggregation Module (PCAM), which performs aggregation of display items with an attention mechanism; 2) Feedback-Context Fusion Module (FCFM), which fuses the representation of clicks and display contexts through non-linear feature interaction; 3) Interest Matching Module (IMM), which activates interests related with the target item. Moreover, we provide our hands-on solution to implement our DCIN model on large-scale industrial systems. The significant improvements in both offline and online evaluations demonstrate the superiority of our proposed DCIN method. Notably, DCIN has been deployed on our online advertising system serving the main traffic, which brings 1.5% CTR and 1.5% RPM lift.

cs.IR↗
arXiv2023

Phased Deep Spatio-temporal Learning for Highway Traffic Volume Prediction

Inter-city highway transportation is significant for citizens' modern urban life and generates heterogeneous sensory data with spatio-temporal characteristics. As a routine analysis in transportation domain, daily traffic volume estimation faces challenges for highway toll stations including lacking of exploration of correlative spatio-temporal features from a long-term perspective and effective means to deal with data imbalance which always deteriorates the predictive performance. In this paper, a deep spatio-temporal learning method is proposed to predict daily traffic volume in three phases. In feature pre-processing phase, data is normalized elaborately according to latent long-tail distribution. In spatio-temporal learning phase, a hybrid model is employed combining fully convolution network (FCN) and long short-term memory (LSTM), which considers time, space, meteorology, and calendar from heterogeneous data. In decision phase, traffic volumes on a coming day at network-wide toll stations would be achieved effectively, which is especially calibrated for vital few highway stations. Using real-world data from one Chinese provincial highway, extensive experiments show our method has distinct improvement for predictive accuracy than various traditional models, reaching 5.269 and 0.997 in MPAE and R-squre metrics, respectively.

cs.LG↗
arXiv2023

Delete: Deep Lead Optimization Enveloped in Protein Pocket through Unified Deleting Strategies and a Structure-aware Network

Drug discovery is a highly complicated process, and it is unfeasible to fully commit it to the recently developed molecular generation methods. Deep learning-based lead optimization takes expert knowledge as a starting point, learning from numerous historical cases about how to modify the structure for better drug-forming properties. However, compared with the more established de novo generation schemes, lead optimization is still an area that requires further exploration. Previously developed models are often limited to resolving one (or few) certain subtask(s) of lead optimization, and most of them can only generate the two-dimensional structures of molecules while disregarding the vital protein-ligand interactions based on the three-dimensional binding poses. To address these challenges, we present a novel tool for lead optimization, named Delete (Deep lead optimization enveloped in protein pocket). Our model can handle all subtasks of lead optimization involving fragment growing, linking, and replacement through a unified deleting (masking) strategy, and is aware of the intricate pocket-ligand interactions through the geometric design of networks. Statistical evaluations and case studies conducted on individual subtasks demonstrate that Delete has a significant ability to produce molecules with superior binding affinities to protein targets and reasonable drug-likeness from given fragments or atoms. This feature may assist medicinal chemists in developing not only me-too/me-better products from existing drugs but also hit-to-lead for first-in-class drugs in a highly efficient manner.

q-bio.BM↗
arXiv2023

Reduced Scaling Real-Time Coupled Cluster Theory

Real-time coupled cluster (CC) methods have several advantages over their frequency-domain counterparts, namely, response and equation of motion CC theories. Broadband spectra, strong fields, and pulse manipulation allow for the simulation of complex spectroscopies which are unreachable using frequency-domain approaches. Due to the high-order polynomial scaling, the required numerical time-propagation of the CC residual expressions is a computationally demanding process. This scaling may be reduced by local correlation schemes, which aim to reduce the size of the (virtual) orbital space by truncating it according to user-defined parameters. We present the first application of local correlation to real-time CC. As in previous studies of locally correlated frequency-domain CC, traditional local correlation schemes are of limited utility for field-dependent properties; however, a perturbation-aware scheme proves promising. A detailed analysis of the amplitude dynamics suggests the main challenge is a strong time-dependence of the wave function sparsity.

physics.chem-ph↗
arXiv2023

Optical property measurements of lithium chloride aqueous solution for a novel solar neutrino experiment

The lithium chloride aqueous solution has great potential to be the detection medium of a novel solar neutrino detector. The nuclide \ce{^7 Li} provides a charged-current interaction channel with a high cross-section for the MeV-scale solar electron-neutrinos, enabling measurement of the solar neutrino spectrum. This work measures the optical properties and the light yields of a saturated lithium chloride solution. After adsorption with activated carbon and recrystallization, the solution shows little absorption in the sensitive wavelength range of the bialkali photomultipliers. The attenuation length is evaluated to reach 50 meters at 430 nm. In addition to being a pure Cherenkov detector medium, a wavelength shifter, carbostyril 124, is added to the LiCl aqueous solution. The compatibility and the enhancement of the light yield are confirmed, enabling the development of a water-based Cherenkov-enhanced lithium-rich detector.

physics.ins-det↗
arXiv2023

Multi-Scale And Token Mergence: Make Your ViT More Efficient

Since its inception, Vision Transformer (ViT) has emerged as a prevalent model in the computer vision domain. Nonetheless, the multi-head self-attention (MHSA) mechanism in ViT is computationally expensive due to its calculation of relationships among all tokens. Although some techniques mitigate computational overhead by discarding tokens, this also results in the loss of potential information from those tokens. To tackle these issues, we propose a novel token pruning method that retains information from non-crucial tokens by merging them with more crucial tokens, thereby mitigating the impact of pruning on model performance. Crucial and non-crucial tokens are identified by their importance scores and merged based on similarity scores. Furthermore, multi-scale features are exploited to represent images, which are fused prior to token pruning to produce richer feature representations. Importantly, our method can be seamlessly integrated with various ViTs, enhancing their adaptability. Experimental evidence substantiates the efficacy of our approach in reducing the influence of token pruning on model performance. For instance, on the ImageNet dataset, it achieves a remarkable 33% reduction in computational costs while only incurring a 0.1% decrease in accuracy on DeiT-S.

cs.CV↗
arXiv2023

Evaluation and Control Model Design of Human Factors for Autonomous Driving Systems

With the fast development of driving automation technologies, user psychological acceptance of driving automation has become one of the major obstacles to the adoption of the driving automation technology. The most basic function of a passenger car is to transport passengers or drivers to their destinations safely and comfortably. Thus, the design of the driving automation should not just guarantee the safety of vehicle operation but also ensure occupant subjective level of comfort. Hence this paper proposes a local path planning algorithm for obstacle avoidance with occupant subjective feelings considered. Firstly, turning and obstacle avoidance conditions are designed, and four classifiers in machine learning are used to respectively establish subjective and objective evaluation models that link the objective vehicle dynamics parameters and occupant subjective confidence. Then, two potential fields are established based on the artificial potential field, reflecting the psychological feeling of drivers on obstacles and road boundaries. Accordingly, a path planning algorithm and a path tracking algorithm are designed respectively based on model predictive control, and the psychological safety boundary and the optimal classifier are used as part of cost functions. Finally, co-simulations of MATLAB/Simulink and CarSim are carried out. The results confirm the effectiveness of the proposed control algorithm, which can avoid obstacles satisfactorily and improve the psychological feeling of occupants effectively.

cs.RO↗
arXiv2023

Transformer with Selective Shuffled Position Embedding and Key-Patch Exchange Strategy for Early Detection of Knee Osteoarthritis

Knee OsteoArthritis (KOA) is a widespread musculoskeletal disorder that can severely impact the mobility of older individuals. Insufficient medical data presents a significant obstacle for effectively training models due to the high cost associated with data labelling. Currently, deep learning-based models extensively utilize data augmentation techniques to improve their generalization ability and alleviate overfitting. However, conventional data augmentation techniques are primarily based on the original data and fail to introduce substantial diversity to the dataset. In this paper, we propose a novel approach based on the Vision Transformer (ViT) model with original Selective Shuffled Position Embedding (SSPE) and key-patch exchange strategies to obtain different input sequences as a method of data augmentation for early detection of KOA (KL-0 vs KL-2). More specifically, we fix and shuffle the position embedding of key and non-key patches, respectively. Then, for the target image, we randomly select other candidate images from the training set to exchange their key patches and thus obtain different input sequences. Finally, a hybrid loss function is developed by incorporating multiple loss functions for different types of the sequences. According to the experimental results, the generated data are considered valid as they lead to a notable improvement in the model's classification performance.

eess.IV↗
arXiv2023

An Efficient Sparse Inference Software Accelerator for Transformer-based Language Models on CPUs

In recent years, Transformer-based language models have become the standard approach for natural language processing tasks. However, stringent throughput and latency requirements in industrial applications are limiting their adoption. To mitigate the gap, model compression techniques such as structured pruning are being used to improve inference efficiency. However, most existing neural network inference runtimes lack adequate support for structured sparsity. In this paper, we propose an efficient sparse deep learning inference software stack for Transformer-based language models where the weights are pruned with constant block size. Our sparse software accelerator leverages Intel Deep Learning Boost to maximize the performance of sparse matrix - dense matrix multiplication (commonly abbreviated as SpMM) on CPUs. Our SpMM kernel outperforms the existing sparse libraries (oneMKL, TVM, and LIBXSMM) by an order of magnitude on a wide range of GEMM shapes under 5 representative sparsity ratios (70%, 75%, 80%, 85%, 90%). Moreover, our SpMM kernel shows up to 5x speedup over dense GEMM kernel of oneDNN, a well-optimized dense library widely used in industry. We apply our sparse accelerator on widely-used Transformer-based language models including Bert-Mini, DistilBERT, Bert-Base, and BERT-Large. Our sparse inference software shows up to 1.5x speedup over Neural Magic's Deepsparse under same configurations on Xeon on Amazon Web Services under proxy production latency constraints. We also compare our solution with two framework-based inference solutions, ONNX Runtime and PyTorch, and demonstrate up to 37x speedup over ONNX Runtime and 345x over PyTorch on Xeon under the latency constraints. All the source code is publicly available on Github: https://github.com/intel/intel-extension-for-transformers.

cs.LG↗
arXiv2023

Highly Accurate Prediction of NMR Chemical Shifts from Low-Level Quantum Mechanics Calculations Using Machine Learning

Theoretical predictions of NMR chemical shifts from first-principles can greatly facilitate experimental interpretation and structure identification. However, accurate prediction of chemical shifts using the best coupled cluster methods can be prohibitively expensive for systems larger than ten to twenty non-hydrogen atoms on today's computers. By contrast machine learning methods offer inexpensive alternatives but are hampered by generalization to molecules outside the original training set. Here we propose a novel machine learning feature representation informed by intermediate calculations of atomic chemical shielding tensors within a molecular environment using an inexpensive quantum mechanics method, and training it to predict NMR chemical shieldings of a high-level composite theory that is comparable to CCSD(T) in the complete basis set limit. The inexpensive shift machine learning (iShiftML) algorithm is trained through a new progressive active learning workflow that reduces the total number of expensive calculations required when constructing the dataset, while allowing the model to continuously improve on data it has never seen. Furthermore, we show that the error estimations from our model correlate quite well with actual errors to provide confidence values on new predictions. We illustrate the predictive capacity of iShiftML across gas phase experimental chemical shifts for small organic molecules and much larger and more complex natural products in which we can accurately differentiate between subtle diastereomers based on chemical shift assignments.

physics.chem-ph↗
arXiv2023

Federated Learning-based Vehicle Trajectory Prediction against Cyberattacks

With the development of the Internet of Vehicles (IoV), vehicle wireless communication poses serious cybersecurity challenges. Faulty information, such as fake vehicle positions and speeds sent by surrounding vehicles, could cause vehicle collisions, traffic jams, and even casualties. Additionally, private vehicle data leakages, such as vehicle trajectory and user account information, may damage user property and security. Therefore, achieving a cyberattack-defense scheme in the IoV system with faulty data saturation is necessary. This paper proposes a Federated Learning-based Vehicle Trajectory Prediction Algorithm against Cyberattacks (FL-TP) to address the above problems. The FL-TP is intensively trained and tested using a publicly available Vehicular Reference Misbehavior (VeReMi) dataset with five types of cyberattacks: constant, constant offset, random, random offset, and eventual stop. The results show that the proposed FL-TP algorithm can improve cyberattack detection and trajectory prediction by up to 6.99% and 54.86%, respectively, under the maximum cyberattack permeability scenarios compared with benchmark methods.

cs.CR↗
arXiv2023

Progression Cognition Reinforcement Learning with Prioritized Experience for Multi-Vehicle Pursuit

Multi-vehicle pursuit (MVP) such as autonomous police vehicles pursuing suspects is important but very challenging due to its mission and safety critical nature. While multi-agent reinforcement learning (MARL) algorithms have been proposed for MVP problem in structured grid-pattern roads, the existing algorithms use randomly training samples in centralized learning, which leads to homogeneous agents showing low collaboration performance. For the more challenging problem of pursuing multiple evading vehicles, these algorithms typically select a fixed target evading vehicle for pursuing vehicles without considering dynamic traffic situation, which significantly reduces pursuing success rate. To address the above problems, this paper proposes a Progression Cognition Reinforcement Learning with Prioritized Experience for MVP (PEPCRL-MVP) in urban multi-intersection dynamic traffic scenes. PEPCRL-MVP uses a prioritization network to assess the transitions in the global experience replay buffer according to the parameters of each MARL agent. With the personalized and prioritized experience set selected via the prioritization network, diversity is introduced to the learning process of MARL, which can improve collaboration and task related performance. Furthermore, PEPCRL-MVP employs an attention module to extract critical features from complex urban traffic environments. These features are used to develop progression cognition method to adaptively group pursuing vehicles. Each group efficiently target one evading vehicle in dynamic driving environments. Extensive experiments conducted with a simulator over unstructured roads of an urban area show that PEPCRL-MVP is superior to other state-of-the-art methods. Specifically, PEPCRL-MVP improves pursuing efficiency by 3.95% over TD3-DMAP and its success rate is 34.78% higher than that of MADDPG. Codes are open sourced.

cs.AI↗
arXiv2023

Performance of the 1-ton Prototype Neutrino Detector at CJPL-I

China Jinping Underground Laboratory provides an ideal site for solar, geo-, and supernova neutrino studies. With a prototype neutrino detector running since 2017, containing 1-ton liquid scintillator, we tested its experimental hardware, performed the detector calibration and simulation, and measured its radioactive backgrounds, as an early stage of the Jinping Neutrino Experiment (JNE). We investigated the radon background and implemented the nitrogen sealing technology to control it. This paper presents the details of these studies and will serve as a key reference for the construction and optimization of the future large detector of JNE.

hep-ex↗
arXiv2023

Variational Bihamiltonian Cohomologies and Integrable Hierarchies III: Linear Reciprocal Transformations

For an integrable hierarchy which possesses a bihamiltonian structure with semisimple hydrodynamic limit, we prove that the linear reciprocal transformation with respect to any of its symmetry transforms it to another bihamiltonian integrable hierarchy. Moreover, we show that the central invariants of the bihamiltonian structure are preserved under such a linear reciprocal transformation.

nlin.SI↗
arXiv2023

On-chip arrayed waveguide grating fabricated on thin film lithium niobate

We design an on-chip 8-channel TFLN AWG and fabricate the device using photolithography assisted chemo-mechanical etching (PLACE) technique. We experimentally measure the transmission of the fabricated TFLN AWG near the central wavelength of 1550 nm. We obtain an on-chip loss as low as 3.32 dB, a single-channel bandwidth of 1.6 nm and a total-channel bandwidth of 12.8 nm. The crosstalk between adjacent channels was measured to be below -7.01 dB within the wavelength range from 1543 nm to 1558 nm, and the crosstalk between non-adjacent channels was below -15 dB.

physics.optics↗
arXiv2023

Ultrafast dynamics of optically excited charge carriers in the room-temperature antiferromagnetic semiconductor $α$-MnTe

We report on time-resolved optical and terahertz ultrafast spectroscopy of charge-carrier dynamics in the room-temperature antiferromagnetic semiconductor $α$-MnTe. By optically pumping the system with 1.55 eV photons at room temperature, we excite charge carriers in the conduction band through the indirect band gap and investigate the dynamical response of nonequilibrium states using optical as well as terahertz transmission probes. Three relaxation processes are revealed by their characteristic relaxation times of the order of 1, 10, and 100~ps, whose exact values are functions of the pump fluence. For high pump fluences nonlinear dependence on the pump fluence is observed both in the optical and terahertz probes.

cond-mat.mtrl-sci↗