arXiv subjects

Jie Zhu

Publications and source records attributed to Jie Zhu.

At least 145 records · Page 8Linked to original sources

arXiv2023

E-CRF: Embedded Conditional Random Field for Boundary-caused Class Weights Confusion in Semantic Segmentation

Modern semantic segmentation methods devote much effect to adjusting image feature representations to improve the segmentation performance in various ways, such as architecture design, attention mechnism, etc. However, almost all those methods neglect the particularity of class weights (in the classification layer) in segmentation models. In this paper, we notice that the class weights of categories that tend to share many adjacent boundary pixels lack discrimination, thereby limiting the performance. We call this issue Boundary-caused Class Weights Confusion (BCWC). We try to focus on this problem and propose a novel method named Embedded Conditional Random Field (E-CRF) to alleviate it. E-CRF innovatively fuses the CRF into the CNN network as an organic whole for more effective end-to-end optimization. The reasons are two folds. It utilizes CRF to guide the message passing between pixels in high-level features to purify the feature representation of boundary pixels, with the help of inner pixels belonging to the same object. More importantly, it enables optimizing class weights from both scale and direction during backpropagation. We make detailed theoretical analysis to prove it. Besides, superpixel is integrated into E-CRF and served as an auxiliary to exploit the local object prior for more reliable message passing. Finally, our proposed method yields impressive results on ADE20K, Cityscapes, and Pascal Context datasets.

cs.CV↗

arXiv2023

Numerical Framework for Modeling Quantum Electromagnetic Systems Involving Finite-Sized Lossy Dielectric Objects in Free Space

The modified Langevin noise formalism has been proposed for the correct charaterization of quantum electromagnetic fields in the presence of finite-sized lossy dielectric objects in free space. The main modification to the original one (also known as the Green's function approach available only for bulk inhomogeneous lossy dielectric medium) was to add fluctuating sources in reaction to the radiation loss. Consequently, a resulting electric field operator is now determined by (i) boundary-assisted and (ii) medium-assisted fields on an equal footing, which are fluctuating sources due to radiation and medium losses, respectively. However, due to the lengthy mathematical manipulation and complicated concepts, the validity of the modified Langevin noise formalism has not been clearly checked yet. In this work, we propose and develop a novel numerical framework for the modified Langevin noise formalism by exploiting computational electromagnetic methods (CEM). Specifically, we utilize the finite-element method to numerically solve plane-wave-scattering and point-source-radiation problems whose solutions are boundary-assisted and medium-assisted fields, respectively. Based on the developed numerical framework, we calculate the Purcell factor of a two-level atom inside or outside a lossy dielectric slab. It is numerically proved, for the first time, that one can retrieve the conventional expression of the spontaneous emission rate, viz., the imaginary part of the Green's function. The proposed numerical framework is particularly useful for estimating the dynamics of multi-level atoms near practical plasmonic structures or metasurfaces.

quant-ph↗

arXiv2023

Generalized Helmholtz Decomposition for Modal Analysis of Electromagnetic Problems in Inhomogeneous Media

Potential-based formulation with generalized Lorenz gauge can be used in the quantization of electromagnetic fields in inhomogeneous media. However, one often faces the redundancy of modes when finding eigenmodes from potential-based formulation. In free space, this can be explained by the connection to the well-known Helmholtz decomposition. In this work, we generalize the Helmholtz decomposition to its generalized form, echoing the use of generalized Lorenz gauge in inhomogeneous media. We formulate electromagnetics eigenvalue problems using vector potential formulation which is often used in numerical quantization. The properties of the differential operators are mathematically analyzed. Orthogonality relations between the two classes of modes are proved in both continuous and discrete space. Completeness of two sets of modes and the orthogonality relations are numerically validated in inhomogeneous anisotropic media. This work serves as a foundation for numerical quantization of electromagnetic fields in inhomogeneous media with potential-based formulation.

physics.optics↗

arXiv2022

Transient non-Hermitian skin effect

The discovery of non-Hermitian skin effect (NHSE) has opened an exciting direction for unveiling unusual physics and phenomena in non-Hermitian system. Despite notable theoretical breakthroughs, actual observation of NHSE's whole evolvement, however, relies mainly on gain medium to provide amplified mode. It typically impedes the development of simple, robust system. Here, we show that a passive system is fully capable of supporting the observation of the complete evolution picture of NHSE, without the need of any gain medium. With a simple lattice model and acoustic ring resonators, we use complex-frequency excitation to create virtual gain effect, and experimentally demonstrate that exact NHSE can persist in a totally passive system during a quasi-stationary stage. This results in the transient NHSE: passive construction of NHSE in a short time window. Despite the general energy decay, the localization character of skin modes can still be clearly witnessed and successfully exploited. Our findings unveil the importance of excitation in realizing NHSE and paves the way towards studying the peculiar features of non-Hermitian physics with diverse passive platforms.

cond-mat.mtrl-sci↗

arXiv2022

Anomalous Floquet non-Hermitian skin effect in a ring resonator lattice

We present a one-dimensional coupled ring resonator lattice exhibiting a variant of the non- Hermitian skin effect (NHSE) that we call the anomalous Floquet NHSE. Unlike existing approaches to achieving the NHSE by engineering gain and loss on different ring segments, our design uses fixed on-site gain or loss in each ring. The anomalous Floquet NHSE is marked by the existence of skin modes at every value of the Floquet quasienergy, allowing for broadband asymmetric transmission. Varying the gain/loss induces a non-Hermitian topological phase transition, reversing the localization direction of the skin modes. An experimental implementation in an acoustic lattice yields good agreement with theoretical predictions, with a very broad relative bandwidth of around 40%.

cond-mat.mes-hall↗

arXiv2022

Acoustic mirror Chern insulator with projective parity-time symmetry

In condensed matter physics, symmetry profoundly governs the fundamentals of topological matter. The emergence of new topological phase is typically linked to the enrichment of symmetries. Different parity-time symmetry relations distinguish between spinless and spinful physical systems. In spinless systems, creating pseudo-spins can realize fragile topological phase but not break the time-reversal symmetry. Therefore, growing attentions were recently focused on the strong topological phase in spinless systems. Here we break the framework of crystallographic symmetry groups by utilizing the projective symmetry gauge field, to realize the Mirror Chern Insulator in a bilayer twisted Hofstadter model. In experiments, the edge modes were unambiguously observed with odd-shaped boundaries, confirming the strong topological features.The clockwise and anti-clockwise edge states with opposite group velocities were completely separated via an energy drain. In addition, we demonstrate that MCI has robust topological whispering gallery modes. Our work establishes a strong foundation for investigating exotic topological effects arising from the interplays between artificial gauge fields and wave systems. The clockwise and anti-clockwise edge states with opposite group velocities were completely separated via an energy drain. In addition, we demonstrate that MCI has robust topological whispering gallery modes. Our work establishes a strong foundation for investigating exotic topological effects arising from the interplays between artificial gauge fields and wave systems.

cond-mat.mes-hall↗

arXiv2022

Safety and Performance, Why not Both? Bi-Objective Optimized Model Compression toward AI Software Deployment

The size of deep learning models in artificial intelligence (AI) software is increasing rapidly, which hinders the large-scale deployment on resource-restricted devices (e.g., smartphones). To mitigate this issue, AI software compression plays a crucial role, which aims to compress model size while keeping high performance. However, the intrinsic defects in the big model may be inherited by the compressed one. Such defects may be easily leveraged by attackers, since the compressed models are usually deployed in a large number of devices without adequate protection. In this paper, we try to address the safe model compression problem from a safety-performance co-optimization perspective. Specifically, inspired by the test-driven development (TDD) paradigm in software engineering, we propose a test-driven sparse training framework called SafeCompress. By simulating the attack mechanism as the safety test, SafeCompress can automatically compress a big model to a small one following the dynamic sparse training paradigm. Further, considering a representative attack, i.e., membership inference attack (MIA), we develop a concrete safe model compression mechanism, called MIA-SafeCompress. Extensive experiments are conducted to evaluate MIA-SafeCompress on five datasets for both computer vision and natural language processing tasks. The results verify the effectiveness and generalization of our method. We also discuss how to adapt SafeCompress to other attacks besides MIA, demonstrating the flexibility of SafeCompress.

cs.LG↗

arXiv2022

Lorentz-violation-induced arrival time delay of astroparticles in Finsler spacetime

Finsler geometry is a natural and fundamental generalization of Riemann geometry. The Finsler structure depends on both coordinates and velocities. We present the arrival time delay of astroparticles subject to Lorentz violation in the framework of Finsler geometry, and the result corresponds to that derived by Jacob and Piran in the standard model of cosmology.

gr-qc↗

arXiv2022

ET White Paper: To Find the First Earth 2.0

We propose to develop a wide-field and ultra-high-precision photometric survey mission, temporarily named "Earth 2.0 (ET)". This mission is designed to measure, for the first time, the occurrence rate and the orbital distributions of Earth-sized planets. ET consists of seven 30cm telescopes, to be launched to the Earth-Sun's L2 point. Six of these are transit telescopes with a field of view of 500 square degrees. Staring in the direction that encompasses the original Kepler field for four continuous years, this monitoring will return tens of thousands of transiting planets, including the elusive Earth twins orbiting solar-type stars. The seventh telescope is a 30cm microlensing telescope that will monitor an area of 4 square degrees toward the galactic bulge. This, combined with simultaneous ground-based KMTNet observations, will measure masses for hundreds of long-period and free-floating planets. Together, the transit and the microlensing telescopes will revolutionize our understandings of terrestrial planets across a large swath of orbital distances and free space. In addition, the survey data will also facilitate studies in the fields of asteroseismology, Galactic archeology, time-domain sciences, and black holes in binaries.

astro-ph.IM↗

arXiv2022

Causal inference for observational longitudinal studies using deep survival models

Causal inference for observational longitudinal studies often requires the accurate estimation of treatment effects on time-to-event outcomes in the presence of time-dependent patient history and time-dependent covariates. To tackle this longitudinal treatment effect estimation problem, we have developed a time-variant causal survival (TCS) model that uses the potential outcomes framework with an ensemble of recurrent subnetworks to estimate the difference in survival probabilities and its confidence interval over time as a function of time-dependent covariates and treatments. Using simulated survival datasets, the TCS model showed good causal effect estimation performance across scenarios of varying sample dimensions, event rates, confounding and overlapping. However, increasing the sample size was not effective in alleviating the adverse impact of a high level of confounding. In a large clinical cohort study, TCS identified the expected conditional average treatment effect and detected individual treatment effect heterogeneity over time. TCS provides an efficient way to estimate and update individualized treatment effects over time, in order to improve clinical decisions. The use of a propensity score layer and potential outcome subnetworks helps correcting for selection bias. However, the proposed model is limited in its ability to correct the bias from unmeasured confounding, and more extensive testing of TCS under extreme scenarios such as low overlapping and the presence of unmeasured confounders is desired and left for future work.

stat.ML↗

arXiv2022

Attending Category Disentangled Global Context for Image Classification

In this paper, we propose a general framework for image classification using the attention mechanism and global context, which could incorporate with various network architectures to improve their performance. To investigate the capability of the global context, we compare four mathematical models and observe the global context encoded in the category disentangled conditional generative model could give more guidance as "know what is task irrelevant will also know what is relevant". Based on this observation, we define a novel Category Disentangled Global Context (CDGC) and devise a deep network to obtain it. By attending CDGC, the baseline networks could identify the objects of interest more accurately, thus improving the performance. We apply the framework to many different network architectures and compare with the state-of-the-art on four publicly available datasets. Extensive results validate the effectiveness and superiority of our approach. Code will be made public upon paper acceptance.

cs.CV↗

arXiv2022

Psychologically-Inspired Music Recommendation System

In the last few years, automated recommendation systems have been a major focus in the music field, where companies such as Spotify, Amazon, and Apple are competing in the ability to generate the most personalized music suggestions for their users. One of the challenges developers still fail to tackle is taking into account the psychological and emotional aspects of the music. Our goal is to find a way to integrate users' personal traits and their current emotional state into a single music recommendation system with both collaborative and content-based filtering. We seek to relate the personality and the current emotional state of the listener to the audio features in order to build an emotion-aware MRS. We compare the results both quantitatively and qualitatively to the output of the traditional MRS based on the Spotify API data to understand if our advancements make a significant impact on the quality of music recommendations.

cs.IR↗

arXiv2022

Merging Control Strategies of Connected and Autonomous Vehicles at Freeway On-Ramps: A Comprehensive Review

On-ramp merging areas are typical bottlenecks in the freeway network, since merging on-ramp vehicles may cause intensive disturbances on the mainline traffic flow and lead to various negative impacts on traffic efficiency and safety. The connected and autonomous vehicles (CAVs), with their capabilities of real-time communication and precise motion control, hold a great potential to facilitate ramp merging operation through enhanced coordination strategies. This paper presents a comprehensive review of the existing ramp merging strategies leveraging CAVs, focusing on the latest trends and developments in the research field. The review comprehensively covers 44 papers recently published in leading transportation journals. Based on the application context, control strategies are categorized into three categories: merging into sing-lane freeways with total CAVs, merging into sing-lane freeways with mixed traffic flows, and merging into multilane freeways. Relevant literature is reviewed regarding the required technologies, control decision level, applied methods, and impacts on traffic performance. More importantly, we identify the existing research gaps and provide insightful discussions on the potential and promising directions for future research based on the review, which facilitates further advancement in this research topic.

eess.SY↗

arXiv2022

Learning Consistency from High-quality Pseudo-labels for Weakly Supervised Object Localization

Pseudo-supervised learning methods have been shown to be effective for weakly supervised object localization tasks. However, the effectiveness depends on the powerful regularization ability of deep neural networks. Based on the assumption that the localization network should have similar location predictions on different versions of the same image, we propose a two-stage approach to learn more consistent localization. In the first stage, we propose a mask-based pseudo label generator algorithm, and use the pseudo-supervised learning method to initialize an object localization network. In the second stage, we propose a simple and effective method for evaluating the confidence of pseudo-labels based on classification discrimination, and by learning consistency from high-quality pseudo-labels, we further refine the localization network to get better localization performance. Experimental results show that our proposed approach achieves excellent performance in three benchmark datasets including CUB-200-2011, ImageNet-1k and Tiny-ImageNet, which demonstrates its effectiveness.

cs.CV↗

arXiv2022

Sky Computing: Accelerating Geo-distributed Computing in Federated Learning

Federated learning is proposed by Google to safeguard data privacy through training models locally on users' devices. However, with deep learning models growing in size to achieve better results, it becomes increasingly difficult to accommodate the whole model on one single device. Thus, model parallelism is then used to divide the model weights among several devices. With this logic, the approach currently used evenly allocates weights among devices. However, in reality, a computation bottleneck may occur resulting from variant computing power of different users' devices. To address this problem, load balancing is needed to allocate the model weights based on the computational capability of the device. In this paper, we proposed Sky Computing, a load-balanced model parallelism framework to adaptively allocate the weights to devices. Sky Computing outperforms the baseline method by 55% in training time when training 160-layer BERT with 64 nodes. The source code can be found at https://github.com/hpcaitech/SkyComputing.

cs.LG↗

arXiv2022

Flow-level Coordination of Connected and Autonomous Vehicles in Multilane Freeway Ramp Merging Areas

On-ramp merging areas are deemed to be typical bottlenecks for freeway networks due to the intensive disturbances induced by the frequent merging, weaving, and lane-changing behaviors. The Connected and Autonomous Vehicles (CAVs), benefited from their capabilities of real-time communication and precise motion control, hold an opportunity to promote ramp merging operation through enhanced cooperation. The existing CAV cooperation strategies are mainly designed for single-lane freeways, although multilane configurations are more prevailing in the real-world. In this paper, we present a flow-level CAV coordination strategy to facilitate merging operation in multilane freeways. The coordination integrates lane-change rules between mainstream lanes, proactive creation of large merging gaps, and platooning of ramp vehicles for enhanced benefits in traffic flow stability and efficiency. The strategy is formulated under an optimization framework, where the optimal control plan is determined based on real-time traffic conditions. The impacts of tunable model parameters on the produced control plan are discussed in detail. The efficiency of the proposed multilane strategy is demonstrated in a micro-simulation environment. The results show that the coordination can substantially improve the overall ramp merging efficiency and prevent recurrent traffic congestions, especially under high traffic volume conditions.

eess.SY↗

arXiv2022

DeepRelease: Language-agnostic Release Notes Generation from Pull Requests of Open-source Software

The release note is an essential software artifact of open-source software that documents crucial information about changes, such as new features and bug fixes. With the help of release notes, both developers and users could have a general understanding of the latest version without browsing the source code. However, it is a daunting and time-consuming job for developers to produce release notes. Although prior studies have provided some automatic approaches, they generate release notes mainly by extracting information from code changes. This will result in language-specific and not being general enough to be applicable. Therefore, helping developers produce release notes effectively remains an unsolved challenge. To address the problem, we first conduct a manual study on the release notes of 900 GitHub projects, which reveals that more than 54% of projects produce their release notes with pull requests. Based on the empirical finding, we propose a deep learning based approach named DeepRelease (Deep learning based Release notes generator) to generate release notes according to pull requests. The process of release notes generation in DeepRelease includes the change entries generation and the change category (i.e., new features or bug fixes) generation, which are formulated as a text summarization task and a multi-class classification problem, respectively. Since DeepRelease fully employs text information from pull requests to summarize changes and identify the change category, it is language-agnostic and can be used for projects in any language. We build a dataset with over 46K release notes and evaluate DeepRelease on the dataset. The experimental results indicate that DeepRelease outperforms four baselines and can generate release notes similar to those manually written ones in a fraction of the time.

cs.SE↗

arXiv2021

MSP : Refine Boundary Segmentation via Multiscale Superpixel

In this paper, we propose a simple but effective message passing method to improve the boundary quality for the semantic segmentation result. Inspired by the generated sharp edges of superpixel blocks, we employ superpixel to guide the information passing within feature map. Simultaneously, the sharp boundaries of the blocks also restrict the message passing scope. Specifically, we average features that the superpixel block covers within feature map, and add the result back to each feature vector. Further, to obtain sharper edges and farther spatial dependence, we develop a multiscale superpixel module (MSP) by a cascade of different scales superpixel blocks. Our method can be served as a plug-and-play module and easily inserted into any segmentation network without introducing new parameters. Extensive experiments are conducted on three strong baselines, namely PSPNet, DeeplabV3, and DeepLabV3+, and four challenging scene parsing datasets including ADE20K, Cityscapes, PASCAL VOC, and PASCAL Context. The experimental results verify its effectiveness and generalizability.

cs.CV↗