arXiv subjects

Yu Liu

Publications and source records attributed to Yu Liu.

At least 739 records · Page 41Linked to original sources

arXiv2023

Deep Learning for Video-Text Retrieval: a Review

Video-Text Retrieval (VTR) aims to search for the most relevant video related to the semantics in a given sentence, and vice versa. In general, this retrieval task is composed of four successive steps: video and textual feature representation extraction, feature embedding and matching, and objective functions. In the last, a list of samples retrieved from the dataset is ranked based on their matching similarities to the query. In recent years, significant and flourishing progress has been achieved by deep learning techniques, however, VTR is still a challenging task due to the problems like how to learn an efficient spatial-temporal video feature and how to narrow the cross-modal gap. In this survey, we review and summarize over 100 research papers related to VTR, demonstrate state-of-the-art performance on several commonly benchmarked datasets, and discuss potential challenges and directions, with the expectation to provide some insights for researchers in the field of video-text retrieval.

cs.CV↗

arXiv2023

Composer: Creative and Controllable Image Synthesis with Composable Conditions

Recent large-scale generative models learned on big data are capable of synthesizing incredible images yet suffer from limited controllability. This work offers a new generation paradigm that allows flexible control of the output image, such as spatial layout and palette, while maintaining the synthesis quality and model creativity. With compositionality as the core idea, we first decompose an image into representative factors, and then train a diffusion model with all these factors as the conditions to recompose the input. At the inference stage, the rich intermediate representations work as composable elements, leading to a huge design space (i.e., exponentially proportional to the number of decomposed factors) for customizable content creation. It is noteworthy that our approach, which we call Composer, supports various levels of conditions, such as text description as the global information, depth map and sketch as the local guidance, color histogram for low-level details, etc. Besides improving controllability, we confirm that Composer serves as a general framework and facilitates a wide range of classical generative tasks without retraining. Code and models will be made available.

cs.CV↗

arXiv2023

Improving Object-centric Learning with Query Optimization

The ability to decompose complex natural scenes into meaningful object-centric abstractions lies at the core of human perception and reasoning. In the recent culmination of unsupervised object-centric learning, the Slot-Attention module has played an important role with its simple yet effective design and fostered many powerful variants. These methods, however, have been exceedingly difficult to train without supervision and are ambiguous in the notion of object, especially for complex natural scenes. In this paper, we propose to address these issues by investigating the potential of learnable queries as initializations for Slot-Attention learning, uniting it with efforts from existing attempts on improving Slot-Attention learning with bi-level optimization. With simple code adjustments on Slot-Attention, our model, Bi-level Optimized Query Slot Attention, achieves state-of-the-art results on 3 challenging synthetic and 7 complex real-world datasets in unsupervised image segmentation and reconstruction, outperforming previous baselines by a large margin. We provide thorough ablative studies to validate the necessity and effectiveness of our design. Additionally, our model exhibits great potential for concept binding and zero-shot learning. Our work is made publicly available at https://bo-qsa.github.io

cs.CV↗

arXiv2023

Channel Tracking for RIS-aided mmWave Communications Under High Mobility Scenarios

The emerging reconfigurable intelligent surface (RIS) technology is promising for applications in the millimeter wave (mmWave) communication systems to effectively compensate for propagation loss or tackle the blockage issue. Considering the high mobility of users in realistic scenarios, it is essential to adjust the phase shifts in real time to align the beam towards the mobile users, which requires to frequently estimate the channel state information. Hence, it is imperative to design efficient channel tracking schemes to avoid the complex channel estimation procedure. In this paper, we develop a novel channel tracking scheme with two advantages over conventional schemes. First, our tracking scheme is based on the cascaded angles at the RIS instead of the accurate angle values, which is more practical. Second, it can be employed under a more general setting where the noise can be non-Gaussian. Simulation results show the high tracking accuracy of our proposed scheme, and validate the superiority to the existing EKF-based tracking scheme.

eess.SP↗

arXiv2023

Pressure-induced magnetic properties of quasi-2D Cr2Si2Te6 and Mn3Si2Te6

Recently, the pressure has been used as external stimuli to induce structural and magnetic phase transitions in many layered quantum materials whose layers are linked by van der Waals forces. Such materials with weakly held layers allow relatively easy manipulation of the superexchange mechanism and lead to novel magnetic behavior. Using the hydrostatic pressure as a disorderless means to manipulate the interlayer coupling, we applied pressure on two quasi-2D sister compounds, namely, Cr2Si2Te6 (CST) and Mn3Si2Te6 (MST), up to ~1 GPa. Magnetic property measurements with the application of pressure revealed that the ferromagnetic transition temperature decreases in CST while the opposite trend occurs for the ferrimagnet MST. In MST, the magnetization decreases with the increase in the pressure, and such trend is not clearly noticed in CST, within the pressure range studied. Theoretical calculations showed the overall pressure effect on layer separation, bond angle, and exchange coupling, strongly influencing the change in subsequent magnetic characteristics. Exchange coupling in Mn3Si2Te6 is strongly frustrated and the first nearest neighbor interaction is the most dominant of the components with the strongest pressure dependence. Whereas, in Cr2Si2Te6, the exchange coupling parameters exhibit very little dependence on the pressure. This combined experimental and theoretical work has the potential to expand to other relevant quantum materials.

cond-mat.mtrl-sci↗

arXiv2023

Centralized Cooperative Exploration Policy for Continuous Control Tasks

The deep reinforcement learning (DRL) algorithm works brilliantly on solving various complex control tasks. This phenomenal success can be partly attributed to DRL encouraging intelligent agents to sufficiently explore the environment and collect diverse experiences during the agent training process. Therefore, exploration plays a significant role in accessing an optimal policy for DRL. Despite recent works making great progress in continuous control tasks, exploration in these tasks has remained insufficiently investigated. To explicitly encourage exploration in continuous control tasks, we propose CCEP (Centralized Cooperative Exploration Policy), which utilizes underestimation and overestimation of value functions to maintain the capacity of exploration. CCEP first keeps two value functions initialized with different parameters, and generates diverse policies with multiple exploration styles from a pair of value functions. In addition, a centralized policy framework ensures that CCEP achieves message delivery between multiple policies, furthermore contributing to exploring the environment cooperatively. Extensive experimental results demonstrate that CCEP achieves higher exploration capacity. Empirical analysis shows diverse exploration styles in the learned policies by CCEP, reaping benefits in more exploration regions. And this exploration capacity of CCEP ensures it outperforms the current state-of-the-art methods across multiple continuous control tasks shown in experiments.

cs.LG↗

arXiv2023

Semantic Encoder Guided Generative Adversarial Face Ultra-Resolution Network

Face super-resolution is a domain-specific image super-resolution, which aims to generate High-Resolution (HR) face images from their Low-Resolution (LR) counterparts. In this paper, we propose a novel face super-resolution method, namely Semantic Encoder guided Generative Adversarial Face Ultra-Resolution Network (SEGA-FURN) to ultra-resolve an unaligned tiny LR face image to its HR counterpart with multiple ultra-upscaling factors (e.g., 4x and 8x). The proposed network is composed of a novel semantic encoder that has the ability to capture the embedded semantics to guide adversarial learning and a novel generator that uses a hierarchical architecture named Residual in Internal Dense Block (RIDB). Moreover, we propose a joint discriminator which discriminates both image data and embedded semantics. The joint discriminator learns the joint probability distribution of the image space and latent space. We also use a Relativistic average Least Squares loss (RaLS) as the adversarial loss to alleviate the gradient vanishing problem and enhance the stability of the training procedure. Extensive experiments on large face datasets have proved that the proposed method can achieve superior super-resolution results and significantly outperform other state-of-the-art methods in both qualitative and quantitative comparisons.

cs.CV↗

arXiv2023

Real-World Image Super Resolution via Unsupervised Bi-directional Cycle Domain Transfer Learning based Generative Adversarial Network

Deep Convolutional Neural Networks (DCNNs) have exhibited impressive performance on image super-resolution tasks. However, these deep learning-based super-resolution methods perform poorly in real-world super-resolution tasks, where the paired high-resolution and low-resolution images are unavailable and the low-resolution images are degraded by complicated and unknown kernels. To break these limitations, we propose the Unsupervised Bi-directional Cycle Domain Transfer Learning-based Generative Adversarial Network (UBCDTL-GAN), which consists of an Unsupervised Bi-directional Cycle Domain Transfer Network (UBCDTN) and the Semantic Encoder guided Super Resolution Network (SESRN). First, the UBCDTN is able to produce an approximated real-like LR image through transferring the LR image from an artificially degraded domain to the real-world LR image domain. Second, the SESRN has the ability to super-resolve the approximated real-like LR image to a photo-realistic HR image. Extensive experiments on unpaired real-world image benchmark datasets demonstrate that the proposed method achieves superior performance compared to state-of-the-art methods.

cs.CV↗

arXiv2023

Epitaxially Driven Phase Selectivity of Sn in Hybrid Quantum Nanowires

Hybrid semiconductor/superconductor nanowires constitute a pervasive platform for studying gate-tunable superconductivity and the emergence of topological behavior. Their low-dimensionality and crystal structure flexibility facilitate novel heterostructure growth and efficient material optimization; crucial prerequisites for accurately constructing complex multi-component quantum materials. Here, we present an extensive optimization of Sn growth on InSb, InAsSb and InAs nanowires. We demonstrate how the growth conditions and the crystal structure/symmetry of the semiconductor drive the formation of either semi-metallic $\mathrm{α-Sn}$ or superconducting $\mathrm{β-Sn}$. For InAs nanowires, we obtain phase-pure, superconducting $\mathrm{β-Sn}$ shells. However, for InSb and InAsSb nanowires, an initial epitaxial $\mathrm{α-Sn}$ phase evolves into a polycrystalline shell of coexisting $\mathrmα$ and $\mathrmβ$ phases, where the $β/α$ volume ratio increases with Sn shell thickness. Whether these nanowires exhibit superconductivity or not critically relies on the $\mathrm{β-Sn}$ content. Therefore, this work provides key insights into Sn phase control on a variety of semiconductors, with consequences for the yield of superconducting hybrids suitable for generating topological systems.

cond-mat.mtrl-sci↗

arXiv2022

Optimizing Replacement Policies for Content Delivery Network Caching: Beyond Belady to Attain A Seemingly Unattainable Byte Miss Ratio

When facing objects/files of differing sizes in content delivery networks (CDNs) caches, pursuing an optimal object miss ratio (OMR) by approximating Belady no longer ensures an optimal byte miss ratio (BMR), creating confusion about how to achieve a superior BMR in CDNs. To address this issue, we experimentally observe that there exists a time window to delay the eviction of the object with the longest reuse distance to improve BMR without increasing OMR. As a result, we introduce a deep reinforcement learning (RL) model to capture this time window by dynamically monitoring the changes in OMR and BMR, and implementing a BMR-friendly policy in the time window. Based on this policy, we propose a Belady and Size Eviction (LRU-BaSE) algorithm, reducing BMR while maintaining OMR. To make LRU-BaSE efficient and practical, we address the feedback delay problem of RL with a two-pronged approach. On the one hand, our observation of a rear section of the LRU cache queue containing most of the eviction candidates allows LRU-BaSE to shorten the decision region. On the other hand, the request distribution on CDNs makes it feasible to divide the learning region into multiple sub-regions that are each learned with reduced time and increased accuracy. In real CDN systems, compared to LRU, LRU-BaSE can reduce "backing to OS" traffic and access latency by 30.05\% and 17.07\%, respectively, on average. The results on the simulator confirm that LRU-BaSE outperforms the state-of-the-art cache replacement policies, where LRU-BaSE's BMR is 0.63\% and 0.33\% less than that of Belady and Practical Flow-based Offline Optimal (PFOO), respectively, on average. In addition, compared to Learning Relaxed Belady (LRB), LRU-BaSE can yield relatively stable performance when facing workload drift.

cs.NI↗

arXiv2022

Directed search for continuous gravitational waves from the possible kilonova remnant G4.8+6.2

G4.8+6.2 was proposed as a possible kilonova remnant associated with the Korean guest star of AD 1163 in our Milky Way galaxy. Its age is about 860 years according to the historical record. If a neutron star was left in the center of G4.8+6.2, this young neutron star may radiate strong continuous gravitational waves, which could beat the indirect age-based upper limit with current LIGO sensitivity. In this work, we searched such continuous gravitational waves in the frequency band $20-1500 \mathrm{~Hz}$. This search used two days of LIGO O3b data from the Hanford and Livingston detectors. While no signal was found, we placed upper limits on the gravitational wave strain. For comparison, we also showed the latest results of all-sky searches obtained with various search pipelines. With upgrading of the gravitational wave detectors, it will provide the opportunity to see whether a black hole or a neutron star is harbored inside G4.8+6.2.

astro-ph.HE↗

arXiv2022

Hyperbolic Hierarchical Contrastive Hashing

Hierarchical semantic structures, naturally existing in real-world datasets, can assist in capturing the latent distribution of data to learn robust hash codes for retrieval systems. Although hierarchical semantic structures can be simply expressed by integrating semantically relevant data into a high-level taxon with coarser-grained semantics, the construction, embedding, and exploitation of the structures remain tricky for unsupervised hash learning. To tackle these problems, we propose a novel unsupervised hashing method named Hyperbolic Hierarchical Contrastive Hashing (HHCH). We propose to embed continuous hash codes into hyperbolic space for accurate semantic expression since embedding hierarchies in hyperbolic space generates less distortion than in hyper-sphere space and Euclidean space. In addition, we extend the K-Means algorithm to hyperbolic space and perform the proposed hierarchical hyperbolic K-Means algorithm to construct hierarchical semantic structures adaptively. To exploit the hierarchical semantic structures in hyperbolic space, we designed the hierarchical contrastive learning algorithm, including hierarchical instance-wise and hierarchical prototype-wise contrastive learning. Extensive experiments on four benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art unsupervised hashing methods. Codes will be released.

cs.CV↗

arXiv2022

Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer

Large-scale deployment of autonomous vehicles has been continually delayed due to safety concerns. On the one hand, comprehensive scene understanding is indispensable, a lack of which would result in vulnerability to rare but complex traffic situations, such as the sudden emergence of unknown objects. However, reasoning from a global context requires access to sensors of multiple types and adequate fusion of multi-modal sensor signals, which is difficult to achieve. On the other hand, the lack of interpretability in learning models also hampers the safety with unverifiable failure causes. In this paper, we propose a safety-enhanced autonomous driving framework, named Interpretable Sensor Fusion Transformer(InterFuser), to fully process and fuse information from multi-modal multi-view sensors for achieving comprehensive scene understanding and adversarial event detection. Besides, intermediate interpretable features are generated from our framework, which provide more semantics and are exploited to better constrain actions to be within the safe sets. We conducted extensive experiments on CARLA benchmarks, where our model outperforms prior methods, ranking the first on the public CARLA Leaderboard. Our code will be made available at https://github.com/opendilab/InterFuser

cs.CV↗

arXiv2022

Node-line Dirac semimetal manipulated by Kondo mechanism in nonsymmorphic CePt$_2$Si$_2$

Dirac node lines (DNLs) are characterized by Dirac-type linear crossings between valence and conduction bands along one-dimensional node lines in the Brillouin zone (BZ). Spin-orbit coupling (SOC) usually shifts the degeneracy at the crossings thus destroys DNLs, and so far the reported DNLs in a few materials are non-interacting type, making the search for robust interacting DNLs in real materials appealing. Here, via first-principle calculations, we reveal that Kondo interaction together with nonsymmorphic lattice symmetries can drive a robust interacting DNLs in a Kondo semimetal CePt_2Si_2, and the feature of DNLs can be significantly manipulated by Kondo behavior in different temperature regions. Based on the density function theory combining dynamical mean-field theory (DFT+DMFT), we predict a transition to Kondo-coherent state at coherent temperature T_coh= 80 K upon cooling, verified by temperature dependence of Ce-4f self-energy, Kondo resonance peak, magnetic susceptibility and momentum-resolved spectral. Below T_coh, well-resolved narrow heavy-fermion bands emerge near the Fermi level, constructing clearly visualized interacting DNLs locating at the BZ boundary, in which the Dirac fermions have strongly enhanced effective mass and reduced velocity. In contrast, above a crossover temperature T_KS =600 K, the destruction of local Kondo screening drives non-interacting DNLs which are comprised by light conduction electrons at the same location. These DNLs are protected by lattice nonsymmorphic symmetries thus robust under intrinsic strong SOC. Our proposal of DNLs which can be significantly manipulated according to Kondo behavior provides an unique realization of interacting Dirac semimetals in real strongly correlated materials, and serves as a convenient platform to investigate the effect of electronic correlations on topological materials.

cond-mat.str-el↗

arXiv2022

ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency

Multi-agent reinforcement learning (MARL) suffers from the non-stationarity problem, which is the ever-changing targets at every iteration when multiple agents update their policies at the same time. Starting from first principle, in this paper, we manage to solve the non-stationarity problem by proposing bidirectional action-dependent Q-learning (ACE). Central to the development of ACE is the sequential decision-making process wherein only one agent is allowed to take action at one time. Within this process, each agent maximizes its value function given the actions taken by the preceding agents at the inference stage. In the learning phase, each agent minimizes the TD error that is dependent on how the subsequent agents have reacted to their chosen action. Given the design of bidirectional dependency, ACE effectively turns a multiagent MDP into a single-agent MDP. We implement the ACE framework by identifying the proper network representation to formulate the action dependency, so that the sequential decision process is computed implicitly in one forward pass. To validate ACE, we compare it with strong baselines on two MARL benchmarks. Empirical experiments demonstrate that ACE outperforms the state-of-the-art algorithms on Google Research Football and StarCraft Multi-Agent Challenge by a large margin. In particular, on SMAC tasks, ACE achieves 100% success rate on almost all the hard and super-hard maps. We further study extensive research problems regarding ACE, including extension, generalization, and practicability. Code is made available to facilitate further research.

cs.LG↗

arXiv2022

Dimensionality-Varying Diffusion Process

Diffusion models, which learn to reverse a signal destruction process to generate new data, typically require the signal at each step to have the same dimension. We argue that, considering the spatial redundancy in image signals, there is no need to maintain a high dimensionality in the evolution process, especially in the early generation phase. To this end, we make a theoretical generalization of the forward diffusion process via signal decomposition. Concretely, we manage to decompose an image into multiple orthogonal components and control the attenuation of each component when perturbing the image. That way, along with the noise strength increasing, we are able to diminish those inconsequential components and thus use a lower-dimensional signal to represent the source, barely losing information. Such a reformulation allows to vary dimensions in both training and inference of diffusion models. Extensive experiments on a range of datasets suggest that our approach substantially reduces the computational cost and achieves on-par or even better synthesis performance compared to baseline methods. We also show that our strategy facilitates high-resolution image synthesis and improves FID of diffusion model trained on FFHQ at $1024\times1024$ resolution from 52.40 to 10.46. Code and models will be made publicly available.

cs.LG↗

arXiv2022

Reciprocal phase transition-enabled electro-optic modulation

Electro-optic (EO) modulation is a well-known and essential topic in the field of communications and sensing. Its ultrahigh efficiency is unprecedentedly desired in the current green and data era. However, dramatically increasing the modulation efficiency is difficult due to the monotonic mapping relationship between the electrical signal and modulated optical signal. Here, a new mechanism termed phase-transition EO modulation is revealed from the reciprocal transition between two distinct phase planes arising from the bifurcation. Remarkably, a monolithically integrated mode-locked laser (MLL) is implemented as a prototype. A 24.8-GHz radio-frequency signal is generated and modulated, achieving a modulation energy efficiency of 3.06 fJ/bit improved by about four orders of magnitude and a contrast ratio exceeding 50 dB. Thus, MLL-based phase-transition EO modulation is characterised by ultrahigh modulation efficiency and ultrahigh contrast ratio, as experimentally proved in radio-over-fibre and underwater acoustic-sensing systems. This phase-transition EO modulation opens a new avenue for green communication and ubiquitous connections.

physics.optics↗

arXiv2022

Teach-DETR: Better Training DETR with Teachers

In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors. We show that the predicted boxes from teacher detectors are effective medium to transfer knowledge of teacher detectors, which could be either RCNN-based or DETR-based detectors, to train a more accurate and robust DETR model. This new training scheme can easily incorporate the predicted boxes from multiple teacher detectors, each of which provides parallel supervisions to the student DETR. Our strategy introduces no additional parameters and adds negligible computational cost to the original detector during training. During inference, Teach-DETR brings zero additional overhead and maintains the merit of requiring no non-maximum suppression. Extensive experiments show that our method leads to consistent improvement for various DETR-based detectors. Specifically, we improve the state-of-the-art detector DINO with Swin-Large backbone, 4 scales of feature maps and 36-epoch training schedule, from 57.8% to 58.9% in terms of mean average precision on MSCOCO 2017 validation set. Code will be available at https://github.com/LeonHLJ/Teach-DETR.

cs.CV↗