arXiv Science⌕ Search

arXiv subjects

Xiao Lin

Publications and source records attributed to Xiao Lin.

At least 109 records · Page 6Linked to original sources
arXiv2024

SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environments

Semantic reasoning and dynamic planning capabilities are crucial for an autonomous agent to perform complex navigation tasks in unknown environments. It requires a large amount of common-sense knowledge, that humans possess, to succeed in these tasks. We present SayNav, a new approach that leverages human knowledge from Large Language Models (LLMs) for efficient generalization to complex navigation tasks in unknown large-scale environments. SayNav uses a novel grounding mechanism, that incrementally builds a 3D scene graph of the explored environment as inputs to LLMs, for generating feasible and contextually appropriate high-level plans for navigation. The LLM-generated plan is then executed by a pre-trained low-level planner, that treats each planned step as a short-distance point-goal navigation sub-task. SayNav dynamically generates step-by-step instructions during navigation and continuously refines future steps based on newly perceived information. We evaluate SayNav on multi-object navigation (MultiON) task, that requires the agent to utilize a massive amount of human knowledge to efficiently search multiple different objects in an unknown environment. We also introduce a benchmark dataset for MultiON task employing ProcTHOR framework that provides large photo-realistic indoor environments with variety of objects. SayNav achieves state-of-the-art results and even outperforms an oracle based baseline with strong ground-truth assumptions by more than 8% in terms of success rate, highlighting its ability to generate dynamic plans for successfully locating objects in large-scale new environments. The code, benchmark dataset and demonstration videos are accessible at https://www.sri.com/ics/computer-vision/saynav.

cs.RO↗
arXiv2024

Maximal quantum interaction between free electrons and photons

The emerging field of free-electron quantum optics enables electron-photon entanglement and holds the potential for generating nontrivial photon states for quantum information processing. Although recent experimental studies have entered the quantum regime, rapid theoretical developments predict that qualitatively unique phenomena only emerge beyond a certain interaction strength. It is thus pertinent to identify the maximal electron-photon interaction strength and the materials, geometries, and particle energies that enable one to approach it. We derive an upper limit to the quantum vacuum interaction strength between free electrons and single-mode photons, which illuminates the conditions for the strongest interaction. Crucially, we obtain an explicit energy selection recipe for electrons and photons to achieve maximal interaction at arbitrary separations and identify two optimal regimes favoring either fast or slow electrons over those with intermediate velocities. We validate the limit by analytical and numerical calculations on canonical geometries and provide near-optimal designs indicating the feasibility of strong quantum interactions. Our findings offer fundamental intuition for maximizing the quantum interaction between free electrons and photons and provide practical design rules for future experiments on electron-photon and electron-mediated photon-photon entanglement. They should also enable the evaluation of key metrics for applications such as the maximum power of free-electron radiation sources and the maximum acceleration gradient of dielectric laser accelerators.

quant-ph↗
arXiv2024

Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation

Category-level 6D object pose estimation aims to estimate the rotation, translation and size of unseen instances within specific categories. In this area, dense correspondence-based methods have achieved leading performance. However, they do not explicitly consider the local and global geometric information of different instances, resulting in poor generalization ability to unseen instances with significant shape variations. To deal with this problem, we propose a novel Instance-Adaptive and Geometric-Aware Keypoint Learning method for category-level 6D object pose estimation (AG-Pose), which includes two key designs: (1) The first design is an Instance-Adaptive Keypoint Detection module, which can adaptively detect a set of sparse keypoints for various instances to represent their geometric structures. (2) The second design is a Geometric-Aware Feature Aggregation module, which can efficiently integrate the local and global geometric information into keypoint features. These two modules can work together to establish robust keypoint-level correspondences for unseen instances, thus enhancing the generalization ability of the model.Experimental results on CAMERA25 and REAL275 datasets show that the proposed AG-Pose outperforms state-of-the-art methods by a large margin without category-specific shape priors.

cs.CV↗
arXiv2024

Task-Agnostic Detector for Insertion-Based Backdoor Attacks

Textual backdoor attacks pose significant security threats. Current detection approaches, typically relying on intermediate feature representation or reconstructing potential triggers, are task-specific and less effective beyond sentence classification, struggling with tasks like question answering and named entity recognition. We introduce TABDet (Task-Agnostic Backdoor Detector), a pioneering task-agnostic method for backdoor detection. TABDet leverages final layer logits combined with an efficient pooling technique, enabling unified logit representation across three prominent NLP tasks. TABDet can jointly learn from diverse task-specific models, demonstrating superior detection efficacy over traditional task-specific methods.

cs.CL↗
arXiv2024

Observation of Topological Corner State Arrays in Photonic Quasicrystals

Recently, the studies of topological corner states (TCSs) are extended from crystals to quasicrystals, which are referred to as higher-order topological quasicrystalline insulators (HOTQIs). However, the TCSs of complete quasi-periodic structure in photonic systems have yet to be demonstrated. Moreover, there is only one TCS in each corner region in higher-order topological insulators (HOTIs). Increasing the number of TCS is expected to increase application potential of TCSs. In this work, HOTQIs in photonic systems are experimentally observed. It is found that HOTQIs possess TCS arrays, and each TCS array contains several TCSs. Furthermore, the universal theoretical framework of the multimer analysis method is improved, and the difference in the average charge density is proposed as a real-space topological index. These results will open up new ideas for investigating highly integrated, multi-region localized TCSs and are expected to provide new ways to explore topological phenomena and the applications of photonic quasicrystals.

cond-mat.mtrl-sci↗
arXiv2024

BeMap: Balanced Message Passing for Fair Graph Neural Network

Fairness in graph neural networks has been actively studied recently. However, existing works often do not explicitly consider the role of message passing in introducing or amplifying the bias. In this paper, we first investigate the problem of bias amplification in message passing. We empirically and theoretically demonstrate that message passing could amplify the bias when the 1-hop neighbors from different demographic groups are unbalanced. Guided by such analyses, we propose BeMap, a fair message passing method, that leverages a balance-aware sampling strategy to balance the number of the 1-hop neighbors of each node among different demographic groups. Extensive experiments on node classification demonstrate the efficacy of BeMap in mitigating bias while maintaining classification accuracy. The code is available at https://github.com/xiaolin-cs/BeMap.

cs.LG↗
arXiv2024

CLIPose: Category-Level Object Pose Estimation with Pre-trained Vision-Language Knowledge

Most of existing category-level object pose estimation methods devote to learning the object category information from point cloud modality. However, the scale of 3D datasets is limited due to the high cost of 3D data collection and annotation. Consequently, the category features extracted from these limited point cloud samples may not be comprehensive. This motivates us to investigate whether we can draw on knowledge of other modalities to obtain category information. Inspired by this motivation, we propose CLIPose, a novel 6D pose framework that employs the pre-trained vision-language model to develop better learning of object category information, which can fully leverage abundant semantic knowledge in image and text modalities. To make the 3D encoder learn category-specific features more efficiently, we align representations of three modalities in feature space via multi-modal contrastive learning. In addition to exploiting the pre-trained knowledge of the CLIP's model, we also expect it to be more sensitive with pose parameters. Therefore, we introduce a prompt tuning approach to fine-tune image encoder while we incorporate rotations and translations information in the text descriptions. CLIPose achieves state-of-the-art performance on two mainstream benchmark datasets, REAL275 and CAMERA25, and runs in real-time during inference (40FPS).

cs.CV↗
arXiv2024

Hybridnet for depth estimation and semantic segmentation

Semantic segmentation and depth estimation are two important tasks in the area of image processing. Traditionally, these two tasks are addressed in an independent manner. However, for those applications where geometric and semantic information is required, such as robotics or autonomous navigation,depth or semantic segmentation alone are not sufficient. In this paper, depth estimation and semantic segmentation are addressed together from a single input image through a hybrid convolutional network. Different from the state of the art methods where features are extracted by a sole feature extraction network for both tasks, the proposed HybridNet improves the features extraction by separating the relevant features for one task from those which are relevant for both. Experimental results demonstrate that HybridNet results are comparable with the state of the art methods, as well as the single task methods that HybridNet is based on.

cs.CV↗
arXiv2024

Performance Analysis of Integrated Data and Energy Transfer Assisted by Fluid Antenna Systems

Fluid antenna multiple access (FAMA) is capable of exploiting the high spatial diversity of wireless channels to mitigate multi-user interference via flexible port switching, which achieves a better performance than traditional multi-input-multi-output (MIMO) systems. Moreover, integrated data and energy transfer (IDET) is able to provide both the wireless data transfer (WDT) and wireless energy transfer (WET) services towards low-power devices. In this paper, a FAMA assisted IDET system is studied, where $N$ access points (APs) provide dedicated IDET services towards $N$ user equipments (UEs). Each UE is equipped with a single fluid antenna. The performance of WDT and WET , i.e., the WDT outage probability, the WET outage probability, the reliable throughput and the average energy harvesting amount, are analysed theoretically by using time switching (TS) between WDT and WET. Numerical results validate our theoretical analysis, which reveals that the number of UEs and TS ratio should be optimized to achieve a trade-off between the WDT and WET performance. Moreover, FAMA assisted IDET achieves a better performance in terms of both WDT and WET than traditional MIMO with the same antenna size.

cs.IT↗
arXiv2024

RHOBIN Challenge: Reconstruction of Human Object Interaction

Modeling the interaction between humans and objects has been an emerging research direction in recent years. Capturing human-object interaction is however a very challenging task due to heavy occlusion and complex dynamics, which requires understanding not only 3D human pose, and object pose but also the interaction between them. Reconstruction of 3D humans and objects has been two separate research fields in computer vision for a long time. We hence proposed the first RHOBIN challenge: reconstruction of human-object interactions in conjunction with the RHOBIN workshop. It was aimed at bringing the research communities of human and object reconstruction as well as interaction modeling together to discuss techniques and exchange ideas. Our challenge consists of three tracks of 3D reconstruction from monocular RGB images with a focus on dealing with challenging interaction scenarios. Our challenge attracted more than 100 participants with more than 300 submissions, indicating the broad interest in the research communities. This paper describes the settings of our challenge and discusses the winning methods of each track in more detail. We observe that the human reconstruction task is becoming mature even under heavy occlusion settings while object pose estimation and joint reconstruction remain challenging tasks. With the growing interest in interaction modeling, we hope this report can provide useful insights and foster future research in this direction. Our workshop website can be found at \href{https://rhobin-challenge.github.io/}{https://rhobin-challenge.github.io/}.

cs.CV↗
arXiv2023

Human Body Model based ID using Shape and Pose Parameters

We present a Human Body model based IDentification system (HMID) system that is jointly trained for shape, pose and biometric identification. HMID is based on the Human Mesh Recovery (HMR) network and we propose additional losses to improve and stabilize shape estimation and biometric identification while maintaining the pose and shape output. We show that when our HMID network is trained using additional shape and pose losses, it shows a significant improvement in biometric identification performance when compared to an identical model that does not use such losses. The HMID model uses raw images instead of silhouettes and is able to perform robust recognition on images collected at range and altitude as many anthropometric properties are reasonably invariant to clothing, view and range. We show results on the USF dataset as well as the BRIAR dataset which includes probes with both clothing and view changes. Our approach (using body model losses) shows a significant improvement in Rank20 accuracy and True Accuracy Rate on the BRIAR evaluation dataset.

cs.CV↗
arXiv2023

Clues to potential dipolar-Kondo and RKKY interactions in a polar metal

The coexistence of electric dipoles and itinerant electrons in a solid was postulated decades ago, before being experimentally established in several 'polar metals' during the last decade. Here, we report a concentration-driven polar-to-nonpolar phase transition in electron-doped BaTiO_3. Comparing our case with other polar metals, we find a particular threshold concentration (n*) linked to the dipole density (n_d). The universal ratio n_d/n*=8(0.6) suggests a common mechanism across different polar systems, possibly explained by a dipolar Ruderman-Kittel-Kasuya-Yosida theory. Moreover, in BaTiO_3, we observe enhanced thermopower and upturn on resistivity at low temperatures near n*, resembling the Kondo effect. We argue that local electric dipoles act as two-level-systems, whose fluctuations couple with surrounding electron clouds, giving rise to a potential dipolar-counterpart of the Kondo effect. Our findings unveil a mostly uncharted territory for exploring emerging physics associated with electron-dipole correlations, encouraging further theoretical work on dipolar-RKKY and Kondo interactions.

cond-mat.str-el↗
arXiv2023

PyPose v0.6: The Imperative Programming Interface for Robotics

PyPose is an open-source library for robot learning. It combines a learning-based approach with physics-based optimization, which enables seamless end-to-end robot learning. It has been used in many tasks due to its meticulously designed application programming interface (API) and efficient implementation. From its initial launch in early 2022, PyPose has experienced significant enhancements, incorporating a wide variety of new features into its platform. To satisfy the growing demand for understanding and utilizing the library and reduce the learning curve of new users, we present the fundamental design principle of the imperative programming interface, and showcase the flexible usage of diverse functionalities and modules using an extremely simple Dubins car example. We also demonstrate that the PyPose can be easily used to navigate a real quadruped robot with a few lines of code.

cs.RO↗
arXiv2023

Confidence Calibration for Systems with Cascaded Predictive Modules

Existing conformal prediction algorithms estimate prediction intervals at target confidence levels to characterize the performance of a regression model on new test samples. However, considering an autonomous system consisting of multiple modules, prediction intervals constructed for individual modules fall short of accommodating uncertainty propagation over different modules and thus cannot provide reliable predictions on system behavior. We address this limitation and present novel solutions based on conformal prediction to provide prediction intervals calibrated for a predictive system consisting of cascaded modules (e.g., an upstream feature extraction module and a downstream regression module). Our key idea is to leverage module-level validation data to characterize the system-level error distribution without direct access to end-to-end validation data. We provide theoretical justification and empirical experimental results to demonstrate the effectiveness of proposed solutions. In comparison to prediction intervals calibrated for individual modules, our solutions generate improved intervals with more accurate performance guarantees for system predictions, which are demonstrated on both synthetic systems and real-world systems performing overlap prediction for indoor navigation using the Matterport3D dataset.

cs.LG↗
arXiv2023

An electronic band sculpted by oxygen vacancies and indispensable for dilute superconductivity

Dilute superconductivity survives in bulk strontium titanate when the Fermi temperature falls well below the Debye temperature. Here, we show that the onset of the superconducting dome is dopant-dependent. When mobile electrons are introduced by removing oxygen atoms, the superconducting transition survives down to $2 \times 10^{17}$ cm$^{-3}$, but when they are brought by substituting Nb with Ti, the threshold density for superconductivity is an order of magnitude higher. Our study of quantum oscillations reveals a significant difference in the band dispersion between the dilute metals made by these doping routes and our band calculations demonstrate that the rigid band approximation does not hold when mobile electrons are introduced by oxygen vacancies. We identify the band sculpted by theses vacancies as the exclusive locus of superconducting instability in the ultra-dilute limit.

cond-mat.supr-con↗
arXiv2023

Discrete Conditional Diffusion for Reranking in Recommendation

Reranking plays a crucial role in modern multi-stage recommender systems by rearranging the initial ranking list to model interplay between items. Considering the inherent challenges of reranking such as combinatorial searching space, some previous studies have adopted the evaluator-generator paradigm, with a generator producing feasible sequences and a evaluator selecting the best one based on estimated listwise utility. Inspired by the remarkable success of diffusion generative models, this paper explores the potential of diffusion models for generating high-quality sequences in reranking. However, we argue that it is nontrivial to take diffusion models as the generator in the context of recommendation. Firstly, diffusion models primarily operate in continuous data space, differing from the discrete data space of item permutations. Secondly, the recommendation task is different from conventional generation tasks as the purpose of recommender systems is to fulfill user interests. Lastly, real-life recommender systems require efficiency, posing challenges for the inference of diffusion models. To overcome these challenges, we propose a novel Discrete Conditional Diffusion Reranking (DCDR) framework for recommendation. DCDR extends traditional diffusion models by introducing a discrete forward process with tractable posteriors, which adds noise to item sequences through step-wise discrete operations (e.g., swapping). Additionally, DCDR incorporates a conditional reverse process that generates item sequences conditioned on expected user responses. Extensive offline experiments conducted on public datasets demonstrate that DCDR outperforms state-of-the-art reranking methods. Furthermore, DCDR has been deployed in a real-world video app with over 300 million daily active users, significantly enhancing online recommendation quality.

cs.IR↗
arXiv2023

Polar discontinuities and interfacial electronic properties of Bi$_2$O$_2$Se on SrTiO$_3$

The layered oxychalcogenide semiconductor Bi$_2$O$_2$Se (BOS) hosts a multitude of unusual properties including high electron mobility. Owing to similar crystal symmetry and lattice constants, the perovskite oxide SrTiO$_3$ (STO) has been demonstrated to be an excellent substrate for wafer-scale growth of atomically thin BOS films. However, the structural and electronic properties of the BOS/STO interface remain poorly understood. Here, through first-principles study, we reveal that polar discontinuities and interfacial contact configurations have a strong impact on the electronic properties of ideal BOS/STO interfaces. The lowest-energy [Bi-TiO$_2$] contact type, which features the contact between a Bi$_2$O$_2$ layer of BOS with the TiO$_2$-terminated surface of STO, incurs significant interfacial charge transfer from BOS to STO, producing a BOS/STO-mixed, $n$-type metallic state at the interface. By contrast, the [Se-SrO] contact type, which is the most stable contact configuration between BOS and SrO-terminated STO substrate, has a much smaller interfacial charge transfer from STO to BOS and exhibits $p$-type electronic structure with much weaker interfacial hybridization between BOS and STO. These results indicate that BOS grown on TiO$_2$-terminated STO substrates could be a fruitful system for exploring emergent phenomena at the interface between an oxychalcogenide and an oxide, whereas BOS grown on SrO-terminated substrates may be more advantageous for preserving the excellent intrinsic transport properties of BOS.

cond-mat.mtrl-sci↗
arXiv2023

TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models

We present a Multimodal Backdoor Defense technique TIJO (Trigger Inversion using Joint Optimization). Recent work arXiv:2112.07668 has demonstrated successful backdoor attacks on multimodal models for the Visual Question Answering task. Their dual-key backdoor trigger is split across two modalities (image and text), such that the backdoor is activated if and only if the trigger is present in both modalities. We propose TIJO that defends against dual-key attacks through a joint optimization that reverse-engineers the trigger in both the image and text modalities. This joint optimization is challenging in multimodal models due to the disconnected nature of the visual pipeline which consists of an offline feature extractor, whose output is then fused with the text using a fusion module. The key insight enabling the joint optimization in TIJO is that the trigger inversion needs to be carried out in the object detection box feature space as opposed to the pixel space. We demonstrate the effectiveness of our method on the TrojVQA benchmark, where TIJO improves upon the state-of-the-art unimodal methods from an AUC of 0.6 to 0.92 on multimodal dual-key backdoors. Furthermore, our method also improves upon the unimodal baselines on unimodal backdoors. We present ablation studies and qualitative results to provide insights into our algorithm such as the critical importance of overlaying the inverted feature triggers on all visual features during trigger inversion. The prototype implementation of TIJO is available at https://github.com/SRI-CSL/TIJO.

cs.CV↗