arXiv subjects

Jing Xu

Publications and source records attributed to Jing Xu.

At least 199 records · Page 11Linked to original sources

arXiv2023

Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation

In this paper, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of the sensors, we designed a fully physics-grounded simulation pipeline that includes material acquisition, ray-tracing-based infrared (IR) image rendering, IR noise simulation, and depth estimation. The pipeline is able to generate depth maps with material-dependent error patterns similar to a real depth sensor in real time. We conduct real experiments to show that perception algorithms and reinforcement learning policies trained in our simulation platform could transfer well to the real-world test cases without any fine-tuning. Furthermore, due to the high degree of realism of this simulation, our depth sensor simulator can be used as a convenient testbed to evaluate the algorithm performance in the real world, which will largely reduce the human effort in developing robotic algorithms. The entire pipeline has been integrated into the SAPIEN simulator and is open-sourced to promote the research of vision and robotics communities.

cs.RO↗

arXiv2022

Unreliability of two-band model analysis of magnetoresistivities in unveiling temperature-driven Lifshitz transition

Recently, anomalies in the temperature dependences of the carrier density and/or mobility derived from analysis of the magnetoresistivities using the conventional two-band model have been used to unveil intriguing temperature-induced Lifshitz transitions in various materials. For instance, two temperature-driven Lifshitz transitions were inferred to exist in the Dirac nodal-line semimetal ZrSiSe, based on two-band model analysis of the Hall magnetoconductivities where the second band exhibits a change in the carrier type from holes to electrons when the temperature decreases below T = 106 K and a dip is observed in the mobility versus temperature curve at T = 80 K. Here, we revisit the experiments and two-band model analysis on ZrSiSe. We show that the anomalies in the second band may be spurious, because the first band dominates the Hall magnetoconductivities at T > 80 K, making the carrier type and mobility obtained for the second band from the two-band model analysis unreliable. That is, care must be taken in interpreting these anomalies as evidences for temperature-driven Lifshitz transitions. Our skepticism on the existence of such phase transitions in ZrSiSe is further supported by the validation of the Kohler's rule for magnetoresistances at temperatures below 180 K. This work showcases potential issues in interpreting anomalies in the temperature dependence of the carrier density and mobility derived from the analysis of magnetoconductivities or magnetoresistivities using the conventional two-band model.

cond-mat.mtrl-sci↗

arXiv2022

TriPINet: Tripartite Progressive Integration Network for Image Manipulation Localization

Image manipulation localization aims at distinguishing forged regions from the whole test image. Although many outstanding prior arts have been proposed for this task, there are still two issues that need to be further studied: 1) how to fuse diverse types of features with forgery clues; 2) how to progressively integrate multistage features for better localization performance. In this paper, we propose a tripartite progressive integration network (TriPINet) for end-to-end image manipulation localization. First, we extract both visual perception information, e.g., RGB input images, and visual imperceptible features, e.g., frequency and noise traces for forensic feature learning. Second, we develop a guided cross-modality dual-attention (gCMDA) module to fuse different types of forged clues. Third, we design a set of progressive integration squeeze-and-excitation (PI-SE) modules to improve localization performance by appropriately incorporating multiscale features in the decoder. Extensive experiments are conducted to compare our method with state-of-the-art image forensics approaches. The proposed TriPINet obtains competitive results on several benchmark datasets.

cs.CV↗

arXiv2022

The omnidirectional receiver for UWOC systems based on the diffractive deep neural network

The link alignment requirement in underwater wireless optical communication (UWOC) systems is a knotty problem. The diffractive deep neural network (D2NN) has shown great potential in accomplishing tasks all optically these years. In this paper, an omnidirectional receiver based on 7-layer D2NN is first proposed to alleviate the link alignment difficulties in UWOC systems. In addition, the vectorial diffraction theory is introduced into the training of the D2NN to obtain more accurate diffraction calculations compared with the prevalently adopted scalar diffraction theory. Simulation results verify the validity of the vectorial diffraction theory and demonstrate that the presented method can focus incident light waves with tilt angles from 0 to 89 degrees in a 6.25% area of the detection plane with an average focusing efficiency of 90.96%, proving the feasibility of omnidirectional focusing at the receiver end. Extra simulations further reveal that more layers do not lead to a sustained performance improvement but rather reach a bottleneck, and the D2NN can achieve omnidirectional focusing with a certain range of focusing region size. With more effort, the proposed receiver design, which can be highly integrated with detectors, holds promise to realize both omnidirectional and reliable link establishment in UWOC systems in the future.

physics.optics↗

arXiv2022

The Cowen-Douglas Theory for Operator Tuples and Similarity

We are concerned with the similarity problem for Cowen-Douglas operator tuples. The unitary equivalence counterpart was already investigated in the 1970's and geometric concepts including vector bundles and curvature appeared in the description. As the Cowen-Douglas conjecture show, the study of the similarity problem has not been so successful until quite recently. The latest results reveal the close correlation between complex geometry, the corona problem, and the similarity problem for single Cowen-Douglas operators. Without making use of the corona theorems that no longer hold in the multi-variable setting, we prove that the single operator results for similarity remain true for commuting Cowen-Douglas operator tuples as well.

math.FA↗

arXiv2022

Watermarking Graph Neural Networks based on Backdoor Attacks

Graph Neural Networks (GNNs) have achieved promising performance in various real-world applications. Building a powerful GNN model is not a trivial task, as it requires a large amount of training data, powerful computing resources, and human expertise in fine-tuning the model. Moreover, with the development of adversarial attacks, e.g., model stealing attacks, GNNs raise challenges to model authentication. To avoid copyright infringement on GNNs, verifying the ownership of the GNN models is necessary. This paper presents a watermarking framework for GNNs for both graph and node classification tasks. We 1) design two strategies to generate watermarked data for the graph classification task and one for the node classification task, 2) embed the watermark into the host model through training to obtain the watermarked GNN model, and 3) verify the ownership of the suspicious model in a black-box setting. The experiments show that our framework can verify the ownership of GNN models with a very high probability (up to $99\%$) for both tasks. Finally, we experimentally show that our watermarking approach is robust against a state-of-the-art model extraction technique and four state-of-the-art defenses against backdoor attacks.

cs.LG↗

arXiv2022

Multi-domain Cooperative SLAM: The Enabler for Integrated Sensing and Communications

Simultaneous localization and mapping (SLAM) provides user tracking and environmental mapping capabilities, enabling communication systems to gain situational awareness. Advanced communication networks with ultra-wideband, multiple antennas, and a large number of connections present opportunities for deep integration of sensing and communications. First, the development of integrated sensing and communications (ISAC) is reviewed in this study, and the differences between ISAC and traditional communication are revealed. Then, efficient mechanisms for multi-domain collaborative SLAM are presented. In particular, research opportunities and challenges for cross-sensing, cross-user, cross-frequency, and cross-device SLAM mechanisms are proposed. In addition, SLAM-aided communication strategies are explicitly discussed. We prove that the multi-domain cooperative SLAM mechanisms based on hybrid sensing and crowdsourcing can considerably improve the accuracy of localization and mapping in complex multipath propagation environments through numerical analysis. Furthermore, we conduct testbed experiments to show that the proposed SLAM mechanisms can achieve decimeter-level localization and mapping accuracy in practical scenarios, thereby proving the application prospect of multi-domain collaborative SLAM in ISAC.

cs.IT↗

arXiv2022

The CRINGE Loss: Learning what language not to model

Standard language model training employs gold human documents or human-human interaction data, and treats all training data as positive examples. Growing evidence shows that even with very large amounts of positive training data, issues remain that can be alleviated with relatively small amounts of negative data -- examples of what the model should not do. In this work, we propose a novel procedure to train with such data called the CRINGE loss (ContRastive Iterative Negative GEneration). We show the effectiveness of this approach across three different experiments on the tasks of safe generation, contradiction avoidance, and open-domain dialogue. Our models outperform multiple strong baselines and are conceptually simple, easy to train and implement.

cs.CL↗

arXiv2022

Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid

Few-shot learning (FSL) targets at generalization of vision models towards unseen tasks without sufficient annotations. Despite the emergence of a number of few-shot learning methods, the sample selection bias problem, i.e., the sensitivity to the limited amount of support data, has not been well understood. In this paper, we find that this problem usually occurs when the positions of support samples are in the vicinity of task centroid -- the mean of all class centroids in the task. This motivates us to propose an extremely simple feature transformation to alleviate this problem, dubbed Task Centroid Projection Removing (TCPR). TCPR is applied directly to all image features in a given task, aiming at removing the dimension of features along the direction of the task centroid. While the exact task centroid cannot be accurately obtained from limited data, we estimate it using base features that are each similar to one of the support features. Our method effectively prevents features from being too close to the task centroid. Extensive experiments over ten datasets from different domains show that TCPR can reliably improve classification accuracy across various feature extractors, training algorithms and datasets. The code has been made available at https://github.com/KikimorMay/FSL-TCBR.

cs.CV↗

arXiv2022

When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels

Deployed dialogue agents have the potential to integrate human feedback to continuously improve themselves. However, humans may not always provide explicit signals when the chatbot makes mistakes during interactions. In this work, we propose Juicer, a framework to make use of both binary and free-form textual human feedback. It works by: (i) extending sparse binary feedback by training a satisfaction classifier to label the unlabeled data; and (ii) training a reply corrector to map the bad replies to good ones. We find that augmenting training with model-corrected replies improves the final dialogue model, and we can further improve performance by using both positive and negative replies through the recently proposed Director model.

cs.CL↗

arXiv2022

A Multi-turn Machine Reading Comprehension Framework with Rethink Mechanism for Emotion-Cause Pair Extraction

Emotion-cause pair extraction (ECPE) is an emerging task in emotion cause analysis, which extracts potential emotion-cause pairs from an emotional document. Most recent studies use end-to-end methods to tackle the ECPE task. However, these methods either suffer from a label sparsity problem or fail to model complicated relations between emotions and causes. Furthermore, they all do not consider explicit semantic information of clauses. To this end, we transform the ECPE task into a document-level machine reading comprehension (MRC) task and propose a Multi-turn MRC framework with Rethink mechanism (MM-R). Our framework can model complicated relations between emotions and causes while avoiding generating the pairing matrix (the leading cause of the label sparsity problem). Besides, the multi-turn structure can fuse explicit semantic information flow between emotions and causes. Extensive experiments on the benchmark emotion cause corpus demonstrate the effectiveness of our proposed framework, which outperforms existing state-of-the-art methods.

cs.CL↗

arXiv2022

On the $N$-hypercontractions and similarity of multivariable weighted shifts

In \cite{SH}, A. L. Shields proved a well-known theorem for the similarity of unilateral weighted shift operators. By using the generalization of this theorem for multivariable weighted shifts and the curvature of holomorphic bundles, we give a necessary and sufficient condition for the similarity of $m$-tuples in Cowen-Douglas class. We also present a necessary condition for commuting $m$-tuples of backward weighted shift operators to be $n$-hypercontractive in terms of the weight sequences.

math.FA↗

arXiv2022

Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback

Frozen models trained to mimic static datasets can never improve their performance. Models that can employ internet-retrieval for up-to-date information and obtain feedback from humans during deployment provide the promise of both adapting to new information, and improving their performance. In this work we study how to improve internet-driven conversational skills in such a learning framework. We collect deployment data, which we make publicly available, of human interactions, and collect various types of human feedback -- including binary quality measurements, free-form text feedback, and fine-grained reasons for failure. We then study various algorithms for improving from such feedback, including standard supervised learning, rejection sampling, model-guiding and reward-based learning, in order to make recommendations on which type of feedback and algorithms work best. We find the recently introduced Director model (Arora et al., '22) shows significant improvements over other existing approaches.

cs.CL↗

arXiv2022

BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage

We present BlenderBot 3, a 175B parameter dialogue model capable of open-domain conversation with access to the internet and a long-term memory, and having been trained on a large number of user defined tasks. We release both the model weights and code, and have also deployed the model on a public web page to interact with organic users. This technical report describes how the model was built (architecture, model and training scheme), and details of its deployment, including safety mechanisms. Human evaluations show its superiority to existing open-domain dialogue agents, including its predecessors (Roller et al., 2021; Komeili et al., 2022). Finally, we detail our plan for continual learning using the data collected from deployment, which will also be publicly released. The goal of this research program is thus to enable the community to study ever-improving responsible agents that learn through interaction.

cs.CL↗

arXiv2022

KIC 10417986: Spectroscopic confirmation of the nature of the binary system with a δ Scuti component

KIC 10417986 is a short orbital period (0.0737 d) ellipsoidal variable star with a δ Scuti and γ Doradus hybrid pulsations component discovered by Kepler. The ground-based spectroscopic observations were carried out in the winters of 2020 and 2021 to investigate the binary nature of this star. We derive the orbital parameters using the rvfit code with a result of K1 = 29.7 $\pm$ 1.5 km/s, γ = -18.7 $\pm$ 1.7 km/s, and confirm an orbital period of 0.84495 d instead of the result given by Kepler. The atmospheric parameters of the primary are determined by the synthetic spectra fitting technique with the estimated values of Teff = 7411 $\pm$ 187 K, log g = 4.2 $\pm$ 0.3 dex, [M/H] = 0.08 $\pm$ 0.09 dex and vsini = 52 $\pm$ 11 km/s. KIC 10417986 is a circular orbit binary system. From the single-lined nature and mass function of the star, the derived orbital inclination is 26 $\pm$ 6°, and the mass of the secondary is from 0.43 to 0.7 M_sun, which should be a late-K to early-M type star. Fourteen frequencies are extracted from Kepler light curves, of which six independent frequencies in the high-frequency region are identified as the p-mode pulsations of δ Scuti star, and one independent frequency in the low-frequency region (f2 = 1.3033 c/d) is probably the rotational frequency due to the starspots rather than the ellipsoidal effect or g-mode of γ Doradus.

astro-ph.SR↗

arXiv2022

Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trolls

The promise of interaction between intelligent conversational agents and humans is that models can learn from such feedback in order to improve. Unfortunately, such exchanges in the wild will not always involve human utterances that are benign or of high quality, and will include a mixture of engaged (helpers) and unengaged or even malicious users (trolls). In this work we study how to perform robust learning in such an environment. We introduce a benchmark evaluation, SafetyMix, which can evaluate methods that learn safe vs. toxic language in a variety of adversarial settings to test their robustness. We propose and analyze several mitigating learning algorithms that identify trolls either at the example or at the user level. Our main finding is that user-based methods, that take into account that troll users will exhibit adversarial behavior across multiple examples, work best in a variety of settings on our benchmark. We then test these methods in a further real-life setting of conversations collected during deployment, with similar results.

cs.CL↗

arXiv2022

SSformer: A Lightweight Transformer for Semantic Segmentation

It is well believed that Transformer performs better in semantic segmentation compared to convolutional neural networks. Nevertheless, the original Vision Transformer may lack of inductive biases of local neighborhoods and possess a high time complexity. Recently, Swin Transformer sets a new record in various vision tasks by using hierarchical architecture and shifted windows while being more efficient. However, as Swin Transformer is specifically designed for image classification, it may achieve suboptimal performance on dense prediction-based segmentation task. Further, simply combing Swin Transformer with existing methods would lead to the boost of model size and parameters for the final segmentation model. In this paper, we rethink the Swin Transformer for semantic segmentation, and design a lightweight yet effective transformer model, called SSformer. In this model, considering the inherent hierarchical design of Swin Transformer, we propose a decoder to aggregate information from different layers, thus obtaining both local and global attentions. Experimental results show the proposed SSformer yields comparable mIoU performance with state-of-the-art models, while maintaining a smaller model size and lower compute.

cs.CV↗

arXiv2022

On Anytime Learning at Macroscale

In many practical applications of machine learning data arrives sequentially over time in large chunks. Practitioners have then to decide how to allocate their computational budget in order to obtain the best performance at any point in time. Online learning theory for convex optimization suggests that the best strategy is to use data as soon as it arrives. However, this might not be the best strategy when using deep non-linear networks, particularly when these perform multiple passes over each chunk of data rendering the overall distribution non i.i.d.. In this paper, we formalize this learning setting in the simplest scenario in which each data chunk is drawn from the same underlying distribution, and make a first attempt at empirically answering the following questions: How long should the learner wait before training on the newly arrived chunks? What architecture should the learner adopt? Should the learner increase capacity over time as more data is observed? We probe this learning setting using convolutional neural networks trained on classic computer vision benchmarks as well as a large transformer model trained on a large-scale language modeling task. Code is available at \url{www.github.com/facebookresearch/ALMA}.

cs.LG↗