arXiv Science⌕ Search

arXiv subjects

Yue Wang

Publications and source records attributed to Yue Wang.

At least 613 records · Page 34Linked to original sources
arXiv2023

Discounted Thompson Sampling for Non-Stationary Bandit Problems

Non-stationary multi-armed bandit (NS-MAB) problems have recently received significant attention. NS-MAB are typically modelled in two scenarios: abruptly changing, where reward distributions remain constant for a certain period and change at unknown time steps, and smoothly changing, where reward distributions evolve smoothly based on unknown dynamics. In this paper, we propose Discounted Thompson Sampling (DS-TS) with Gaussian priors to address both non-stationary settings. Our algorithm passively adapts to changes by incorporating a discounted factor into Thompson Sampling. DS-TS method has been experimentally validated, but analysis of the regret upper bound is currently lacking. Under mild assumptions, we show that DS-TS with Gaussian priors can achieve nearly optimal regret bound on the order of $\tilde{O}(\sqrt{TB_T})$ for abruptly changing and $\tilde{O}(T^β)$ for smoothly changing, where $T$ is the number of time steps, $B_T$ is the number of breakpoints, $β$ is associated with the smoothly changing environment and $\tilde{O}$ hides the parameters independent of $T$ as well as logarithmic terms. Furthermore, empirical comparisons between DS-TS and other non-stationary bandit algorithms demonstrate its competitive performance. Specifically, when prior knowledge of the maximum expected reward is available, DS-TS has the potential to outperform state-of-the-art algorithms.

cs.LG↗
arXiv2023

PaCaNet: A Study on CycleGAN with Transfer Learning for Diversifying Fused Chinese Painting and Calligraphy

AI-Generated Content (AIGC) has recently gained a surge in popularity, powered by its high efficiency and consistency in production, and its capability of being customized and diversified. The cross-modality nature of the representation learning mechanism in most AIGC technology allows for more freedom and flexibility in exploring new types of art that would be impossible in the past. Inspired by the pictogram subset of Chinese characters, we proposed PaCaNet, a CycleGAN-based pipeline for producing novel artworks that fuse two different art types, traditional Chinese painting and calligraphy. In an effort to produce stable and diversified output, we adopted three main technical innovations: 1. Using one-shot learning to increase the creativity of pre-trained models and diversify the content of the fused images. 2. Controlling the preference over generated Chinese calligraphy by freezing randomly sampled parameters in pre-trained models. 3. Using a regularization method to encourage the models to produce images similar to Chinese paintings. Furthermore, we conducted a systematic study to explore the performance of PaCaNet in diversifying fused Chinese painting and calligraphy, which showed satisfying results. In conclusion, we provide a new direction of creating arts by fusing the visual information in paintings and the stroke features in Chinese calligraphy. Our approach creates a unique aesthetic experience rooted in the origination of Chinese hieroglyph characters. It is also a unique opportunity to delve deeper into traditional artwork and, in doing so, to create a meaningful impact on preserving and revitalizing traditional heritage.

cs.CV↗
arXiv2023

CodeT5+: Open Code Large Language Models for Code Understanding and Generation

Large language models (LLMs) pretrained on vast source code have achieved prominent progress in code intelligence. However, existing code LLMs have two main limitations in terms of architecture and pretraining tasks. First, they often adopt a specific architecture (encoder-only or decoder-only) or rely on a unified encoder-decoder network for different downstream tasks. The former paradigm is limited by inflexibility in applications while in the latter, the model is treated as a single system for all tasks, leading to suboptimal performance on a subset of tasks. Secondly, they often employ a limited set of pretraining objectives which might not be relevant to some downstream tasks and hence result in substantial performance degrade. To address these limitations, we propose ``CodeT5+'', a family of encoder-decoder LLMs for code in which component modules can be flexibly combined to suit a wide range of downstream code tasks. Such flexibility is enabled by our proposed mixture of pretraining objectives to mitigate the pretrain-finetune discrepancy. These objectives cover span denoising, contrastive learning, text-code matching, and causal LM pretraining tasks, on both unimodal and bimodal multilingual code corpora. Furthermore, we propose to initialize CodeT5+ with frozen off-the-shelf LLMs without training from scratch to efficiently scale up our models, and explore instruction-tuning to align with natural language instructions. We extensively evaluate CodeT5+ on over 20 code-related benchmarks in different settings, including zero-shot, finetuning, and instruction-tuning. We observe state-of-the-art (SoTA) model performance on various code-related tasks, such as code generation and completion, math programming, and text-to-code retrieval tasks. Particularly, our instruction-tuned CodeT5+ 16B achieves new SoTA results on HumanEval code generation task against other open code LLMs.

cs.CL↗
arXiv2023

Model-Free Robust Average-Reward Reinforcement Learning

Robust Markov decision processes (MDPs) address the challenge of model uncertainty by optimizing the worst-case performance over an uncertainty set of MDPs. In this paper, we focus on the robust average-reward MDPs under the model-free setting. We first theoretically characterize the structure of solutions to the robust average-reward Bellman equation, which is essential for our later convergence analysis. We then design two model-free algorithms, robust relative value iteration (RVI) TD and robust RVI Q-learning, and theoretically prove their convergence to the optimal solution. We provide several widely used uncertainty sets as examples, including those defined by the contamination model, total variation, Chi-squared divergence, Kullback-Leibler (KL) divergence and Wasserstein distance.

cs.LG↗
arXiv2023

GeoMAE: Masked Geometric Target Prediction for Self-supervised Point Cloud Pre-Training

This paper tries to address a fundamental question in point cloud self-supervised learning: what is a good signal we should leverage to learn features from point clouds without annotations? To answer that, we introduce a point cloud representation learning framework, based on geometric feature reconstruction. In contrast to recent papers that directly adopt masked autoencoder (MAE) and only predict original coordinates or occupancy from masked point clouds, our method revisits differences between images and point clouds and identifies three self-supervised learning objectives peculiar to point clouds, namely centroid prediction, normal estimation, and curvature prediction. Combined with occupancy prediction, these four objectives yield an nontrivial self-supervised learning task and mutually facilitate models to better reason fine-grained geometry of point clouds. Our pipeline is conceptually simple and it consists of two major steps: first, it randomly masks out groups of points, followed by a Transformer-based point cloud encoder; second, a lightweight Transformer decoder predicts centroid, normal, and curvature for points in each voxel. We transfer the pre-trained Transformer encoder to a downstream peception model. On the nuScene Datset, our model achieves 3.38 mAP improvment for object detection, 2.1 mIoU gain for segmentation, and 1.7 AMOTA gain for multi-object tracking. We also conduct experiments on the Waymo Open Dataset and achieve significant performance improvements over baselines as well.

cs.CV↗
arXiv2023

Van der Waals Nanoantennas on Gold as Hosts for Hybrid Mie-Plasmonic Resonances

Dielectric nanoresonators have been shown to circumvent the heavy optical losses associated with plasmonic devices, however they suffer from less confined resonances. By constructing a hybrid system of both dielectric and metallic materials, one can retain the low losses of dielectric resonances, whilst gaining additional control over the tuning of the modes with the metal, and achieving stronger mode confinement. In particular, multi-layered van der Waals materials are emerging as promising candidates for integration with metals owing to their weak attractive forces, which enable deposition onto such substrates without the requirement of lattice matching. Here we use layered, high refractive index WS$_2$ exfoliated on gold, to fabricate and optically characterize a hybrid nanoantenna-on-gold system. We experimentally observe a hybridization of Mie resonances, Fabry-Pérot modes, and surface plasmon-polaritons launched from the nanoantennas into the substrate. We achieve experimental quality factors of Mie-plasmonic modes of up to 20 times that of Mie resonances in nanoantennas on silica, and observe signatures of a supercavity mode with a Q factor of 263 $\pm$ 28, resulting from strong mode coupling between a higher-order anapole and Fabry-Pérot-plasmonic mode. We further simulate WS$_2$ nanoantennas on gold with an hBN spacer, resulting in calculated electric field enhancements exceeding 2600, and a Purcell factor of 713. Our results demonstrate dramatic changes in the optical response of dielectric nanophotonic structures placed on gold, opening new possibilities for nanophotonics and sensing with simple-to-fabricate devices.

cond-mat.mes-hall↗
arXiv2023

An Efficient Multi-solution Solver for the Inverse Kinematics of 3-Section Constant-Curvature Robots

Piecewise constant curvature is a popular kinematics framework for continuum robots. Computing the model parameters from the desired end pose, known as the inverse kinematics problem, is fundamental in manipulation, tracking and planning tasks. In this paper, we propose an efficient multi-solution solver to address the inverse kinematics problem of 3-section constant-curvature robots by bridging both the theoretical reduction and numerical correction. We derive analytical conditions to simplify the original problem into a one-dimensional problem. Further, the equivalence of the two problems is formalised. In addition, we introduce an approximation with bounded error so that the one dimension becomes traversable while the remaining parameters analytically solvable. With the theoretical results, the global search and numerical correction are employed to implement the solver. The experiments validate the better efficiency and higher success rate of our solver than the numerical methods when one solution is required, and demonstrate the ability of obtaining multiple solutions with optimal path planning in a space with obstacles.

cs.RO↗
arXiv2023

Learning adaptive manipulation of objects with revolute joint: A case study on varied cabinet doors opening

This paper introduces a learning-based framework for robot adaptive manipulating the object with a revolute joint in unstructured environments. We concentrate our discussion on various cabinet door opening tasks. To improve the performance of Deep Reinforcement Learning in this scene, we analytically provide an efficient sampling manner utilizing the constraints of the objects. To open various kinds of doors, we add encoded environment parameters that define the various environments to the input of out policy. To transfer the policy into the real world, we train an adaptation module in simulation and fine-tune the adaptation module to cut down the impact of the policy-unaware environment parameters. We design a series of experiments to validate the efficacy of our framework. Additionally, we testify to the model's performance in the real world compared to the traditional door opening method.

cs.RO↗
arXiv2023

Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flow

Acquiring driving policies that can transfer to unseen environments is challenging when driving in dense traffic flows. The design of traffic flow is essential and previous studies are unable to balance interaction and safety-criticism. To tackle this problem, we propose a socially adversarial traffic flow. We propose a Contextual Partially-Observable Stochastic Game to model traffic flow and assign Social Value Orientation (SVO) as context. We then adopt a two-stage framework. In Stage 1, each agent in our socially-aware traffic flow is driven by a hierarchical policy where upper-level policy communicates genuine SVOs of all agents, which the lower-level policy takes as input. In Stage 2, each agent in the socially adversarial traffic flow is driven by the hierarchical policy where upper-level communicates mistaken SVOs, taken by the lower-level policy trained in Stage 1. Driving policy is adversarially trained through a zero-sum game formulation with upper-level policies, resulting in a policy with enhanced zero-shot transfer capability to unseen traffic flows. Comprehensive experiments on cross-validation verify the superior zero-shot transfer performance of our method.

cs.RO↗
arXiv2023

Gate-tunable ferromagnetism in a van der Waals magnetic semimetal

Magnetic semimetals form an attractive class of materials because of the non-trivial contributions of itinerant electrons to magnetism. Due to their relatively low-carrier-density nature, a doping level of those materials could be largely tuned by a gating technique. Here we demonstrate gate-tunable ferromagnetism in an emergent van der Waals magnetic semimetal Cr3Te4 based on an ion-gating technique. Upon doping electrons into the system, the Curie temperature (TC) sharply increases, approaching near to room temperature, then decreases to some extent. Interestingly, this non-monotonous variation of TC accompanies the switching of the magnetic anisotropy. Furthermore, such evolutions of TC and anisotropy occur synchronously with the sigh changes of the ordinary and anomalous Hall effects. Those results clearly elucidate that the magnetism in Cr3Te4 should be governed by its semimetallic band nature, where the band crossing points play a crucial role both for the magneto-transport properties and magnetism itself.

cond-mat.mtrl-sci↗
arXiv2023

How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

Deep reinforcement learning (DRL) for fluidic pinball, three individually rotating cylinders in the uniform flow arranged in an equilaterally triangular configuration, can learn the efficient flow control strategies due to the validity of self-learning and data-driven state estimation for complex fluid dynamic problems. In this work, we present a DRL-based real-time feedback strategy to control the hydrodynamic force on fluidic pinball, i.e., force extremum and tracking, from cylinders' rotation. By adequately designing reward functions and encoding historical observations, and after automatic learning of thousands of iterations, the DRL-based control was shown to make reasonable and valid control decisions in nonparametric control parameter space, which is comparable to and even better than the optimal policy found through lengthy brute-force searching. Subsequently, one of these results was analyzed by a machine learning model that enabled us to shed light on the basis of decision-making and physical mechanisms of the force tracking process. The finding from this work can control hydrodynamic force on the operation of fluidic pinball system and potentially pave the way for exploring efficient active flow control strategies in other complex fluid dynamic problems.

eess.SY↗
arXiv2023

Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward

The remarkable success of reinforcement learning (RL) heavily relies on observing the reward of every visited state-action pair. In many real world applications, however, an agent can observe only a score that represents the quality of the whole trajectory, which is referred to as the {\em trajectory-wise reward}. In such a situation, it is difficult for standard RL methods to well utilize trajectory-wise reward, and large bias and variance errors can be incurred in policy evaluation. In this work, we propose a novel offline RL algorithm, called Pessimistic vAlue iteRaTion with rEward Decomposition (PARTED), which decomposes the trajectory return into per-step proxy rewards via least-squares-based reward redistribution, and then performs pessimistic value iteration based on the learned proxy reward. To ensure the value functions constructed by PARTED are always pessimistic with respect to the optimal ones, we design a new penalty term to offset the uncertainty of the proxy reward. For general episodic MDPs with large state space, we show that PARTED with overparameterized neural network function approximation achieves an $\tilde{\mathcal{O}}(D_{\text{eff}}H^2/\sqrt{N})$ suboptimality, where $H$ is the length of episode, $N$ is the total number of samples, and $D_{\text{eff}}$ is the effective dimension of the neural tangent kernel matrix. To further illustrate the result, we show that PARTED achieves an $\tilde{\mathcal{O}}(dH^3/\sqrt{N})$ suboptimality with linear MDPs, where $d$ is the feature dimension, which matches with that with neural network function approximation, when $D_{\text{eff}}=dH$. To the best of our knowledge, PARTED is the first offline RL algorithm that is provably efficient in general MDP with trajectory-wise reward.

cs.LG↗
arXiv2023

A Hyper-network Based End-to-end Visual Servoing with Arbitrary Desired Poses

Recently, several works achieve end-to-end visual servoing (VS) for robotic manipulation by replacing traditional controller with differentiable neural networks, but lose the ability to servo arbitrary desired poses. This letter proposes a differentiable architecture for arbitrary pose servoing: a hyper-network based neural controller (HPN-NC). To achieve this, HPN-NC consists of a hyper net and a low-level controller, where the hyper net learns to generate the parameters of the low-level controller and the controller uses the 2D keypoints error for control like traditional image-based visual servoing (IBVS). HPN-NC can complete 6 degree of freedom visual servoing with large initial offset. Taking advantage of the fully differentiable nature of HPN-NC, we provide a three-stage training procedure to servo real world objects. With self-supervised end-to-end training, the performance of the integrated model can be further improved in unseen scenes and the amount of manual annotations can be significantly reduced.

cs.RO↗
arXiv2023

Graph tilings in incompatibility systems

An \emph{incompatibility system} $(G,\mathcal{F})$ consists of a graph $G$ and a family $\mathcal{F}=\{F_v\}_{v\in V(G)}$ over $G$ with $F_v\subseteq \{\{e,e'\}\in {E(G)\choose 2}: e\cap e'=\{v\}\}$. We say that two edges $e,e'\in E(G)$ are \emph{incompatible} if $\{e,e'\}\in F_v$ for some $v\in V(G)$, and otherwise \emph{compatible}. A subgraph $H$ of $G$ is \emph{compatible} if every pair of edges in $H$ are compatible. An incompatibility system $(G,\mathcal{F})$ is \emph{$Δ$-bounded} if for any vertex $v$ and any edge $e$ incident with $v$, there are at most $Δ$ members of $F_v$ containing $e$. This notion was partly motivated by a concept of transition system introduced by Kotzig in 1968, and first formulated by Krivelevich, Lee and Sudakov to study the robustness of Hamiltonicity of Dirac graphs. We prove that for any $α>0$ and any graph $H$ with $h$ vertices, there exists a constant $μ>0$ such that for any sufficiently large $n$ with $n\in h\mathbb{N}$, if $G$ is an $n$-vertex graph with $δ(G)\ge(1-\frac{1}{χ^*(H)}+α)n$ and $(G,\mathcal{F})$ is a $μn$-bounded incompatibility system, then there exists a compatible $H$-factor in $G$, where the value $χ^*(H)$ is either the chromatic number $χ(H)$ or the critical chromatic number $χ_{cr}(H)$ and we provide a dichotomy as in the Kühn--Osthus result. Moreover, we give examples $H$ for which there exists an $μn$-bounded incompatibility system $(G, \mathcal{F})$ with $n\in h\mathbb{N}$ and $δ(G)\ge(1-\frac{1}{χ^*(H)}+\fracμ{2})n$ such that $G$ contains no compatible $H$-factor. Unlike in the previous work of Kühn and Osthus on embedding $H$-factors, our proof uses the lattice-based absorption method.

math.CO↗
arXiv2023

FUTR3D: A Unified Sensor Fusion Framework for 3D Detection

Sensor fusion is an essential topic in many perception systems, such as autonomous driving and robotics. Existing multi-modal 3D detection models usually involve customized designs depending on the sensor combinations or setups. In this work, we propose the first unified end-to-end sensor fusion framework for 3D detection, named FUTR3D, which can be used in (almost) any sensor configuration. FUTR3D employs a query-based Modality-Agnostic Feature Sampler (MAFS), together with a transformer decoder with a set-to-set loss for 3D detection, thus avoiding using late fusion heuristics and post-processing tricks. We validate the effectiveness of our framework on various combinations of cameras, low-resolution LiDARs, high-resolution LiDARs, and Radars. On NuScenes dataset, FUTR3D achieves better performance over specifically designed methods across different sensor combinations. Moreover, FUTR3D achieves great flexibility with different sensor configurations and enables low-cost autonomous driving. For example, only using a 4-beam LiDAR with cameras, FUTR3D (58.0 mAP) achieves on par performance with state-of-the-art 3D detection model CenterPoint (56.6 mAP) using a 32-beam LiDAR.

cs.CV↗
arXiv2023

Compressive Spectrum Sensing Using Sampling-Controlled Block Orthogonal Matching Pursuit

This paper proposes two novel schemes of wideband compressive spectrum sensing (CSS) via block orthogonal matching pursuit (BOMP) algorithm, for achieving high sensing accuracy in real time. These schemes aim to reliably recover the spectrum by adaptively adjusting the number of required measurements without inducing unnecessary sampling redundancy. To this end, the minimum number of required measurements for successful recovery is first derived in terms of its probabilistic lower bound. Then, a CSS scheme is proposed by tightening the derived lower bound, where the key is the design of a nonlinear exponential indicator through a general-purpose sampling-controlled algorithm (SCA). In particular, a sampling-controlled BOMP (SC-BOMP) is developed through a holistic integration of the existing BOMP and the proposed SCA. For fast implementation, a modified version of SC-BOMP is further developed by exploring the block orthogonality in the form of sub-coherence of measurement matrices, which allows more compressive sampling in terms of smaller lower bound of the number of measurements. Such a fast SC-BOMP scheme achieves a desired tradeoff between the complexity and the performance. Simulations demonstrate that the two SC-BOMP schemes outperform the other benchmark algorithms.

eess.SP↗
arXiv2023

How Does Imperfect Automatic Indexing Affect Semantic Search Performance?

Documents in the health domain are often annotated with semantic concepts (i.e., terms) from controlled vocabularies. As the volume of these documents gets large, the annotation work is increasingly done by algorithms. Compared to humans, automatic indexing algorithms are imperfect and may assign wrong terms to documents, which affect subsequent search tasks where queries contain these terms. In this work, we aim to understand the performance impact of using imperfectly assigned terms in Boolean semantic searches. We used MeSH terms and biomedical literature search as a case study. We implemented multiple automatic indexing algorithms on real-world Boolean queries that consist of MeSH terms, and found that (1) probabilistic logic can handle inaccurately assigned terms better than traditional Boolean logic, (2) query-level performance is mostly limited by lowest-performing terms in a query, and (3) mixing a small amount of human indexing with automatic indexing can regain excellent query-level performance. These findings provide important implications for future work on automatic indexing.

cs.IR↗
arXiv2023

Object-centric Inference for Language Conditioned Placement: A Foundation Model based Approach

We focus on the task of language-conditioned object placement, in which a robot should generate placements that satisfy all the spatial relational constraints in language instructions. Previous works based on rule-based language parsing or scene-centric visual representation have restrictions on the form of instructions and reference objects or require large amounts of training data. We propose an object-centric framework that leverages foundation models to ground the reference objects and spatial relations for placement, which is more sample efficient and generalizable. Experiments indicate that our model can achieve a 97.75% success rate of placement with only ~0.26M trainable parameters. Besides, our method generalizes better to both unseen objects and instructions. Moreover, with only 25% training data, we still outperform the top competing approach.

cs.RO↗