arXiv subjects

Yan Dai

Publications and source records attributed to Yan Dai.

34 records · Page 2Linked to original sources

arXiv2023

Refined Regret for Adversarial MDPs with Linear Function Approximation

We consider learning in an adversarial Markov Decision Process (MDP) where the loss functions can change arbitrarily over $K$ episodes and the state space can be arbitrarily large. We assume that the Q-function of any policy is linear in some known features, that is, a linear function approximation exists. The best existing regret upper bound for this setting (Luo et al., 2021) is of order $\tilde{\mathcal O}(K^{2/3})$ (omitting all other dependencies), given access to a simulator. This paper provides two algorithms that improve the regret to $\tilde{\mathcal O}(\sqrt K)$ in the same setting. Our first algorithm makes use of a refined analysis of the Follow-the-Regularized-Leader (FTRL) algorithm with the log-barrier regularizer. This analysis allows the loss estimators to be arbitrarily negative and might be of independent interest. Our second algorithm develops a magnitude-reduced loss estimator, further removing the polynomial dependency on the number of actions in the first algorithm and leading to the optimal regret bound (up to logarithmic terms and dependency on the horizon). Moreover, we also extend the first algorithm to simulator-free linear MDPs, which achieves $\tilde{\mathcal O}(K^{8/9})$ regret and greatly improves over the best existing bound $\tilde{\mathcal O}(K^{14/15})$. This algorithm relies on a better alternative to the Matrix Geometric Resampling procedure by Neu & Olkhovskaya (2020), which could again be of independent interest.

cs.LG↗

arXiv2023

Banker Online Mirror Descent: A Universal Approach for Delayed Online Bandit Learning

We propose Banker Online Mirror Descent (Banker-OMD), a novel framework generalizing the classical Online Mirror Descent (OMD) technique in the online learning literature. The Banker-OMD framework almost completely decouples feedback delay handling and the task-specific OMD algorithm design, thus facilitating the design of new algorithms capable of efficiently and robustly handling feedback delays. Specifically, it offers a general methodology for achieving $\widetilde{\mathcal O}(\sqrt{T} + \sqrt{D})$-style regret bounds in online bandit learning tasks with delayed feedback, where $T$ is the number of rounds and $D$ is the total feedback delay. We demonstrate the power of \texttt{Banker-OMD} by applications to two important bandit learning scenarios with delayed feedback, including delayed scale-free adversarial Multi-Armed Bandits (MAB) and delayed adversarial linear bandits. \texttt{Banker-OMD} leads to the first delayed scale-free adversarial MAB algorithm achieving $\widetilde{\mathcal O}(\sqrt{K}L(\sqrt T+\sqrt D))$ regret and the first delayed adversarial linear bandit algorithm achieving $\widetilde{\mathcal O}(\text{poly}(n)(\sqrt{T} + \sqrt{D}))$ regret. As a corollary, the first application also implies $\widetilde{\mathcal O}(\sqrt{KT}L)$ regret for non-delayed scale-free adversarial MABs, which is the first to match the $Ω(\sqrt{KT}L)$ lower bound up to logarithmic factors and can be of independent interest.

cs.LG↗

arXiv2023

Variance-Aware Sparse Linear Bandits

It is well-known that for sparse linear bandits, when ignoring the dependency on sparsity which is much smaller than the ambient dimension, the worst-case minimax regret is $\widetildeΘ\left(\sqrt{dT}\right)$ where $d$ is the ambient dimension and $T$ is the number of rounds. On the other hand, in the benign setting where there is no noise and the action set is the unit sphere, one can use divide-and-conquer to achieve $\widetilde{\mathcal O}(1)$ regret, which is (nearly) independent of $d$ and $T$. In this paper, we present the first variance-aware regret guarantee for sparse linear bandits: $\widetilde{\mathcal O}\left(\sqrt{d\sum_{t=1}^T σ_t^2} + 1\right)$, where $σ_t^2$ is the variance of the noise at the $t$-th round. This bound naturally interpolates the regret bounds for the worst-case constant-variance regime (i.e., $σ_t \equiv Ω(1)$) and the benign deterministic regimes (i.e., $σ_t \equiv 0$). To achieve this variance-aware regret guarantee, we develop a general framework that converts any variance-aware linear bandit algorithm to a variance-aware algorithm for sparse linear bandits in a "black-box" manner. Specifically, we take two recent algorithms as black boxes to illustrate that the claimed bounds indeed hold, where the first algorithm can handle unknown-variance cases and the second one is more efficient.

cs.LG↗

arXiv2023

Scale-Free Adversarial Multi-Armed Bandit with Arbitrary Feedback Delays

We consider the Scale-Free Adversarial Multi-Armed Bandit (MAB) problem with unrestricted feedback delays. In contrast to the standard assumption that all losses are $[0,1]$-bounded, in our setting, losses can fall in a general bounded interval $[-L, L]$, unknown to the agent beforehand. Furthermore, the feedback of each arm pull can experience arbitrary delays. We propose a novel approach named Scale-Free Delayed INF (SFD-INF) for this novel setting, which combines a recent "convex combination trick" together with a novel doubling and skipping technique. We then present two instances of SFD-INF, each with carefully designed delay-adapted learning scales. The first one SFD-TINF uses $\frac 12$-Tsallis entropy regularizer and can achieve $\widetilde{\mathcal O}(\sqrt{K(D+T)}L)$ regret when the losses are non-negative, where $K$ is the number of actions, $T$ is the number of steps, and $D$ is the total feedback delay. This bound nearly matches the $Ω((\sqrt{KT}+\sqrt{D\log K})L)$ lower-bound when regarding $K$ as a constant independent of $T$. The second one, SFD-LBINF, works for general scale-free losses and achieves a small-loss style adaptive regret bound $\widetilde{\mathcal O}(\sqrt{K\mathbb{E}[\tilde{\mathfrak L}_T^2]}+\sqrt{KDL})$, which falls to the $\widetilde{\mathcal O}(\sqrt{K(D+T)}L)$ regret in the worst case and is thus more general than SFD-TINF despite a more complicated analysis and several extra logarithmic dependencies. Moreover, both instances also outperform the existing algorithms for non-delayed (i.e., $D=0$) scale-free adversarial MAB problems, which can be of independent interest.

cs.LG↗

arXiv2023

Joint k-TE Space Image Reconstruction and Data Fitting for T2 Mapping

Objectives: To develop a joint k-TE reconstruction algorithm to reconstruct the T2-weighted (T2W) images and T2 map simultaneously. Materials and Methods: The joint k-TE reconstruction model was formulated as an optimization problem subject to a self-consistency condition of the exponential decay relationship between the T2W images and T2 map. The objective function included a data fidelity term enforcing the agreement between the solution and the measured k-space data, together with a spatial regularization term on image properties of the T2W images. The optimization problem was solved using Alternating-Direction Method of Multipliers (ADMM). We tested the joint k-TE method in phantom data and healthy volunteer scans with fully-sampled and under-sampled k-space lines. Image quality of the reconstructed T2W images and T2 map, and the accuracy of T2 measurements derived by the joint k- TE and the conventional signal fitting method were compared. Results: The proposed method improved image quality with reduced noise and less artifacts on both T2W images and T2 map, and increased measurement consistency in T2 relaxation time measurements compared with the conventional method in all data sets. Conclusions: The proposed reconstruction method outperformed the conventional magnitude image-based signal fitting method in image quality and stability of quantitative T2 measurements

physics.med-ph↗

arXiv2022

Follow-the-Perturbed-Leader for Adversarial Markov Decision Processes with Bandit Feedback

We consider regret minimization for Adversarial Markov Decision Processes (AMDPs), where the loss functions are changing over time and adversarially chosen, and the learner only observes the losses for the visited state-action pairs (i.e., bandit feedback). While there has been a surge of studies on this problem using Online-Mirror-Descent (OMD) methods, very little is known about the Follow-the-Perturbed-Leader (FTPL) methods, which are usually computationally more efficient and also easier to implement since it only requires solving an offline planning problem. Motivated by this, we take a closer look at FTPL for learning AMDPs, starting from the standard episodic finite-horizon setting. We find some unique and intriguing difficulties in the analysis and propose a workaround to eventually show that FTPL is also able to achieve near-optimal regret bounds in this case. More importantly, we then find two significant applications: First, the analysis of FTPL turns out to be readily generalizable to delayed bandit feedback with order-optimal regret, while OMD methods exhibit extra difficulties (Jin et al., 2022). Second, using FTPL, we also develop the first no-regret algorithm for learning communicating AMDPs in the infinite-horizon setting with bandit feedback and stochastic transitions. Our algorithm is efficient assuming access to an offline planning oracle, while even for the easier full-information setting, the only existing algorithm (Chandrasekaran and Tewari, 2021) is computationally inefficient.

cs.LG↗

arXiv2022

Skeleton-based Action Recognition via Adaptive Cross-Form Learning

Skeleton-based action recognition aims to project skeleton sequences to action categories, where skeleton sequences are derived from multiple forms of pre-detected points. Compared with earlier methods that focus on exploring single-form skeletons via Graph Convolutional Networks (GCNs), existing methods tend to improve GCNs by leveraging multi-form skeletons due to their complementary cues. However, these methods (either adapting structure of GCNs or model ensemble) require the co-existence of all forms of skeletons during both training and inference stages, while a typical situation in real life is the existence of only partial forms for inference. To tackle this issue, we present Adaptive Cross-Form Learning (ACFL), which empowers well-designed GCNs to generate complementary representation from single-form skeletons without changing model capacity. Specifically, each GCN model in ACFL not only learns action representation from the single-form skeletons, but also adaptively mimics useful representations derived from other forms of skeletons. In this way, each GCN can learn how to strengthen what has been learned, thus exploiting model potential and facilitating action recognition as well. Extensive experiments conducted on three challenging benchmarks, i.e., NTU-RGB+D 120, NTU-RGB+D 60 and UAV-Human, demonstrate the effectiveness and generalizability of the proposed method. Specifically, the ACFL significantly improves various GCN models (i.e., CTR-GCN, MS-G3D, and Shift-GCN), achieving a new record for skeleton-based action recognition.

cs.CV↗

arXiv2022

Adaptive Best-of-Both-Worlds Algorithm for Heavy-Tailed Multi-Armed Bandits

In this paper, we generalize the concept of heavy-tailed multi-armed bandits to adversarial environments, and develop robust best-of-both-worlds algorithms for heavy-tailed multi-armed bandits (MAB), where losses have $α$-th ($1<α\le 2$) moments bounded by $σ^α$, while the variances may not exist. Specifically, we design an algorithm \texttt{HTINF}, when the heavy-tail parameters $α$ and $σ$ are known to the agent, \texttt{HTINF} simultaneously achieves the optimal regret for both stochastic and adversarial environments, without knowing the actual environment type a-priori. When $α,σ$ are unknown, \texttt{HTINF} achieves a $\log T$-style instance-dependent regret in stochastic cases and $o(T)$ no-regret guarantee in adversarial cases. We further develop an algorithm \texttt{AdaTINF}, achieving $\mathcal O(σK^{1-\nicefrac 1α}T^{\nicefrac{1}α})$ minimax optimal regret even in adversarial settings, without prior knowledge on $α$ and $σ$. This result matches the known regret lower-bound (Bubeck et al., 2013), which assumed a stochastic environment and $α$ and $σ$ are both known. To our knowledge, the proposed \texttt{HTINF} algorithm is the first to enjoy a best-of-both-worlds regret guarantee, and \texttt{AdaTINF} is the first algorithm that can adapt to both $α$ and $σ$ to achieve optimal gap-indepedent regret bound in classical heavy-tailed stochastic MAB setting and our novel adversarial formulation.

cs.LG↗

arXiv2021

Redshift evolution of the Amati relation: calibrated results from the Hubble diagram of quasars at high redshifts

Gamma-ray bursts (GRBs) have long been proposed as a complementary probe to type Ia supernovae (SNe Ia) and cosmic microwave background to explore the expansion history of the high-redshift universe, mainly because they are bright enough to be detected at greater distances. Although they lack definite physical explanations, many empirical correlations between GRB isotropic energy/luminosity and some directly detectable spectral/temporal properties have been proposed to make GRBs standard candles. Since the observed GRB rate falls off rapidly at low redshifts, thus preventing a cosmology independent calibration of these correlations. In order to avoid the circularity problem, SN Ia data are usually used to calibrate the luminosity relations of GRBs in the low redshift region (limited by the redshift range for SN Ia sample), and then extrapolate it to the high redshift region. This approach is based on the assumption of no redshift evolution for GRB luminosity relations. In this work, we suggest the use a complete quasar sample in the redshift range of $0.5<z<5.5$ to test such an assumption. We divide the quasar sample into several sub-samples with different redshift bins, and use each sub-sample to calibrate the isotropic $γ$-ray equivalent energy of GRBs in relevant redshift bins. By fitting the newly calibrated data, we find strong evidence that the most commonly used Amati relation between spectral peak energy and isotropic-equivalent radiated energy shows no, or marginal, evolution with redshift. Indeed, at different redshifts, the coefficients in the Amati relation could have a maximum variation of 0.93\% at different redshifts, and there could be no coincidence in the range of 1$σ$.

astro-ph.HE↗

arXiv2021

Intra-day variability of BL Lacertae from 2016 to 2018

We monitored BL Lacertae in the B, V, R and I bands for 14 nights during the period of 2016-2018. The source showed significant intraday variability on 12 nights. We performed colour-magnitude analysis and found that the source exhibited bluer-when-brighter chromatism. This bluer-when-brighter behavior is at least partly caused by the larger variation amplitude at shorter wavelength. The variations at different wavelengths are well correlated and show no inter-band time lag.

astro-ph.GA↗

arXiv2019

From Knowledge Map to Mind Map: Artificial Imagination

Imagination is one of the most important factors which makes an artistic painting unique and impressive. With the rapid development of Artificial Intelligence, more and more researchers try to create painting with AI technology automatically. However, lacking of imagination is still a main problem for AI painting. In this paper, we propose a novel approach to inject rich imagination into a special painting art Mind Map creation. We firstly consider lexical and phonological similarities of seed word, then learn and inherit original painting style of the author, and finally apply Dadaism and impossibility of improvisation principles into painting process. We also design several metrics for imagination evaluation. Experimental results show that our proposed method can increase imagination of painting and also improve its overall quality.

cs.CL↗

arXiv2018

Deploy Large-Scale Deep Neural Networks in Resource Constrained IoT Devices with Local Quantization Region

Implementing large-scale deep neural networks with high computational complexity on low-cost IoT devices may inevitably be constrained by limited computation resource, making the devices hard to respond in real-time. This disjunction makes the state-of-art deep learning algorithms, i.e. CNN (Convolutional Neural Networks), incompatible with IoT world. We present a low-bit (range from 8-bit to 1-bit) scheme with our local quantization region algorithm. We use models in Caffe model zoo as our example tasks to evaluate the effect of our low precision data representation scheme. With the available of local quantization region, we find implementations on top of those schemes could greatly retain the model accuracy, besides the reduction of computational complexity. For example, our 8-bit scheme has no drops on top-1 and top-5 accuracy with 2x speedup on Intel Edison IoT platform. Implementations based on our 4-bit, 2-bit or 1-bit scheme are also applicable to IoT devices with advances of low computational complexity. For example, the drop on our task is only 0.7% when using 2-bit scheme, a scheme which could largely save transistors. Making low-bit scheme usable here opens a new door for further optimization on commodity IoT controller, i.e. extra speed-up could be achieved by replacing multiply-accumulate operations with the proposed table look-up operations. The whole study offers a new approach to relief the challenge of bring advanced deep learning algorithm to resource constrained low-cost IoT device.

cs.LG↗

arXiv2017

Intraday Optical Variability of BL Lacertae

We monitored BL Lacertae simultaneously in the optical B, V, R and I bands for 13 nights during the period 2012-2016. The variations were well correlated in all bands and the source showed significant intraday variability (IDV). We also studied its optical flux and colour behaviour, and searched for inter-band time lags. A strong bluer-when-brighter chromatism was found on the intra-night time-scale. The spectral changes are not sensitive to the host galaxy contribution. Cross-correlation analysis revealed possible time delay of about 10 min between variations in the V and R bands. We interpreted the observed flares in terms of the model consisting of individual synchrotron pulses.

astro-ph.GA↗

arXiv2017

The Exit Distribution for Smart Kinetic Walk with Symmetric and Asymmetric Transition Probability

It has been proved that the distribution of the point where the Smart Kinetic Walk (SKW) exits a domain converges in distribution to harmonic measure on the hexagonal lattice. For other lattices, it is believed that this result still holds, and there is good numerical evidence to support this conjecture. Here we examine the effect of the symmetry and asymmetry of the transition probability on each step of the SKW on the square lattice and test if the exit distribution converges in distribution to harmonic measure as well. From our simulations, the limiting exit distribution of the SKW with a non-uniform but symmetric transition probability as the lattice spacing goes to zero is the harmonic measure. This result does not hold for asymmetric transition probability. We are also interested in the difference between the SKW with symmetric transition probability exit distribution and harmonic measure. Our simulations provide strong support for a explicit conjecture about this first order difference. The explicit formula for the conjecture will be given below.

math.PR↗

arXiv2012

Seven-Year Multi-Color Optical Monitoring of BL Lacertae Object S5 0716+714

We have monitored the BL Lac object S5 0716+714 in five intermediate optical wavebands from 2004 September to 2011 April. Here we present the data that include 8661 measurements which represents one of the largest databases obtained for an object at optical domain. A simple analysis of the data indicates that the object was active in most time, and intraday variability was frequently observed. In total, the object varied by 2.614 magnitudes in the $i$ band. Strong bluer-when-brighter chromatism was observed on long, intermediate, and short timescales.

astro-ph.HE↗

arXiv2010

Color Behavior Of BL Lacertae Object OJ 287 During Optical Outburst

This paper aims to study the color behavior of the BL Lac object OJ 287 during optical outburst. According to the revisit of the data from the OJ-94 monitoring project and the analysis the data obtained with the 60/90 cm Schmidt Telescope of NAOC, we found a bluer-when-brighter chromatism in this object. The amplitude of variation tends to decrease with the decrease of frequency. These results are consistent with the shock-in-jet model. We made some simulations and confirmed that both amplitude difference and time delay between variations at different wavelengths can result in the phenomenon of bluer-when-brighter. Our observations confirmed that OJ 287 underwent a double-peaked outburst after about 12 years from 1996, which provides further evidence for the binary black hole model in this object.

astro-ph.GA↗