arXiv subjects

Tao Yang

Publications and source records attributed to Tao Yang.

At least 181 records · Page 10Linked to original sources

arXiv2024

Examining the Commitments and Difficulties Inherent in Multimodal Foundation Models for Street View Imagery

The emergence of Large Language Models (LLMs) and multimodal foundation models (FMs) has generated heightened interest in their applications that integrate vision and language. This paper investigates the capabilities of ChatGPT-4V and Gemini Pro for Street View Imagery, Built Environment, and Interior by evaluating their performance across various tasks. The assessments include street furniture identification, pedestrian and car counts, and road width measurement in Street View Imagery; building function classification, building age analysis, building height analysis, and building structure classification in the Built Environment; and interior room classification, interior design style analysis, interior furniture counts, and interior length measurement in Interior. The results reveal proficiency in length measurement, style analysis, question answering, and basic image understanding, but highlight limitations in detailed recognition and counting tasks. While zero-shot learning shows potential, performance varies depending on the problem domains and image complexities. This study provides new insights into the strengths and weaknesses of multimodal foundation models for practical challenges in Street View Imagery, Built Environment, and Interior. Overall, the findings demonstrate foundational multimodal intelligence, emphasizing the potential of FMs to drive forward interdisciplinary applications at the intersection of computer vision and language.

cs.CV↗

arXiv2024

cc-DRL: a Convex Combined Deep Reinforcement Learning Flight Control Design for a Morphing Quadrotor

In comparison to common quadrotors, the shape change of morphing quadrotors endows it with a more better flight performance but also results in more complex flight dynamics. Generally, it is extremely difficult or even impossible for morphing quadrotors to establish an accurate mathematical model describing their complex flight dynamics. To figure out the issue of flight control design for morphing quadrotors, this paper resorts to a combination of model-free control techniques (e.g., deep reinforcement learning, DRL) and convex combination (CC) technique, and proposes a convex-combined-DRL (cc-DRL) flight control algorithm for position and attitude of a class of morphing quadrotors, where the shape change is realized by the length variation of four arm rods. In the proposed cc-DRL flight control algorithm, proximal policy optimization algorithm that is a model-free DRL algorithm is utilized to off-line train the corresponding optimal flight control laws for some selected representative arm length modes and hereby a cc-DRL flight control scheme is constructed by the convex combination technique. Finally, simulation results are presented to show the effectiveness and merit of the proposed flight control algorithm.

cs.RO↗

arXiv2024

Factorized-Dreamer: Training A High-Quality Video Generator with Limited and Low-Quality Data

Text-to-video (T2V) generation has gained significant attention due to its wide applications to video generation, editing, enhancement and translation, \etc. However, high-quality (HQ) video synthesis is extremely challenging because of the diverse and complex motions existed in real world. Most existing works struggle to address this problem by collecting large-scale HQ videos, which are inaccessible to the community. In this work, we show that publicly available limited and low-quality (LQ) data are sufficient to train a HQ video generator without recaptioning or finetuning. We factorize the whole T2V generation process into two steps: generating an image conditioned on a highly descriptive caption, and synthesizing the video conditioned on the generated image and a concise caption of motion details. Specifically, we present \emph{Factorized-Dreamer}, a factorized spatiotemporal framework with several critical designs for T2V generation, including an adapter to combine text and image embeddings, a pixel-aware cross attention module to capture pixel-level image information, a T5 text encoder to better understand motion description, and a PredictNet to supervise optical flows. We further present a noise schedule, which plays a key role in ensuring the quality and stability of video generation. Our model lowers the requirements in detailed captions and HQ videos, and can be directly trained on limited LQ datasets with noisy and brief captions such as WebVid-10M, largely alleviating the cost to collect large-scale HQ video-text pairs. Extensive experiments in a variety of T2V and image-to-video generation tasks demonstrate the effectiveness of our proposed Factorized-Dreamer. Our source codes are available at \url{https://github.com/yangxy/Factorized-Dreamer/}.

cs.CV↗

arXiv2024

Exact surface energy of the Hubbard model with nonparallel boundary magnetic fields

In this study, we explore the precise physical quantities in the thermodynamic limit of the one-dimensional Hubbard model with nonparallel boundary magnetic fields based on the off-diagonal Bethe ansatz solution. A particular emphasis is placed on the half-filling condition to investigate the distinct patterns of Bethe roots in the reduced Bethe ansatz equations for different boundary parameters. The ground state of the system can be divided into five regions according to the distribution of Bethe roots. By analyzing these patterns, we calculate the densities of states, ground-state energy density, and surface energy. The results reveal the existence of stableboundary bound states, which are dependent on specific constraints regarding the boundary magnetic fields.

math-ph↗

arXiv2024

Importance of eccentricities in parameter estimation of compact binary inspirals with decihertz gravitational-wave detectors

During its inspiral stage, a binary black hole (BBH) produces characteristic gravitational wave (GW) signals. The waveform of the GW signals can be described by the physical parameters of BBH, such as the masses of the black holes and the orbital eccentricity. Precise and accurate estimation of these parameters is crucial for GW astrophysics. In the aspect of precision, decihertz GW detectors are promising proposals, as they are anticipated to allow us to obtain highly precise parameter estimations for stellar-mass BBHs. However, the high-precision parameter estimation requires accurate GW waveform modeling. Otherwise, systematic errors can arise in estimated parameters. We emphasize the importance of considering the orbital eccentricity in constructing an accurate GW waveform model. B-DECIGO and MAGIS are used as benchmarks for decihertz GW detectors. We examine the significance of systematic error for a population of stellar-mass BBH inspirals. We found that the quasicircular GW waveform model exhibits significant systematic errors for BBH with a very small eccentricity $\sim 10^{-4}$ at GW frequency $0.1\, \text{Hz}$. The modeling accuracy can be substantially enhanced by incorporating the leading-order correction to GW phase evolution associated with eccentricity smaller than 0.01. The higher-order post-Newtonian corrections induced by eccentricity should be important only for eccentricity larger than 0.01.

gr-qc↗

arXiv2024

CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices

Recently, there has been a demand to deploy Large Language Models (LLMs) on personal devices such as laptops and smartphones. These LLMs have different model variants when handling different tasks. However, personal devices have limited resources and require reduced storage overhead. To address this, there are two key methods available: the first is model compression, which compresses LLMs into smaller sizes; the second is LoRA, which can transfer an LLM to other tasks with very few parameters, avoiding the storage of multiple model variants in multi-task scenarios by only preserving LoRAs. However, our experiments show that directly combining these two methods yields sub-optimal performance. Considering that the open-source community has already contributed many LoRAs to LLMs, we propose to adapt these existing LoRAs from the LLMs to their compressed version and introduce a Compression-Aware LoRA (CA-LoRA) framework. We incorporate knowledge inheritance and recovery strategies to recover the lost knowledge caused by model compression. Experiment results demonstrate that CA-LoRA outperforms the vanilla LoRA methods applied to a compressed LLM and achieves comparable performance to the non-compressed LLM with existing LoRA modules. The source code of CA-LoRA is available at https://github.com/thunlp/CA-LoRA.

cs.CL↗

arXiv2024

Computational Identification and Stuart-Landau Modeling of Collective Dynamical Behaviors of Octuple Laminar Diffusion Flame Oscillators

Annular chambers, consisting of multiple flame nozzles, are frequently used in many industrial processes, for example, rocket engines and gas turbines. In the study, we proposed a novel approach to the problem of annular combustion with emphasis on the collective dynamical behaviors that its individuals do not have. A series of circular arrays of octuple flickering laminar buoyant diffusion flames were investigated computationally and theoretically. Five distinct dynamical modes, such as the merged, in-phase mode, rotation, flickering death, partially flickering death, and anti-phase modes, were computationally identified and interpreted from the perspective of vortex dynamics. A unified regime diagram was obtained in terms of the normalized flame frequency f/f_0 and the combined parameter (α-1)Gr^1/2, where α=l/D is the ratio of the flame separation distance l to the flame nozzle diameter D and Gr is the Grashof number. The bifurcation transition from the in-phase mode and the anti-phase mode to the totally or partially flickering death occurs at (α-1)Gr^1/2=655+-55. In addition, a Stuart-Landau model with a time-delay coupling was utilized to reproduce the general features and collective modes of the octuple oscillators flame systems.

physics.flu-dyn↗

arXiv2024

Acoustic Bound States in the Continuum in Coupled Helmholtz Resonators

Resonant states underlie a variety of metastructures that exhibit remarkable capabilities for effective control of acoustic waves at subwavelength scales. The development of metamaterials relies on the rigorous mode engineering providing the implementation of the desired properties. At the same time, the application of metamaterials is still limited as their building blocks are frequently characterized by complicated geometry and can't be tuned easily. In this work, we consider a simple system of coupled Helmholtz resonators and study their properties associated with the tuning of coupling strength and symmetry breaking. We numerically and experimentally demonstrate the excitation of quasi-bound state in the continuum in the resonators placed in a free space and in a rectangular cavity. It is also shown that tuning the intrinsic losses via introducing porous inserts can lead to spectral splitting or merging of quasi-\textit{bound states in the continuum} and occurrence of \textit{exceptional points}. The obtained results will open new opportunities for the development of simple and easy-tunable metastructures based on Helmholtz resonances.

physics.app-ph↗

arXiv2024

Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization

Diffusion models have demonstrated impressive performance in various image generation, editing, enhancement and translation tasks. In particular, the pre-trained text-to-image stable diffusion models provide a potential solution to the challenging realistic image super-resolution (Real-ISR) and image stylization problems with their strong generative priors. However, the existing methods along this line often fail to keep faithful pixel-wise image structures. If extra skip connections between the encoder and the decoder of a VAE are used to reproduce details, additional training in image space will be required, limiting the application to tasks in latent space such as image stylization. In this work, we propose a pixel-aware stable diffusion (PASD) network to achieve robust Real-ISR and personalized image stylization. Specifically, a pixel-aware cross attention module is introduced to enable diffusion models perceiving image local structures in pixel-wise level, while a degradation removal module is used to extract degradation insensitive features to guide the diffusion process together with image high level information. An adjustable noise schedule is introduced to further improve the image restoration results. By simply replacing the base diffusion model with a stylized one, PASD can generate diverse stylized images without collecting pairwise training data, and by shifting the base model with an aesthetic one, PASD can bring old photos back to life. Extensive experiments in a variety of image enhancement and stylization tasks demonstrate the effectiveness of our proposed PASD approach. Our source codes are available at \url{https://github.com/yangxy/PASD/}.

cs.CV↗

arXiv2024

Susceptibility indicator for chiral topological orders emergent from correlated fermions

Chiral topological orders formed in correlated fermion systems have been widely explored. However, the mechanism on how they emerge from interacting fermions is still unclear. Here, we propose a susceptibility condition. Under this condition, we show that chiral topological orders can spontaneously take place in correlated fermion systems. The condition leads to a low-energy effective theory of bosons with strong frustration, mimicking the flat band systems. The frustration then melts the long-range orders and results in topological orders with time-reversal symmetry breaking. We apply the theory to strongly-correlated semiconductors doped to the metallic phase. A novel excitonic topological order with semionic excitations and chiral excitonic edge state is revealed. We also discuss the application to frustrated magnets. The theory predicts a chiral spin liquid state, which is numerically confirmed by our tensor network calculations. These results demonstrate an unprecedented indicator for chiral topological orders, which bridges the existing gap between interacting fermions and correlated topological matter.

cond-mat.str-el↗

arXiv2024

Distributed Event-Triggered Bandit Convex Optimization with Time-Varying Constraints

This paper considers the distributed bandit convex optimization problem with time-varying inequality constraints over a network of agents, where the goal is to minimize network regret and cumulative constraint violation. Existing distributed online algorithms require that each agent broadcasts its decision to its neighbors at each iteration. To better utilize the limited communication resources, we propose a distributed event-triggered online primal--dual algorithm with two-point bandit feedback. Under several classes of appropriately chosen decreasing parameter sequences and non-increasing event-triggered threshold sequences, we establish dynamic network regret and network cumulative constraint violation bounds. These bounds are comparable to the results achieved by distributed event-triggered online algorithms with full-information feedback. Finally, a numerical example is provided to verify the theoretical results.

math.OC↗

arXiv2024

Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement

Disentangled representation learning strives to extract the intrinsic factors within observed data. Factorizing these representations in an unsupervised manner is notably challenging and usually requires tailored loss functions or specific structural designs. In this paper, we introduce a new perspective and framework, demonstrating that diffusion models with cross-attention can serve as a powerful inductive bias to facilitate the learning of disentangled representations. We propose to encode an image to a set of concept tokens and treat them as the condition of the latent diffusion for image reconstruction, where cross-attention over the concept tokens is used to bridge the interaction between the encoder and diffusion. Without any additional regularization, this framework achieves superior disentanglement performance on the benchmark datasets, surpassing all previous methods with intricate designs. We have conducted comprehensive ablation studies and visualization analysis, shedding light on the functioning of this model. This is the first work to reveal the potent disentanglement capability of diffusion models with cross-attention, requiring no complex designs. We anticipate that our findings will inspire more investigation on exploring diffusion for disentangled representation learning towards more sophisticated data analysis and understanding.

cs.CV↗

arXiv2024

Weighted KL-Divergence for Document Ranking Model Refinement

Transformer-based retrieval and reranking models for text document search are often refined through knowledge distillation together with contrastive learning. A tight distribution matching between the teacher and student models can be hard as over-calibration may degrade training effectiveness when a teacher does not perform well. This paper contrastively reweights KL divergence terms to prioritize the alignment between a student and a teacher model for proper separation of positive and negative documents. This paper analyzes and evaluates the proposed loss function on the MS MARCO and BEIR datasets to demonstrate its effectiveness in improving the relevance of tested student models.

cs.IR↗

arXiv2024

Breaking through the learning plateaus of in-context learning in Transformer

In-context learning, i.e., learning from context examples, is an impressive ability of Transformer. Training Transformers to possess this in-context learning skill is computationally intensive due to the occurrence of learning plateaus, which are periods within the training process where there is minimal or no enhancement in the model's in-context learning capability. To study the mechanism behind the learning plateaus, we conceptually seperate a component within the model's internal representation that is exclusively affected by the model's weights. We call this the "weights component", and the remainder is identified as the "context component". By conducting meticulous and controlled experiments on synthetic tasks, we note that the persistence of learning plateaus correlates with compromised functionality of the weights component. Recognizing the impaired performance of the weights component as a fundamental behavior drives learning plateaus, we have developed three strategies to expedite the learning of Transformers. The effectiveness of these strategies is further confirmed in natural language processing tasks. In conclusion, our research demonstrates the feasibility of cultivating a powerful in-context learning ability within AI systems in an eco-friendly manner.

cs.LG↗

arXiv2024

SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution

Owe to the powerful generative priors, the pre-trained text-to-image (T2I) diffusion models have become increasingly popular in solving the real-world image super-resolution problem. However, as a consequence of the heavy quality degradation of input low-resolution (LR) images, the destruction of local structures can lead to ambiguous image semantics. As a result, the content of reproduced high-resolution image may have semantic errors, deteriorating the super-resolution performance. To address this issue, we present a semantics-aware approach to better preserve the semantic fidelity of generative real-world image super-resolution. First, we train a degradation-aware prompt extractor, which can generate accurate soft and hard semantic prompts even under strong degradation. The hard semantic prompts refer to the image tags, aiming to enhance the local perception ability of the T2I model, while the soft semantic prompts compensate for the hard ones to provide additional representation information. These semantic prompts encourage the T2I model to generate detailed and semantically accurate results. Furthermore, during the inference process, we integrate the LR images into the initial sampling noise to mitigate the diffusion model's tendency to generate excessive random details. The experiments show that our method can reproduce more realistic image details and hold better the semantics. The source code of our method can be found at https://github.com/cswry/SeeSR.

cs.CV↗

arXiv2024

Eccentricity enables the earliest warning and localization of gravitational waves with ground-based detectors

The early and precise localization of gravitational waves (GWs) is pivotal in detecting their electromagnetic (EM) counterparts, especially for binary neutron stars (BNS) and neutron star-black hole binaries (NSBH). In this letter, we pioneer the exploration of utilizing the higher harmonic modes induced by the eccentricity of compact binaries to localize GWs with ground-based detectors even before the quadrupole baseline $\ell=2$ mode enters the detector band. Our theoretical analysis marks a first in proposing a strategy for gaining the earliest possible warning and maximizing preparation time for observing pre- and/or post-merger EM counterparts. We simulate three typical binaries from GWTC-3 with eccentricities ranging from 0.05 to 0.4. Our results reveal that the third-generation (3G) detectors (low frequency cut-off $f_0=5$ Hz) can accumulate sufficient signal-to-noise ratios through higher modes before the onset of the baseline $\ell=2$ mode entry into the band. Notably, relying solely on the higher modes, the 3G detector network ET+2CE achieves an average localization on the order of $1-10^2~\rm deg^2$ around 1-1.8 hours before the merger of a GW170817-like BNS, and $10-10^3~\rm deg^2$ approximately 18-30 minutes prior to the merger of a GW200115-like NSBH. A $100~\rm deg^2$ localization is attainable even 2-4 hours prior to a BNS merger. Moreover, in the near face-on orientations which are generally more favorable for EM counterpart detection, the localization can be further improved.

gr-qc↗

arXiv2024

Distributed online constrained convex optimization with event-triggered communication

This paper focuses on the distributed online convex optimization problem with time-varying inequality constraints over a network of agents, where each agent collaborates with its neighboring agents to minimize the cumulative network-wide loss over time. To reduce communication overhead between the agents, we propose a distributed event-triggered online primal-dual algorithm over a time-varying directed graph. With several classes of appropriately chose decreasing parameter sequences and non-increasing event-triggered threshold sequences, we establish dynamic network regret and network cumulative constraint violation bounds. Finally, a numerical simulation example is provided to verify the theoretical results.

math.OC↗

arXiv2024

Approximate Cluster-Based Sparse Document Retrieval with Segmented Maximum Term Weights

This paper revisits cluster-based retrieval that partitions the inverted index into multiple groups and skips the index partially at cluster and document levels during online inference using a learned sparse representation. It proposes an approximate search scheme with two parameters to control the rank-safeness competitiveness of pruning with segmented maximum term weights within each cluster. Cluster-level maximum weight segmentation allows an improvement in the rank score bound estimation and threshold-based pruning to be approximately adaptive to bound estimation tightness, resulting in better relevance and efficiency. The experiments with MS MARCO passage ranking and BEIR datasets demonstrate the usefulness of the proposed scheme with a comparison to the baselines. This paper presents the design of this approximate retrieval scheme with rank-safeness analysis, compares clustering and segmentation options, and reports evaluation results.

cs.IR↗