arXiv subjects

Tao Wang

Publications and source records attributed to Tao Wang.

At least 415 records · Page 23Linked to original sources

arXiv2024

MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation

Foley audio, critical for enhancing the immersive experience in multimedia content, faces significant challenges in the AI-generated content (AIGC) landscape. Despite advancements in AIGC technologies for text and image generation, the foley audio dubbing remains rudimentary due to difficulties in cross-modal scene matching and content correlation. Current text-to-audio technology, which relies on detailed and acoustically relevant textual descriptions, falls short in practical video dubbing applications. Existing datasets like AudioSet, AudioCaps, Clotho, Sound-of-Story, and WavCaps do not fully meet the requirements for real-world foley audio dubbing task. To address this, we introduce the Multi-modal Image and Narrative Text Dubbing Dataset (MINT), designed to enhance mainstream dubbing tasks such as literary story audiobooks dubbing, image/silent video dubbing. Besides, to address the limitations of existing TTA technology in understanding and planning complex prompts, a Foley Audio Content Planning, Generation, and Alignment (CPGA) framework is proposed, which includes a content planning module leveraging large language models for complex multi-modal prompts comprehension. Additionally, the training process is optimized using Proximal Policy Optimization based reinforcement learning, significantly improving the alignment and auditory realism of generated foley audio. Experimental results demonstrate that our approach significantly advances the field of foley audio dubbing, providing robust solutions for the challenges of multi-modal dubbing. Even when utilizing the relatively lightweight GPT-2 model, our framework outperforms open-source multimodal large models such as LLaVA, DeepSeek-VL, and Moondream2. The dataset is available at https://github.com/borisfrb/MINT .

eess.AS↗

arXiv2024

Existence and uniqueness of ground state solutions for the planar Schrödinger-Newton equation on the disc

This paper is concerned with the existence and qualitative properties of positive ground state solutions for the planar Schrödinger-Newton equation on the disc. First, we prove the existence and radial symmetry of all the positive ground state solutions by employing the symmetric decreasing rearrangement and Talenti's inequality. Next, we develop Newton's theorem and then use the contraction mapping principle to establish the uniqueness of the positive ground state solution for the Schrödinger-Newton equation on the disc in the two dimensional case. Finally, we show that the unique positive ground state solution converges to the trivial solution as the radius $R$ tending to infinity, which is totally different from the higher dimensional case in \cite{Guo-Wang-Yi}.

math.AP↗

arXiv2024

PPPR: Portable Plug-in Prompt Refiner for Text to Audio Generation

Text-to-Audio (TTA) aims to generate audio that corresponds to the given text description, playing a crucial role in media production. The text descriptions in TTA datasets lack rich variations and diversity, resulting in a drop in TTA model performance when faced with complex text. To address this issue, we propose a method called Portable Plug-in Prompt Refiner, which utilizes rich knowledge about textual descriptions inherent in large language models to effectively enhance the robustness of TTA acoustic models without altering the acoustic training set. Furthermore, a Chain-of-Thought that mimics human verification is introduced to enhance the accuracy of audio descriptions, thereby improving the accuracy of generated content in practical applications. The experiments show that our method achieves a state-of-the-art Inception Score (IS) of 8.72, surpassing AudioGen, AudioLDM and Tango.

cs.SD↗

arXiv2024

Controlled Decoding from Language Models

KL-regularized reinforcement learning (RL) is a popular alignment framework to control the language model responses towards high reward outcomes. We pose a tokenwise RL objective and propose a modular solver for it, called controlled decoding (CD). CD exerts control through a separate prefix scorer module, which is trained to learn a value function for the reward. The prefix scorer is used at inference time to control the generation from a frozen base model, provably sampling from a solution to the RL objective. We empirically demonstrate that CD is effective as a control mechanism on popular benchmarks. We also show that prefix scorers for multiple rewards may be combined at inference time, effectively solving a multi-objective RL problem with no additional training. We show that the benefits of applying CD transfer to an unseen base model with no further tuning as well. Finally, we show that CD can be applied in a blockwise decoding fashion at inference-time, essentially bridging the gap between the popular best-of-K strategy and tokenwise control through reinforcement learning. This makes CD a promising approach for alignment of language models.

cs.LG↗

arXiv2024

JWST's first glimpse of a z > 2 forming cluster reveals a top-heavy stellar mass function

Clusters and their progenitors (protoclusters) at z = 2-4, the peak epoch of star formation, are ideal laboratories to study the formation process of both the clusters themselves and their member galaxies. However, a complete census of their member galaxies has been challenging due to observational difficulties. Here we present new JWST/NIRCam observations targeting the distant cluster CLJ1001 at z = 2.51 from the COSMOS-Web program, which, in combination with previous narrowband imaging targeting H-alpha emitters and deep millimeter surveys of CO emitters, provide a complete view of massive galaxy assembly in CLJ1001. In particular, JWST reveals a population of massive, extremely red cluster members in the long-wavelength bands that were invisible in previous Hubble Space Telescope (HST)/F160W imaging (HST-dark members). Based on this highly complete spectroscopic sample of member galaxies, we show that the spatial distribution of galaxies in CLJ1001 exhibits a strong central concentration, with the central galaxy density already resembling that of low-z clusters. Moreover, we reveal a "top-heavy" stellar mass function for the star-forming galaxies (SFGs), with an overabundance of massive SFGs piled up in the cluster core. These features strongly suggest that CLJ1001 is caught in a rapid transition, with many of its massive SFGs likely soon becoming quiescent. In the context of cluster formation, these findings suggest that the earliest clusters form from the inside out and top to bottom, with the massive galaxies in the core assembling first, followed by the less massive ones in the outskirts.

astro-ph.GA↗

arXiv2024

Mollification Effects of Policy Gradient Methods

Policy gradient methods have enabled deep reinforcement learning (RL) to approach challenging continuous control problems, even when the underlying systems involve highly nonlinear dynamics that generate complex non-smooth optimization landscapes. We develop a rigorous framework for understanding how policy gradient methods mollify non-smooth optimization landscapes to enable effective policy search, as well as the downside of it: while making the objective function smoother and easier to optimize, the stochastic objective deviates further from the original problem. We demonstrate the equivalence between policy gradient methods and solving backward heat equations. Following the ill-posedness of backward heat equations from PDE theory, we present a fundamental challenge to the use of policy gradient under stochasticity. Moreover, we make the connection between this limitation and the uncertainty principle in harmonic analysis to understand the effects of exploration with stochastic policies in RL. We also provide experimental results to illustrate both the positive and negative aspects of mollification effects in practice.

cs.LG↗

arXiv2024

Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression

Key-value~(KV) caching is an important technique to accelerate the inference of large language models~(LLMs), but incurs significant memory overhead. To compress the size of KV cache, existing methods often compromise precision or require extra data for calibration, limiting their practicality in LLM deployment. In this paper, we introduce \textbf{DecoQuant}, a novel data-free low-bit quantization technique based on tensor decomposition methods, to effectively compress KV cache. Our core idea is to adjust the outlier distribution of the original matrix by performing tensor decomposition, so that the quantization difficulties are migrated from the matrix to decomposed local tensors. Specially, we find that outliers mainly concentrate on small local tensors, while large tensors tend to have a narrower value range. Based on this finding, we propose to apply low-bit quantization to the large tensor, while maintaining high-precision representation for the small tensor. Furthermore, we utilize the proposed quantization method to compress the KV cache of LLMs to accelerate the inference and develop an efficient dequantization kernel tailored specifically for DecoQuant. Through extensive experiments, DecoQuant demonstrates remarkable efficiency gains, showcasing up to a $\sim$75\% reduction in memory footprint while maintaining comparable generation quality.

cs.CL↗

arXiv2024

Dynamic Loss Decay based Robust Oriented Object Detection on Remote Sensing Images with Noisy Labels

The ambiguous appearance, tiny scale, and fine-grained classes of objects in remote sensing imagery inevitably lead to the noisy annotations in category labels of detection dataset. However, the effects and treatments of the label noises are underexplored in modern oriented remote sensing object detectors. To address this issue, we propose a robust oriented remote sensing object detection method through dynamic loss decay (DLD) mechanism, inspired by the two phase ``early-learning'' and ``memorization'' learning dynamics of deep neural networks on clean and noisy samples. To be specific, we first observe the end point of early learning phase termed as EL, after which the models begin to memorize the false labels that significantly degrade the detection accuracy. Secondly, under the guidance of the training indicator, the losses of each sample are ranked in descending order, and we adaptively decay the losses of the top K largest ones (bad samples) in the following epochs. Because these large losses are of high confidence to be calculated with wrong labels. Experimental results show that the method achieves excellent noise resistance performance tested on multiple public datasets such as HRSC2016 and DOTA-v1.0/v2.0 with synthetic category label noise. Our solution also has won the 2st place in the "fine-grained object detection based on sub-meter remote sensing imagery" track with noisy labels of 2023 National Big Data and Computing Intelligence Challenge.

cs.CV↗

arXiv2024

Martian seismic anisotropy underneath Elysium Planitia revealed by direct S wave splitting

Seismic anisotropy, arising from the crystallographic or lattice-preferred orientation of anisotropic minerals or the shape-preferred orientation of melts or cracks, can establish a critical link between Mars's past evolution and its current state. So far, although seismic anisotropy in Mars has been proposed due to different velocities of vertically and horizontally polarized shear waves in the Martian crust, obtained from crustal converted waves, multiples, and surface waves recorded by the InSight seismometer, the evidence is plausible. Notably, the shear wave splitting, which stands out as a straight indicator of seismic anisotropy, has not been reported using marsquake records. In this study, we employ Low-frequency marsquakes detected by the InSight seismometer to reveal shear wave splitting in Mars. We find that the direct S waves of three marsquake recordings (S0173a, S0235b, and S1133c) with high signal-to-noise ratios exhibit the splitting pheonmenon. We rule out the possibility of apparent anisotropy through synthetic tests, affirming the presence of seismic anisotropy in Mars. The delay time (about 1.33 s on average) measured from the direct S wave splitting is too large to be solely attributed to the seismic anisotropy in the upper crust (0 - 10 km) beneath the InSight. Thus, seismic anisotropy in the deeper region of Mars is indispensable. Combined with other geophysical evidence near the InSight landing site, the strong seismic anisotropy observed in this study implies the porous crust with aligned cracks being greater than 10 km beneath the InSight and/or the presence of an active mantle plume underneath the Elysium Planitia of Mars.

physics.geo-ph↗

arXiv2024

Neural Estimation Of Entropic Optimal Transport

Optimal transport (OT) serves as a natural framework for comparing probability measures, with applications in statistics, machine learning, and applied mathematics. Alas, statistical estimation and exact computation of the OT distances suffer from the curse of dimensionality. To circumvent these issues, entropic regularization has emerged as a remedy that enables parametric estimation rates via plug-in and efficient computation using Sinkhorn iterations. Motivated by further scaling up entropic OT (EOT) to data dimensions and sample sizes that appear in modern machine learning applications, we propose a novel neural estimation approach. Our estimator parametrizes a semi-dual representation of the EOT distance by a neural network, approximates expectations by sample means, and optimizes the resulting empirical objective over parameter space. We establish non-asymptotic error bounds on the EOT neural estimator of the cost and optimal plan. Our bounds characterize the effective error in terms of neural network size and the number of samples, revealing optimal scaling laws that guarantee parametric convergence. The bounds hold for compactly supported distributions and imply that the proposed estimator is minimax-rate optimal over that class. Numerical experiments validating our theory are also provided.

math.ST↗

arXiv2024

Investigation of the deposition of $α$-tantalum (110) films on a-plane sapphire substrate by molecular beam epitaxy for superconducting circuit

Polycrystalline α-tantalum (110) films deposited on c-plane sapphire substrate by sputtering are used in superconducting qubits nowadays. However, these films always occasionally form other structures, such as α-tantalum (111) grains and \b{eta}-tantalum grains. To improve the film quality, we investigate the growth of α-tantalum (110) films on a-plane sapphire substrate under varying conditions by molecular beam epitaxy technology. The optimized α-tantalum (110) film is single crystal, with a smooth surface and atomically flat metal-substrate interface. The film with thickness of 30 nm shows a Tc of 4.12K and a high residual resistance ratio of 9.53. The quarter wavelength coplanar waveguide resonators fabricated with the 150 nm optimized α-tantalum (110) film, exhibits intrinsic quality factor of over one million under single photon excitation at millikelvin temperature.

physics.app-ph↗

arXiv2024

Stable and low loss oxide layer on α-Ta (110) film for superconducting qubits

The presence of amorphous oxide layers can significantly affect the coherent time of superconducting qubits due to their high dielectric loss. Typically, the surface oxides of superconductor films exhibit lossy and unstable behavior when exposed to air. To increase the coherence time, it is essential for qubits to have stable and low dielectric loss oxides, either as barrier or passivation layers. In this study, we highlight the robust and stable nature of an amorphous tantalum oxide layer formed on α-Ta (110) film by employing chemical and structural analyses. Such kind of oxide layer forms in a self-limiting process on the surface of α-Ta (110) film in piranha solution, yielding stable thickness and steady chemical composition. Quarter wavelength coplanar waveguide resonators are made to study the loss of this oxide. One resonator has a Qi of 3.0x10^6 in the single photon region. The Qi of most devices are higher than 2.0x10^6. Moreover, most of them are still over 1x10^6 even after exposed to air for months. Based on these findings, we propose an all-tantalum superconducting qubit utilizing such oxide as passivation layers, which possess low dielectric loss and improved stability.

cond-mat.mtrl-sci↗

arXiv2024

Epitaxial α-Ta (110) film on a-plane sapphire substrate for superconducting qubits on wafer scale

Realization of practical superconducting quantum computing requires many qubits of long coherence time. Compared to the commonly used Ta deposited on c-plane sapphire, which occasionally form α-Ta (111) grains and \b{eta}-tantalum grains, high quality Ta (110) film can grow epitaxial on a-plane sapphire because of the atomic relationships at the interface. Well-ordered α-Ta (110) film on wafer-scale a-plane sapphire has been prepared. The film exhibits high residual resistance ratio. Transmon qubits fabricated using these film shows relaxation times exceeding 150 μs. The results suggest Ta film on a-plane sapphire is a promising choice for long coherence time qubit on wafer scale.

quant-ph↗

arXiv2024

Noema formIng Cluster survEy (NICE): Discovery of a starbursting galaxy group with a radio-luminous core at z=3.95

The study of distant galaxy groups and clusters at the peak epoch of star formation is limited by the lack of a statistically and homogeneously selected and spectroscopically confirmed sample. Recent discoveries of concentrated starburst activities in cluster cores have opened a new window to hunt for these structures based on their integrated IR luminosities. Hereby we carry out the large NOEMA (NOrthern Extended Millimeter Array) program targeting a statistical sample of infrared-luminous sources associated with overdensities of massive galaxies at z>2, the Noema formIng Cluster survEy (NICE). We present the first result from the ongoing NICE survey, a compact group at z=3.95 in the Lockman Hole field (LH-SBC3), confirmed via four massive (M_star>10^10.5M_sun) galaxies detected in CO(4-3) and [CI](1-0) lines. The four CO-detected members of LH-SBC3 are distributed over a 180 kpc physical scale, and the entire structure has an estimated halo mass of ~10^13Msun and total star formation rate (SFR) of ~4000Msun/yr. In addition, the most massive galaxy hosts a radio-loud AGN with L_1.4GHz, rest = 3.0*10^25W/Hz. The discovery of LH-SBC3 demonstrates the feasibility of our method to efficiently identify high-z compact groups or forming cluster cores. The existence of these starbursting cluster cores up to z~4 provides critical insights into the mass assembly history of the central massive galaxies in clusters.

astro-ph.GA↗

arXiv2024

USmorph: An Updated Framework of Automatic Classification of Galaxy Morphologies and Its Application to Galaxies in the COSMOS Field

Morphological classification conveys abundant information on the formation, evolution, and environment of galaxies. In this work, we refine the two-step galaxy morphological classification framework ({\tt\string USmorph}), which employs a combination of unsupervised machine learning (UML) and supervised machine learning (SML) techniques, along with a self-consistent and robust data preprocessing step. The updated method is applied to the galaxies with $I_{\rm mag}<25$ at $0.2<z<1.2$ in the COSMOS field. Based on their HST/ACS I-band images, we classify them into five distinct morphological types: spherical (SPH, 15,200), early-type disk (ETD, 17,369), late-type disk (LTD, 21,143), irregular disk (IRR, 28,965), and unclassified (UNC, 17,129). In addition, we have conducted both parametric and nonparametric morphological measurements. For galaxies with stellar masses exceeding $10^{9}M_{\sun}$, a gradual increase in effective radius from SPHs to IRRs is observed, accompanied by a decrease in the Sérsic index. Nonparametric morphologies reveal distinct distributions of galaxies across the $Gini-M_{20}$ and $C-A$ parameter spaces for different categories. Moreover, different categories exhibit significant dissimilarity in their $G_2$ and $Ψ$ distributions. We find morphology to be strongly correlated with redshift and stellar mass. The consistency of these classification results with expected correlations among multiple parameters underscores the validity and reliability of our classification method, rendering it a valuable tool for future studies.

astro-ph.GA↗

arXiv2024

Emerging γ-soft-like spectrum in 196Pt in the SU3-IBM (I)

Recently, it has been argued that a spherical-like spectrum emerges in the SU3-IBM, opening up new approaches to understand the γ-softness in realistic nuclei. In a previous paper, γ-softness with degeneracy of the ground and quasi-γ bands was observed. In this paper, another special point connected with the middle degenerate point is discussed, which is found to be related with the properties of 196Pt. This emergent γ-softness has also been shown to be important for understanding the prolate-oblate asymmetric shape phase transition. The low-lying spectra, B(E2) values and quadrupole moments in 196Pt are discussed showing that the new model can account for several observed features. This is the first part of the discussions on the γ-soft-like spectrum of 196Pt.

nucl-th↗

arXiv2024

Deep Reinforcement Learning-aided Transmission Design for Energy-efficient Link Optimization in Vehicular Communications

This letter presents a deep reinforcement learning (DRL) approach for transmission design to optimize the energy efficiency in vehicle-to-vehicle (V2V) communication links. Considering the dynamic environment of vehicular communications, the optimization problem is non-convex and mathematically difficult to solve. Hence, we propose scenario identification-based double and Dueling deep Q-Network (SI-D3QN), a DRL algorithm integrating both double deep Q-Network and Dueling deep Q-Network, for the joint design of modulation and coding scheme (MCS) selection and power control. To be more specific, we employ SI techique to enhance link performance and assit the D3QN agent in refining its decision-making processes. The experiment results demonstrate that, across various optimization tasks, our proposed SI-D3QN agent outperforms the benchmark algorithms in terms of the valid actions and link performance metrics. Particularly, while ensuring significant improvement in energy efficiency, the agent facilitates a 29.6% enhancement in the link throughput under the same energy consumption.

eess.SP↗

arXiv2024

Floquet engineering tunable periodic gauge fields and simulating real topological phases in cold alkaline-earth atom optical lattice

We propose to synthesize tunable periodic gauge fields via Floquet engineering cold alkaline-earth atoms in one-dimensional optical lattice. The artificial magnetic flux is designed to emerge during the combined process of Floquet photon assisted tunneling and internal state transitions. By varying initial phases of driving protocol, our proposal presents the ability to smoothly tune the periodic flux. Moreover, we demonstrate that the effective two-leg flux ladder model can simulate one typical real topological insulator, which is described by the first Stiefel Whitney class and protected by the $PT$ symmetry. Benefiting from the long lifetime of excited states of alkaline-earth atoms, our work opens new possibilities for exploiting the physics related to gauge fields, such as topological phases, in the current cold atom platform.

cond-mat.quant-gas↗