arXiv subjects

Hui Li

Publications and source records attributed to Hui Li.

At least 343 records · Page 19Linked to original sources

arXiv2024

Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation

Recent advances in latent diffusion-based generative models for portrait image animation, such as Hallo, have achieved impressive results in short-duration video synthesis. In this paper, we present updates to Hallo, introducing several design enhancements to extend its capabilities. First, we extend the method to produce long-duration videos. To address substantial challenges such as appearance drift and temporal artifacts, we investigate augmentation strategies within the image space of conditional motion frames. Specifically, we introduce a patch-drop technique augmented with Gaussian noise to enhance visual consistency and temporal coherence over long duration. Second, we achieve 4K resolution portrait video generation. To accomplish this, we implement vector quantization of latent codes and apply temporal alignment techniques to maintain coherence across the temporal dimension. By integrating a high-quality decoder, we realize visual synthesis at 4K resolution. Third, we incorporate adjustable semantic textual labels for portrait expressions as conditional inputs. This extends beyond traditional audio cues to improve controllability and increase the diversity of the generated content. To the best of our knowledge, Hallo2, proposed in this paper, is the first method to achieve 4K resolution and generate hour-long, audio-driven portrait image animations enhanced with textual prompts. We have conducted extensive experiments to evaluate our method on publicly available datasets, including HDTF, CelebV, and our introduced "Wild" dataset. The experimental results demonstrate that our approach achieves state-of-the-art performance in long-duration portrait video animation, successfully generating rich and controllable content at 4K resolution for duration extending up to tens of minutes. Project page https://fudan-generative-vision.github.io/hallo2

cs.CV↗

arXiv2024

Filaments in and between galaxy clusters at low and mid-frequency with the SKA telescope

Understanding the magnetised Universe is a major challenge in modern astrophysics, and cosmic magnetism has been acknowledged as one of the science key drivers of the most ambitious radio instrument ever planned, the SKA telescope. With this work, we aim to investigate the potential of the SKA telescope and its precursors and pathfinders in the study of magnetic fields in galaxy clusters and filaments through diffuse synchrotron radio emission. Galaxy clusters and filaments of the cosmic web are indeed unique laboratories to investigate turbulent fluid motions and large-scale magnetic fields in action and much of what is known about magnetic fields in galaxy clusters comes from sensitive radio observations. Based on cosmological MHD simulations, we predict radio properties (total intensity and polarisation) of a pair of galaxy clusters connected by a cosmic-web filament. We use our theoretical expectations to explore the potential of polarimetric observations to study large-scale structure magnetic fields in the frequency ranges 50-350MHz and 950-1760MHz. We also present predictions for galaxy cluster polarimetric observations with the SKA precursors and pathfinders (LOFAR2.0 and MeerKAT+). Our findings point out that polarisation observations are particularly powerful for the study of large-scale magnetic fields, since they are not significantly affected by confusion noise. The unprecedented sensitivity and spatial resolution of the intermediate frequency radio telescopes make them the favourite instruments for the study of these sources through polarimetric data, potentially allowing us to understand if the energy density of relativistic electrons is in equipartition with the magnetic field or coupled with the thermal gas density. Our results show that low frequency instruments represent as well a precious tool to study diffuse synchrotron emission in total intensity and polarisation.

astro-ph.CO↗

arXiv2024

The fundamental groups of presymplectic Hamiltonian G-manifolds

We consider presymplectic manifolds equipped with Hamiltonian G-actions, G being a connected compact Lie group. A presymplectic manifold is foliated by the integral submanifolds of the kernel of the presymplectic form. For a presymplectic Hamiltonian G-manifold, Lin and Sjamaar propose a condition under which they show that the moment map image has the same "convex and polyhedral" property as the moment map image of a symplectic Hamiltonian G-manifold, a result proved independently by Atiyah, Guillemin-Sternberg, and Kirwan. In this paper, under the condition Lin and Sjamaar proposed on presymplectic Hamiltonian G-manifolds, we study the fundamental groups of such manifolds, comparing with earlier results on the fundamental groups of symplectic Hamiltonian G-manifolds. We observe that the results on the symplectic case are special cases of the results on the presymplectic case.

math.SG↗

arXiv2024

Analytical strong line diagnostics and their redshift evolution

The \textit{JWST} is allowing new measurements of gas-phase metallicities in galaxies between cosmic noon and cosmic dawn. The most robust approach uses luminosity ratios between the excited auroral transition, [\oiii] 4364\,Å, and the lower [\oiii] 5008\,Å/4960\,Å lines to determine the gas temperature. The ratio of the luminosities in the latter transitions to those in hydrogen Balmer series lines then yield relatively clean metallicity estimates. In the absence of detection of the [\oiii] auroral line, the ratios of various [\oiii], [\oii], [\nii], and Balmer lines are used to determine metallicities. Here we present a refined approach for extracting metallicities from these ``strong line diagnostics''. Our method exploits empirical correlations between the temperature of \oiii/\oii\ regions and gas-phase metallicity. We then show, from first principles, how to extract metallicities and break degeneracies in these estimates using traditional strong line diagnostics, R2, R3, R23, and O3O2, and N2O2. We show that these ratios depend also on volume correction factors, i.e. on accounting for the fraction of the volume of HII regions that are in \oiii\ and \oii, but that these can be determined self-consistently along with the metallicities. We quantify the success of our method using metallicities derived from galaxies with auroral line determinations and show that it generally works better than previous empirical approaches. The scatter in the observed line ratios and redshift evolution are largely explained by O3O2 variations. We provide publicly available routines for extracting metallicities from strong line diagnostics using our methodology.

astro-ph.GA↗

arXiv2024

In-Context Imitation Learning via Next-Token Prediction

We explore how to enhance next-token prediction models to perform in-context imitation learning on a real robot, where the robot executes new tasks by interpreting contextual information provided during the input phase, without updating its underlying policy parameters. We propose In-Context Robot Transformer (ICRT), a causal transformer that performs autoregressive prediction on sensorimotor trajectories without relying on any linguistic data or reward function. This formulation enables flexible and training-free execution of new tasks at test time, achieved by prompting the model with sensorimotor trajectories of the new task composing of image observations, actions and states tuples, collected through human teleoperation. Experiments with a Franka Emika robot demonstrate that the ICRT can adapt to new tasks specified by prompts, even in environment configurations that differ from both the prompt and the training data. In a multitask environment setup, ICRT significantly outperforms current state-of-the-art next-token prediction models in robotics on generalizing to unseen tasks. Code, checkpoints and data are available on https://icrt.dev/

cs.RO↗

arXiv2024

Thermal Conductivity of Cubic Silicon Carbide Single Crystals Heavily Doped by Nitrogen

High-purity cubic silicon carbide possesses the second-highest thermal conductivity among large-scale crystals, surpassed only by diamond, making it crucial for practical applications of thermal management. Recent theoretical studies predict that heavy doping reduces the thermal conductivity of 3C-SiC via phonon-defect and phonon-electron scattering. However, experimental evidence has been limited. In this work, we report the thermal conductivity of heavily nitrogen doped 3C SiC single crystals, grown using the top-seeded solution growth method, measured via time domain thermoreflectance. Our results show that a significant reduction (up to 30%) in thermal conductivity is observed with nitrogen doping concentrations around 1020 cm-3. A comparison with theoretical calculations indicates less intensive scatterings are observed in the measured thermal conductivity. We speculate that the electron-phonon scattering may have a smaller impact than previously anticipated or the distribution of defects are nonuniform which leads to less intensive scatterings. These findings shed light on understanding the doping effects on thermal transport in semiconductors and support further exploration of 3C SiC for thermal management in electronics.

cond-mat.mtrl-sci↗

arXiv2024

Post-$GW$ theory and its application to pseudogap in strongly correlated system

The $GW$ approximation is a widely used framework for studying correlated materials, but it struggles with certain limitations, such as its inability to explain pseudogap phenomena. To overcome these problems, we propose a systematic theoretical framework for Green's function corrections and apply it specifically to the $GW$ approximation. In this new theory, the screened potential is reconnected to the physical response function, i.e. the covariant response function proposed in \cite{cGW_2023}, rather than using the RPA formula. We apply our scheme to calculate Green's function, the spectral function, and the charge compressibility in the two-dimensional Hubbard model. Our scheme yields significant qualitative and quantitative improvements over the standard $GW$ method and successfully captures the pseudogap behavior.

cond-mat.str-el↗

arXiv2024

Agents in Software Engineering: Survey, Landscape, and Vision

In recent years, Large Language Models (LLMs) have achieved remarkable success and have been widely used in various downstream tasks, especially in the tasks of the software engineering (SE) field. We find that many studies combining LLMs with SE have employed the concept of agents either explicitly or implicitly. However, there is a lack of an in-depth survey to sort out the development context of existing works, analyze how existing works combine the LLM-based agent technologies to optimize various tasks, and clarify the framework of LLM-based agents in SE. In this paper, we conduct the first survey of the studies on combining LLM-based agents with SE and present a framework of LLM-based agents in SE which includes three key modules: perception, memory, and action. We also summarize the current challenges in combining the two fields and propose future opportunities in response to existing challenges. We maintain a GitHub repository of the related papers at: https://github.com/DeepSoftwareAnalytics/Awesome-Agent4SE.

cs.SE↗

arXiv2024

Seed-Music: A Unified Framework for High Quality and Controlled Music Generation

We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled music generation and post-production editing. For controlled music generation, our system enables vocal music generation with performance controls from multi-modal inputs, including style descriptions, audio references, musical scores, and voice prompts. For post-production editing, it offers interactive tools for editing lyrics and vocal melodies directly in the generated audio. We encourage readers to listen to demo audio examples at https://team.doubao.com/seed-music "https://team.doubao.com/seed-music".

cs.SD↗

arXiv2024

Formation of the Supersonic Solar Wind: Parker's Theory Revisited

We examine and propose to fundamentally modify the classical theory of solar wind formation. To form a supersonic solar wind, the classical theory requires that a subsonic flow speed must start at a specific initial speed from the coronal base, called eigenspeed, go along a continuous eigenfunction, and reach the sonic point, which is where the flow speed equals the sonic speed, while the critical condition, which is where the effective driving force is zero, is satisfied. Any mismatch between the sonic point and critical condition distances results in either subsonic winds when the initial speed is below the eigenspeed, or no solar wind when the initial speed is above the eigenspeed. Because the critical condition is determined by the solar wind temperature profile, which depends on ionization process at the top of the chromosphere and the heating process around the coronal base but not by the processes at the sonic point, the required match between the two is generally not met and hence the momentum equation in the conventional theory encounters difficulty when the initial speed is above the eigenspeed. To resolve the difficulty, we propose a discontinuity between the sonic point and the critical condition to reach supersonic solar wind solutions. As a result, supersonic solar winds can be produced when the initial speed in the coronal base is greater than the eigenspeed. The critical solution or eigen function provided by the conventional solar wind model describes the condition that separates the supersonic solar winds from subsonic ones.

astro-ph.SR↗

arXiv2024

Real-Time and Accurate: Zero-shot High-Fidelity Singing Voice Conversion with Multi-Condition Flow Synthesis

Singing voice conversion is to convert the source singing voice into the target singing voice except for the content. Currently, flow-based models can complete the task of voice conversion, but they struggle to effectively extract latent variables in the more rhythmically rich and emotionally expressive task of singing voice conversion, while also facing issues with low efficiency in speech processing. In this paper, we propose a high-fidelity flow-based model based on multi-decoupling feature constraints called RASVC, which enhances the capture of vocal details by integrating multiple latent attribute encoders. We also use Multi-stream inverse short-time Fourier transform(MS-iSTFT) to enhance the speed of speech processing by skipping some complicated decoder processing steps. We compare the synthesized singing voice with other models from multiple dimensions, and our proposed model is highly consistent with the current state-of-the-art, with the demo which is available at \url{https://lazycat1119.github.io/RASVC-demo/}.

eess.AS↗

arXiv2024

A New Perspective on Speaker Verification: Joint Modeling with DFSMN and Transformer

Speaker verification is to judge the similarity between two unknown voices in an open set, where the ideal speaker embedding should be able to condense discriminant information into a compact utterance-level representation that has small intra-speaker distances and large inter-speaker distances. We propose Voice Transformer (VOT), a novel model for speaker verification, which integrates parallel transformers at multiple scales. A deep feedforward sequential memory network (DFSMN) is incorporated into the attention part of these transformers to increase feature granularity. The attentive statistics pooling layer is added to focus on important frames and form utterance-level features. We propose Additive Angular Margin Focal Loss (AAMF) to address the hard samples problem. We evaluate the proposed approach on the VoxCeleb1 and CN-Celeb2 datasets, demonstrating that VOT surpasses most mainstream models. The code is available on GitHub\footnote{\url{https://github.com/luckyerr/Voice-Transformer_Speaker-Verification}}.

eess.AS↗

arXiv2024

RIGEL: Simulating dwarf galaxies at solar mass resolution with radiative transfer and feedback from individual massive stars

We introduce the RIGEL model, a novel framework to self-consistently model the effects of stellar feedback in the multiphase ISM of dwarf galaxies with radiative transfer (RT) on a star-by-star basis. The RIGEL model integrates detailed implementations of feedback from individual massive stars into the RHD code, AREPO-RT. It forms individual massive stars from the resolved multiphase ISM by sampling the IMF and tracks their evolution individually. The lifetimes, photon production rates, mass-loss rates, and wind velocities of these stars are determined by their initial masses and metallicities based on a library that incorporates a variety of stellar models. The RT equations are solved in seven spectral bins accounting for the IR to HeII ionizing bands, using an M1 RT scheme. The thermochemistry model tracks the non-equilibrium H, He chemistry and the equilibrium abundance of CI, CII, OI, OII, and CO to capture the thermodynamics of all ISM phases. We evaluated the performance of the RIGEL model using $1\,{\rm M}_\odot$ resolution simulations of isolated dwarf galaxies. We found that the SFR and ISRF show strong positive correlations to the metallicity of the galaxy. Photoionization and photoheating can reduce the SFR by an order of magnitude by removing the available cold-dense gas fuel for star formation. The ISRF also changes the thermal structure of the ISM. Radiative feedback occurs immediately after the birth of massive stars and rapidly disperses the molecular clouds within 1 Myr. As a consequence, radiative feedback reduces the age spread of star clusters to less than 2 Myr, prohibits the formation of massive star clusters, and shapes the cluster initial mass function to a steep power-law form with a slope of $\sim-2$. The mass-loading factor of the fiducial galaxy has a median of $\sim50$, while turning off radiative feedback reduces this factor by an order of magnitude.

astro-ph.GA↗

arXiv2024

Skin Effect of Nonlinear Optical Responses in Antiferromagnets

Nonlinear optics plays important roles in the research of fundamental physics and the applications of high-performance optoelectronic devices. The bulk nonlinear optical responses arise from the uniform light absorption in noncentrosymmetric crystals, and hence are usually considered to be the collective phenomena of all atoms. Here we show, in contrast to this common expectation, the nonlinear optical responses in antiferromagnets can be selectively accumulated near the surfaces, representing a skin effect. This is because the inversion symmetry, despite being broken globally by magnetism, is barely violated locally deeply inside these antiferromagnets. Using A-type layered antiferromagnets as the representatives, we predict that the spatial-dependent nonlinear optical responses, such as bulk photovoltaic effect (BPVE) and second harmonic generation (SHG), are notable in the top- and bottom-most layers and decay rapidly when moving away from the surfaces. Such a phenomenon is strongly associated with the antiferromagnetism and exists in a broad range of antiferromagnets composed of centrosymmetric sublattices, offering promising device applications using these antiferromagnets. Our work uncovers a previously overlooked property of nonlinear optical responses and opens new opportunities for high-performance antiferromagnetic optospintronics.

cond-mat.mtrl-sci↗

arXiv2024

NeuralOOD: Improving Out-of-Distribution Generalization Performance with Brain-machine Fusion Learning Framework

Deep Neural Networks (DNNs) have demonstrated exceptional recognition capabilities in traditional computer vision (CV) tasks. However, existing CV models often suffer a significant decrease in accuracy when confronted with out-of-distribution (OOD) data. In contrast to these DNN models, human can maintain a consistently low error rate when facing OOD scenes, partly attributed to the rich prior cognitive knowledge stored in the human brain. Previous OOD generalization researches only focus on the single modal, overlooking the advantages of multimodal learning method. In this paper, we utilize the multimodal learning method to improve the OOD generalization and propose a novel Brain-machine Fusion Learning (BMFL) framework. We adopt the cross-attention mechanism to fuse the visual knowledge from CV model and prior cognitive knowledge from the human brain. Specially, we employ a pre-trained visual neural encoding model to predict the functional Magnetic Resonance Imaging (fMRI) from visual features which eliminates the need for the fMRI data collection and pre-processing, effectively reduces the workload associated with conventional BMFL methods. Furthermore, we construct a brain transformer to facilitate the extraction of knowledge inside the fMRI data. Moreover, we introduce the Pearson correlation coefficient maximization regularization method into the training process, which improves the fusion capability with better constrains. Our model outperforms the DINOv2 and baseline models on the ImageNet-1k validation dataset as well as six curated OOD datasets, showcasing its superior performance in diverse scenarios.

cs.CV↗

arXiv2024

The FAST Core Array

The Five-hundred-meter Aperture Spherical Radio Telescope (FAST) Core Array is a proposed extension of FAST, integrating 24 secondary 40-m antennas implanted within 5 km of the FAST site. This original array design will combine the unprecedented sensitivity of FAST with a high angular resolution (4.3" at a frequency of 1.4 GHz), thereby exceeding the capabilities at similar frequencies of next-generation arrays such as the Square Kilometre Array Phase 1 or the next-generation Very Large Array. This article presents the technical specifications of the FAST Core Array, evaluates its potential relatively to existing radio telescope arrays, and describes its expected scientific prospects. The proposed array will be equipped with technologically advanced backend devices, such as real-time signal processing systems. A phased array feed receiver will be mounted on FAST to improve the survey efficiency of the FAST Core Array, whose broad frequency coverage and large field of view (FOV) will be essential to study transient cosmic phenomena such as fast radio bursts and gravitational wave events, to conduct surveys and resolve structures in neutral hydrogen galaxies, to monitor or detect pulsars, and to investigate exoplanetary systems. Finally, the FAST Core Array can strengthen China's major role in the global radio astronomy community, owing to a wide range of potential scientific applications from cosmology to exoplanet science.

astro-ph.IM↗

arXiv2024

QUSL: Quantum Unsupervised Image Similarity Learning with Enhanced Performance

Leveraging quantum properties to enhance complex learning tasks has been proven feasible, with excellent recent achievements in the field of unsupervised learning. However, current quantum schemes neglect adaptive adjustments for unsupervised task scenarios. This work proposes a novel quantum unsupervised similarity learning method, QUSL. Firstly, QUSL uses similarity triplets for unsupervised learning, generating positive samples by perturbing anchor images, achieving a learning process independent of classical algorithms. Subsequently, combining the feature interweaving of triplets, QUSL employs metaheuristic algorithms to systematically explore high-performance mapping processes, obtaining quantum circuit architectures more suitable for unsupervised image similarity tasks. Ultimately, QUSL realizes feature learning with lower quantum resource costs. Comprehensive numerical simulations and experiments on quantum computers demonstrate that QUSL outperforms state-of-the-art quantum methods. QUSL achieves over 50% reduction in critical quantum resource utilization. QUSL improves similarity detection correlation by up to 19.5% across multiple datasets, exhibiting robustness in NISQ environments. While using fewer quantum resources, QUSL shows potential for large-scale unsupervised tasks.

quant-ph↗

arXiv2024

Primitive Ideal Space of $C^*(R_+)\rtimes R^\times$

For an integral domain $R$ satisfying certain condition, we characterize the primitive ideal space and its Jacobson topology of the semigroup crossed product $C^*(R_+) \rtimes R^\times$. The main example is when $R=\mathbb{Z}[\sqrt{-3}]$.

math.OA↗