YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality

Ruibin Yuan; Jiahao Pan; Junyan Jiang; Zhiyue Wu; Ziya Zhou; Jiankai Sun; Yizhi Li; Ge Zhang; Yicheng Gu; Zeyue Tian; Junyu Dai; Hanfeng Lin; Kai Li; Shangda Wu; Xuanjie Liu; Jiaming Wang; Zihan Liu; Yue Wang; Yinghao Ma; Hanzhi Yin; Kangrui Chen; Xinyue Zhang; Ziyang Ma; Mengqi Liao; Hejia Zhao; Guowei Huang; Chao Yan; Lei Ke; Jianwei Yu; Bei Liu; Joe Guo; Liumeng Xue; Gus Xia; Wei Xue; Yike Guo

arXiv · 2609.33757

YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality

View original source ↗Read source document ↗Export metadata ↓

Abstract

Symbolic models make melody, harmony, rhythm, and form explicit but typically stop before a finished recording; audio models produce complete songs while leaving composition implicit. We introduce YuE2, which unifies symbolic and audio music generation at frontier quality through symbolic planning. A single AR-NAR Mixture-of-Transformers (MoT) first writes a readable score specifying melody and harmony, expands it into semantic music tokens, and realizes it as full-song audio. In comparisons using the same checkpoint, experts prefer symbolic planning for overall quality and musicality, with 49.3% of overall preferences versus 34.6% without planning. Experts also favor the unified model over a separate language model and diffusion Transformer. On WildSongBench, YuE2 scores 6.73 on SongBench Global Avg, exceeding all evaluated public baselines. Selecting from eight candidates (best-of-8), YuE2 reaches 6.96, the highest observed mean among all evaluated systems. Expert listening further establishes its competitiveness with proprietary song generators, favoring best-of-8 over Suno v4.5 and yielding nearly balanced preferences against Suno v5. To learn this generation process from recordings without aligned scores, we introduce MERT2 and SheetSage2 to supply semantic and symbolic supervision. MERT2 sets a new state of the art in music representation learning, surpassing previous best results on 14 of 15 MARBLE metrics; SheetSage2 leads 12 of 15 benchmark-metric pairs in our lead-sheet transcription comparison. The same checkpoint follows score edits while largely preserving unedited musical content and generates zero-shot covers without cover-specific training. Its readable score also enables agentic music editing, with external language models translating user feedback into revisions of the composition.

Explore related subjects

eess.AS cs.LG cs.SD

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Ruibin Yuan, Jiahao Pan, Junyan Jiang, Zhiyue Wu, Ziya Zhou, Jiankai Sun, Yizhi Li, Ge Zhang, Yicheng Gu, Zeyue Tian, Junyu Dai, Hanfeng Lin, Kai Li, Shangda Wu, Xuanjie Liu, Jiaming Wang, Zihan Liu, Yue Wang, Yinghao Ma, Hanzhi Yin, Kangrui Chen, Xinyue Zhang, Ziyang Ma, Mengqi Liao, Hejia Zhao, Guowei Huang, Chao Yan, Lei Ke, Jianwei Yu, Bei Liu, Joe Guo, Liumeng Xue, Gus Xia, Wei Xue, Yike Guo. 2026-09-27. YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality. https://arxiv.org/abs/2609.33757

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

TACO: Training-free Sound Prompted Segmentation via Semantically Constrained Audio-visual CO-factorization

Large-scale pre-trained audio and image models demonstrate an unprecedented degree of generalization, making them suitable for a wide range of applications. Here, we tackle the specific task of sound-prompted segmentation, aiming to segment image regions corresponding to objects heard in an audio signal. Most existing approaches tackle this problem by fine-tuning pre-trained models or by training additional modules specifically for the task. We adopt a different strategy: we introduce a training-free approach that leverages Non-negative Matrix Factorization (NMF) to co-factorize audio and visual features from pre-trained models so as to reveal shared interpretable concepts. These concepts are passed on to an open-vocabulary segmentation model for precise segmentation maps. By using frozen pre-trained models, our method achieves high generalization and establishes state-of-the-art performance in unsupervised sound-prompted segmentation, significantly surpassing previous unsupervised methods.

eess.AS↗

arXiv2026

ImmersiveFlow: Stereo-to-7.1.4 spatial audio generation with flow matching

Immersive spatial audio has become increasingly critical for applications ranging from AR/VR to home entertainment and automotive sound systems. However, existing generative methods remain constrained to low-dimensional formats such as binaural audio and First-Order Ambisonics (FOA). Binaural rendering is inherently limited to headphone playback, while FOA suffers from spatial aliasing and insufficient resolution for high-frequency. To overcome these limitations, we introduce ImmersiveFlow, the first end-to-end generative framework that directly synthesizes discrete 7.1.4 format spatial audio from stereo input. ImmersiveFlow leverages Flow Matching to learn trajectories from stereo inputs to multichannel spatial features within a pretrained VAE latent space. At inference, the Flow Matching model predicted latent features are decoded by the VAE and converted into the final 7.1.4 waveform. Comprehensive objective and subjective evaluations demonstrate that our method produces perceptually rich sound fields and enhanced spatial impression, significantly outperforming traditional upmixing techniques.

eess.AS↗

arXiv2026

Data Augmentation for L2 English Speaking Assessment using TTS

Automated assessment of second language (L2) speaking proficiency requires substantial annotated speech data, which are scarce compared to written learner corpora. We investigate whether written L2 responses can be transformed into useful synthetic speech for proficiency assessment using text-to-speech (TTS) and voice cloning. Using COREFL, a corpus of paired spoken and written responses from L2 learners of English, we systematically study two factors: how written responses should be transformed into spoken-style language ("speechification") and how synthetic voices and texts should be paired based on shared learner attributes (proficiency level, first language, both, or neither). We generate speechified responses with a large language model and synthesise them using TTS and voice cloning, then evaluate their utility for audio-based (HuBERT) and text-based (ModernBERT) proficiency grading. Results show that pairing voices and texts based on proficiency provides a principled strategy for synthetic data generation, while speechification substantially improves the match between written and spoken L2 and improves downstream grading performance. Augmenting real training data with synthetic speechified responses improves both HuBERT- and ModernBERT-based graders, demonstrating the potential of synthetic spoken data for L2 speaking proficiency assessment.

eess.AS↗