arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.25394

MixiSpeech: Progressive State Distillation for One-Step Vietnamese TTS

Tran Quoc Bao Le·Binh Mai·Hung Dinh·Lam Nguyen·Cong Tran
View original source ↗Read source document ↗Export metadata ↓

Abstract

Conditional-flow text-to-speech (TTS) models can achieve high synthesis quality with relatively few sampling steps, but collapsing the remaining trajectory to a single update can still cause substantial quality degradation. We propose MixiSpeech, a progressive state distillation framework that converts a multi-step conditional-flow acoustic model into a one-step generator while preserving its architecture and speaker conditioning. Rather than training the one-step model directly, MixiSpeech progressively reduces the global sampling budget through a 10-4-2-1 curriculum, with each student trained to match local teacher-integration endpoints using a masked state-matching objective. We instantiate the framework on a 710-speaker Vietnamese Matcha-TTS model trained from scratch on PhoAudiobook. On 100 shared test text-speaker pairs, direct one-step inference substantially degrades synthesis quality, obtaining 1.6846 UTMOS22-Strong and 6.21% WER compared with 2.2923 and 4.65% for the 10-step teacher. In contrast, the distilled one-step model achieves 2.2678 UTMOS22-Strong, 4.63% WER, and 2.83% CER, recovering most of the quality lost at the one-step operating point while requiring only a single acoustic vector-field evaluation. End-to-end inference time decreases from 0.0662 s for the 10-step teacher to 0.0312 s for MixiSpeech, corresponding to a 2.12 measured speedup. These results show that progressive state distillation can make one-step conditional-flow TTS practical without relying on a large multi-step sampling budget. Project page: https://mixispeech.solanai.us/

Explore related subjects

cs.SDcs.MMeess.AS

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Tran Quoc Bao Le, Binh Mai, Hung Dinh, Lam Nguyen, Cong Tran. 2026-10-18. MixiSpeech: Progressive State Distillation for One-Step Vietnamese TTS. https://arxiv.org/abs/2610.25394

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Manifold-Constrained Residual Adaptation with Frame-Adaptive Layer Aggregation for Speaker Verification

Large self-supervised speech models provide powerful representations for speaker verification, but full fine-tuning is expensive. Most conventional adapters insert small trainable modules to modify the output of each layer, while keeping the original residual connections unchanged. Building on our preliminary study of manifold-constrained Hyper-Connections in speaker encoders, we explore parameter-efficient adaptation of both residual paths and feature selection across layers. We first propose a Manifold-Constrained Residual Adapter (MCRA), which splits the hidden dimensions of each Transformer residual state into N groups and uses a small Sinkhorn-normalized matrix to exchange information among them. A learnable gate controls the balance between the original and mixed residual paths. The gate adds only one scalar parameter at each residual connection and initially favors the original path, allowing gradual adaptation. We further propose Frame-Adaptive Layer Aggregation (FALA), an attention-based aggregation method that uses a single learned query shared across all utterances to compute layer weights separately for each speech frame. Because the layer representations depend on the input, the resulting weights vary across both utterances and frames, unlike the fixed weights of a conventional layer-weighted sum. Compared with a frozen WavLM baseline using a fixed layer-weighted sum, our method achieves relative EER reductions of 22.4% on VoxCeleb1-O and 2.9% on CN-Celeb, with trainable adaptation parameters equivalent to only 0.0009% and 0.0016% of the WavLM-Large parameters, respectively.

cs.SD↗
arXiv2026

Structural Context from Audio and Lyric Evidence for Music Structure Analysis Using the New SheetSage-MSA Dataset

Music structure analysis (MSA) aims to identify functional labels and their boundaries in music. Despite recent progress, three challenges remain. First, fine-grained annotations for MSA are scarce. Second, existing systems often share one encoder between boundary detection and functional labeling despite their different information requirements. Third, most end-to-end systems are audio-only and ignore structural cues from lyrics. To address these issues, we construct SheetSage-MSA, an MSA dataset derived from SheetSage, containing 13,880 cleaned excerpt annotations from 9,010 songs and approximately 82.5 hours of annotated audio. We further propose SCALE (Structural Context from Audio and Lyric Evidence), which uses separate task-specific audio pathways to model local transitions and long-range structural context, and incorporates timestamped lyrics and lyrical recurrence to refine chorus predictions. On the HarmonixSet benchmark, SCALE outperforms the existing state-of-the-art SongFormer by 3.5 and 4.2 percentage points in HR.5F and HR3F, respectively, while improving ACC by 0.9 points. An interactive demo page containing the source code is also available at https://multimodal-music-research-lab.github.io/SCALE/.

cs.SD↗
arXiv2026

AudioScorer: Learning Question-Aware Token Relevance for Large Audio Language Models

Large Audio Language Models (LALMs) reason over speech, environmental sounds, music, and other acoustic information. However, their audio encoders produce dense token sequences that make long-form audio costly to process. Existing audio pruning methods reduce this cost using heuristics such as attention, representation similarity, or temporal sampling, but do not explicitly learn which tokens are most useful for a given question. We propose AudioScorer, a lightweight model that scores audio tokens according to the user question. It is trained by distilling the ranking of audio tokens under the frozen LALM's answer-gradient saliency, together with downstream answer supervision. Compared with other pruning methods, AudioScorer achieves the best or tied-best result at 10% token retention on 7 of 10 semantic and acoustic tasks, including speech recognition, emotion recognition, and audio-scene classification.

cs.SD↗