Neural Music Enhancement with Dual Time-Frequency Spectral Representations for Prediction and Discrimination

Fei Liu; Yang Ai; Zhen-Hua Ling

Home/Archive/arXiv

arXiv · 2609.03357

Neural Music Enhancement with Dual Time-Frequency Spectral Representations for Prediction and Discrimination

Fei Liu·Yang Ai·Zhen-Hua Ling

View original source ↗Read source document ↗Export metadata ↓

Abstract

Non-professional music recordings shared online often suffer from background noise and reverberation, degrading perceived quality and limiting reuse. This paper proposes DSME, a music enhancement model based on dual time-frequency spectral representations. Within a generative adversarial framework, DSME uses short-time Fourier transform (STFT) spectra for generation and constant-Q transform (CQT) spectra for discrimination. Leveraging STFT's fixed window, invertibility, and predictability, the generator estimates clean amplitude-phase spectra from degraded inputs and reconstructs waveforms via inverse STFT. Exploiting CQT's log-frequency, variable-window structure aligned with musical octaves, we design an octave-segmented CQT discriminator. We also introduce a chroma-spectrum loss to emphasize pitch and harmonic consistency. Experiments show DSME outperforms baselines in objective and subjective tests, validating the effectiveness of the dual-spectrum approach.

Explore related subjects

cs.SD

Keep this discovery

BibTeX RIS

Fei Liu, Yang Ai, Zhen-Hua Ling. 2026-09-03. Neural Music Enhancement with Dual Time-Frequency Spectral Representations for Prediction and Discrimination. https://arxiv.org/abs/2609.03357

Cite the original work for its findings. Save a collection to share your selection of sources.

Discover connections

cs.SD subject

Connections use source metadata and explicit phrase matches, not verified experimental comparisons.

KEEP EXPLORING

Related papers

arXiv2026

Evoking Harmony via Convolution

I show how to evoke the pitch-class content of a chord from an arbitrary source sound by convolving the source with an impulse response whose grains are one windowed sinusoid per pitch-class, across each octave of hearing range; while, at the same time, minimizing artifacts. A Csound user-defined opcode, chord_convolver, mixes a dry Dirac component into that response, and applies partitioned convolution once. I contrast the effect with a linear-frequency comb filter and with a generic constant-Q resonator bank, and I demonstrate musical use on a twilight field recording alongside the ruins of Chateau de Lagarde.

cs.SD↗

arXiv2026

Test-time adaptation for speech enhancement with an autoregressive speech prior

Test-time adaptation (TTA) offers a promising direction for improving speech enhancement models under mismatched acoustic conditions, without requiring access to labeled target data. In this work, we propose a single-utterance TTA method that regularizes a pretrained speech enhancement model using an autoregressive prior trained on clean speech latent representations extracted from a neural audio codec. Adaptation is performed by minimizing the Kullback-Leibler divergence between the enhanced speech distribution and the clean speech prior. Experiments across multiple noisy speech datasets show consistent improvements in speech quality, particularly under training-testing noise mismatch conditions. Code and audio examples are available online.

cs.SD↗

arXiv2026

BFA: Real-time Multilingual Text-to-speech Forced Alignment

We present Bournemouth Forced Aligner (BFA), a system that combines a Contextless Universal Phoneme Encoder (CUPE) with a connectionist temporal classification (CTC)based decoder. BFA introduces explicit modelling of inter-phoneme gaps and silences and hierarchical decoding strategies, enabling fine-grained boundary prediction. Evaluations on TIMIT and Buckeye corpora show that BFA achieves competitive recall relative to Montreal Forced Aligner at relaxed tolerance levels, while predicting both onset and offset boundaries for richer temporal structure. BFA processes speech up to 240x faster than MFA, enabling faster than real-time alignment. This combination of speed and silence-aware alignment opens opportunities for interactive speech applications previously constrained by slow aligners.

eess.AS↗