arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2110.06280

S3PRL-VC: Open-source Voice Conversion Framework with Self-supervised Speech Representations

Wen-Chin Huang·Shu-Wen Yang·Tomoki Hayashi·Hung-Yi Lee·Shinji Watanabe·Tomoki Toda
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper introduces S3PRL-VC, an open-source voice conversion (VC) framework based on the S3PRL toolkit. In the context of recognition-synthesis VC, self-supervised speech representation (S3R) is valuable in its potential to replace the expensive supervised representation adopted by state-of-the-art VC systems. Moreover, we claim that VC is a good probing task for S3R analysis. In this work, we provide a series of in-depth analyses by benchmarking on the two tasks in VCC2020, namely intra-/cross-lingual any-to-one (A2O) VC, as well as an any-to-any (A2A) setting. We also provide comparisons between not only different S3Rs but also top systems in VCC2020 with supervised representations. Systematic objective and subjective evaluation were conducted, and we show that S3R is comparable with VCC2020 top systems in the A2O setting in terms of similarity, and achieves state-of-the-art in S3R-based A2A VC. We believe the extensive analysis, as well as the toolkit itself, contribute to not only the S3R community but also the VC community. The codebase is now open-sourced.

Explore related subjects

cs.SDcs.CLcs.LGeess.AS

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Wen-Chin Huang, Shu-Wen Yang, Tomoki Hayashi, Hung-Yi Lee, Shinji Watanabe, Tomoki Toda. 2021-10-12. S3PRL-VC: Open-source Voice Conversion Framework with Self-supervised Speech Representations. https://arxiv.org/abs/2110.06280

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

ParsVoice: A Large-Scale Multi-Speaker Persian Speech Corpus for Text-to-Speech Synthesis

Persian remains substantially underrepresented in open speech-text resources, limiting progress in multi-speaker text-to-speech (TTS), speech-language modelling, and low-resource speech processing. We introduce ParsVoice, the largest publicly available Persian speech-text corpus tailored for training multi-speaker TTS systems, along with a scalable pipeline to construct high-quality speech-text data from long-form audiobook recordings. The pipeline combines a fine-tuned ParsBERT sentence-completion classifier, ASR-based boundary optimization, punctuation restoration, speaker identification, and a multi-dimensional quality assessment that covers both audio and Persian-specific text properties. The resulting release contains a 2,200-hour TTS-ready subset with 1.36 million aligned segments from 1,815 automatically inferred speaker IDs, making it more than 25 times larger than the largest previously available open Persian TTS dataset. To validate the corpus, we fine-tune XTTSv2, a zero-shot multilingual TTS model that operates directly on raw Persian text without phoneme representations. The resulting model achieves a naturalness MOS of 3.6/5 and a speaker-similarity MOS of 4.0/5. ParsVoice, its metadata, and the corpus-construction pipeline are publicly available on Hugging Face: https://huggingface.co/datasets/MohammadJRanjbar/ParsVoice and GitHub: https://github.com/MohammadJRanjbar/ParsVoice, supporting reproducible research on Persian speech synthesis and low-resource speech-language technologies.

cs.SD↗
arXiv2026

Speech Generation Speaker Poisoning: Capability Erasure in Zero-Shot Text-to-Speech

Recent zero-shot Text-to-Speech (TTS) systems can clone previously unseen voices from only a few seconds of audio. We formulate Speech Generation Speaker Poisoning (SGSP), a task that seeks to prevent a model from synthesizing targeted speaker identities while maintaining performance on all other speakers. Unlike conventional machine unlearning, removing training examples is insufficient because modern zero-shot TTS systems can reconstruct identities through learned speaker representations and strong generalization capabilities. We evaluate both inference-time filtering and parameter-modification approaches across settings involving 1, 15, and 100 forget speakers, focusing primarily on speakers seen during training, which we show are harder to suppress than unseen speakers. To characterize the trade-off between utility and privacy, we introduce an evaluation framework based on AUC analysis and a proposed metric, Forget Set Similarity (FSSIM). Our results demonstrate effective speaker suppression for up to 15 forget speakers while revealing that worst-case identity leakage (Max-FSSIM) remains unresolved at multi-speaker scale - establishing this as an open challenge for future work. Together, our work establishes targeted speaker poisoning as a task for zero-shot TTS and, using StyleTTS2 as an initial testbed, provides methods and evaluation protocols for future research, with code and model weights.

cs.SD↗
arXiv2026

Tracing the Origins: Legacy Codec Identification in Neural Audio Transcoding

Residual Vector Quantization (RVQ)-based neural audio codecs (NACs) enable high-fidelity audio distribution at unprecedentedly low bitrates through discrete token-based representations. However, this shift disrupts traditional forensics, as non-linear neural transcoding obscures the underlying traces of legacy compression. This study defines the forensic gap and proposes a Transformer-based framework designed to leverage the hierarchical and temporal dependencies inherent in RVQ sequences. By modeling inter-layer causal relationships and dynamic forensic significance, our model effectively disentangles superimposed artifacts from legacy-to-neural transcoding. Experimental results achieve 97%+ accuracy for codec identification and robust joint identification performance across 32-128 kbps. These results demonstrate that traditional codec traces persist even after neural transcoding, supporting the feasibility and necessity of neural-codec-aware audio forensics.

cs.SD↗