arXiv · 2611.23158
MoE--LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study
Abstract
Routed low-rank adapters offer a parameter-efficient means to improve the performance of large language models (LLMs) across tasks. Hence, it is assumed that they can also be used to improve the support of low-resource languages in LLMs, which currently serve English far better than the thousands of other languages spoken worldwide. However, this assumption has never been verified empirically under a shared multilingual evaluation protocol. We address exactly this gap by comparing eight low-rank adaptation (LoRA) methods on a frozen base model using 200 unique languages in more than 9{,}000 evaluation settings. Our results suggest that a reduced-scale search can identify viable configurations of LoRA methods but does not reliably predict their relative performance, as the top-ranking configuration is never best at scale. LoRA reliably improves language-model fit on translation data, consistently degrades reading comprehension, and benefits low-resource languages most. These findings suggest that LoRA methods should be regarded as targeted tools rather than general-purpose means for improving the multilingual capabilities of LLMs.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Nikit Srivastava, Joel Dag, René Speck, Axel-Cyrille Ngonga Ngomo. 2026-11-17. MoE--LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study. https://arxiv.science/abs/2611.23158
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.