arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2511.12045

MUSTEM: A Dual-Modality System for Vibrotactile and Visual Translation of Music as an Assistive Technology

Paloma Sette·Maria Werneck·William Barbosa·Ana Loubacker
View original source ↗Read source document ↗Export metadata ↓

Abstract

The emotional and structural experience of music remains a significant accessibility challenge for the deaf and hard of hearing community. This paper introduces MUSTEM (Multisensorial Emotional Translation), a novel system designed to translate music into a rich, coherent, and scientifically-grounded sensory experience. We present a dual-modality approach addressing this challenge through two interconnected components. First, a low-cost, portable hardware prototype that performs real-time audio analysis, mapping distinct frequency bands (sub-bass, bass, mid-range, treble) to a four-channel vibrotactile system, allowing users to feel the music's rhythmic and foundational structure. Second, to overcome the processing limitations of embedded hardware, we developed a high-fidelity software simulation that demonstrates the full potential of the visual translation. This assistive dashboard decodes musical components - such as rhythm, harmony, and frequency spectrum - into an intuitive and educational visual interface. MUSTEM offers a comprehensive framework for sensory substitution, presenting a viable and accessible pathway for the deaf community to experience music not just as vibration, but as a structured, substantiated and emotionally resonant visual and tactile language. Preliminary feedback from seven deaf users suggests the system's spatial vibrotactile mapping is perceptible and engaging. All source code and hardware designs are released as open-source. Video demonstrations and open-source code are available on the project's official channel.

Explore related subjects

eess.SP

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Paloma Sette, Maria Werneck, William Barbosa, Ana Loubacker. 2025-11-15. MUSTEM: A Dual-Modality System for Vibrotactile and Visual Translation of Music as an Assistive Technology. https://arxiv.org/abs/2511.12045

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Multi-Carrier Rydberg Atomic Quantum Receivers with Enhanced Bandwidth Feature for Communication and Sensing

Rydberg atomic quantum receivers (RAQRs) have attracted significant attention in recent years due to their ultra-high sensitivity. Although capable of precisely detecting the amplitude and phase of weak signals, conventional RAQRs face inherent limitations in accurately receiving wideband RF signals, due to the discrete nature of atomic energy levels and their intrinsic instantaneous bandwidth constraints. These limitations hinder their direct application to multi-carrier communication and sensing. To address this issue, this paper proposes a multi-carrier Rydberg atomic quantum receiver (MC-RAQR) structure with five energy levels. We derive the amplitude and phase of the MC-RAQR and extract the baseband electrical signal for signal processing. In terms of multi-carrier communication and sensing, we analyze the channel capacity and accuracy of angle of arrival (AoA) and distance parameters, respectively. Numerical results validate our proposed model, showing that the MC-RAQR can achieve up to a bandwidth of 11.7 MHz, which is 17-fold larger than the conventional RAQRs. As a result, the channel capacity and the resolution for multi-target sensing are improved significantly. Specifically, the channel capacity of MC-RAQR is 110-fold and 2.8-fold larger than the classical RF receivers and RAQRs, respectively. For sensing performance, the RMSE of AoA estimation for MC-RAQR exhibits 7.6-fold reduction, compared with the conventional RAQRs. Furthermore, the RMSE of distance estimation is $634$-fold smaller than that of the root-CRB of classical RF receivers, showing the superior performance of the MC-RAQR. This demonstrates its compatibility with waveforms such as orthogonal frequency-division multiplexing (OFDM) and its significant advantages for multi-carrier signal reception.

eess.SP↗
arXiv2026

Channel Estimation in MIMO Systems Aided by Microwave Linear Analog Computers (MiLACs)

Microwave linear analog computers (MiLACs) have recently emerged as a promising solution for future gigantic multiple-input multiple-output (MIMO) systems, enabling beamforming with greatly reduced hardware and computational cost. However, channel estimation for MiLAC-aided systems remains an open problem. Conventional least squares (LS) and minimum mean square error (MMSE) estimation rely on intensive digital computation, which undermines the computational advantage offered by MiLACs. In this letter, we propose efficient LS and MMSE channel estimation schemes for MiLAC-aided MIMO systems. By designing the training precoder and combiner implemented by lossless and reciprocal MiLACs, the proposed schemes perform LS and MMSE estimation in the analog domain, leaving only simple digital scaling. They achieve identical estimation performance to their digital counterparts while significantly reducing computational complexity. Numerical results verify the effectiveness of the proposed schemes.

eess.SP↗
arXiv2026

Joint Subcarrier Phase Recovery for Nonlinearity Mitigation

We propose a low-complexity phase recovery scheme that simultaneously mitigates laser phase noise and fiber nonlinearity across several subcarriers. In a long single-span link with Raman amplification, the scheme achieves 0.9 dB gain with 99 real multiplications per complex symbol.

eess.SP↗