arXiv Science⌕ Search

arXiv subjects

Tonio Weidler

Publications and source records attributed to Tonio Weidler.

2 recordsLinked to original sources

Direct Conditioning of Audio Diffusion Transformers on fMRI Reveals Cortical Contributions to Sound Reconstruction

Reconstructing audio from brain activity remains challenging. Existing brain-to-audio pipelines typically map neural responses to predefined stimulus representations before synthesis. While recent work in the visual domain has explored direct conditioning on brain activity, this approach has not yet been extended to audio diffusion models. We introduce MAAS (Multi-Area Auditory Synthesis), an end-to-end framework that conditions a pretrained audio diffusion transformer on ROI-wise fMRI responses without first supervising a decoder to predict an external stimulus representation. MAAS uses region-specific projections and learned brain tokens to condition Stable Audio through a dedicated cross-attention pathway. On a 7T fMRI dataset, we demonstrate consistent improvements over baselines, including two-stage fMRI-to-latent conditioning, across both acoustic and semantic evaluation metrics. In addition, we introduce a fidelity metric grounded in a computational model of auditory neural processing, which allows us to quantify the contribution of individual auditory cortical regions to reconstructed audio representations.

cs.SD↗

Biologically Inspired Semantic Lateral Connectivity for Convolutional Neural Networks

Lateral connections play an important role for sensory processing in visual cortex by supporting discriminable neuronal responses even to highly similar features. In the present work, we show that establishing a biologically inspired Mexican hat lateral connectivity profile along the filter domain can significantly improve the classification accuracy of a variety of lightweight convolutional neural networks without the addition of trainable network parameters. Moreover, we demonstrate that it is possible to analytically determine the stationary distribution of modulated filter activations and thereby avoid using recurrence for modeling temporal dynamics. We furthermore reveal that the Mexican hat connectivity profile has the effect of ordering filters in a sequence resembling the topographic organization of feature selectivity in early visual cortex. In an ordered filter sequence, this profile then sharpens the filters' tuning curves.

cs.CV↗