arXiv subjects

Luca Turchet

Publications and source records attributed to Luca Turchet.

12 recordsLinked to original sources

arXiv2026

CrowdForge: Effects of Audio and Visual Audience Participation Modalities on Agency, Collaboration, and Engagement in AI-Mediated MR Performances

Technology-mediated audience participation (TMAP) enables audiences to influence live performances, yet balancing interaction with meaningful agency remains challenging. While AI-generated music and MR enable new forms of audio and visual audience participation, little is known about how these participation modalities influence audience experiences during live musical performance. In particular, it remains unclear how they shape perceived agency, collaboration, and engagement, and how they complement one another within TMAP systems. We present CrowdForge, an AI- and MR-mediated paradigm where audiences collectively control generative music and virtual environments during a live performance featuring a guitarist and DJ. Through a mixedmethods concert study evaluating three interaction conditions (audio, visual, and combined audio-visual), we demonstrate that observability is the primary driver of audience agency. While visual interaction provided easily observable feedback, audio parameters offered macro-structural control over the music. Audiences tended to use the control enabled by AI audio generation to test musical boundaries, introducing a productive adversarial tension between the crowd and the musicians. This dynamic pushed creative limits, deepened engagement, and generated moments of musical coherence when performers successfully resolved audience-induced challenges. We argue future TMAP systems should prioritize human-centered, highly observable interactions to build agency, while combining the expressiveness of music with the clarity of visual feedback.

cs.HC↗

arXiv2026

When Music Needs Meaning: Semantic Communication as a Transformative Enabler for the Musical Metaverse

The Musical Metaverse (MM) enables geographically distributed musicians, audiences, sound engineers, and AI agents to interact through synchronized audio, visual, kinematic, and haptic streams, imposing latency, synchronization, bandwidth, and scalability requirements that are difficult to satisfy with conventional bit-level communication. This paper argues that semantic communication, understood here as the transmission of musically meaningful information rather than raw audio-visual-haptic samples, provides a principled communication paradigm for the MM. We define the MM from a communication perspective and derive modality-specific require ments; propose a three-pillar framework for musical semantic communication comprising semantics extraction, semantic-aware orchestration, and cross-modal semantic synchronization; survey representative work across semantic communication, music AI and edge orchestration, identifying the gaps that prevent their integration; and propose a semantic-first reference architecture based on device-side semantic encoders, edge semantic routers, and cross-modal semantic synchronization. The resulting framework positions the MM as a concrete and demanding use case for 6G semantic communication.

eess.SY↗

arXiv2026

WebXR and Commercial Game Engines for the Metaverse: A Socio-Technical Analysis of Openness, Interoperability, and Sustainability

The Metaverse is often framed as a persistent, interoperable, and embodied network of virtual and augmented environments. Yet, most contemporary XR applications are developed through commercial game engines and distributed through proprietary app stores, creating tensions between openness and platform dependency. This paper critically examines open WebXR technologies with conventional commercial game-engine pipelines, with particular attention to XR hardware, software architectures, developer workflows, governance, ethics, interoperability, and sustainability. We argue that WebXR may provide a viable route toward a more accessible, device-independent, and institutionally sustainable Metaverse, especially for education, research, cultural heritage, prototyping, and public-interest applications. At the same time, commercial engines remain advantageous for graphically intensive, low-latency, deeply integrated, and large-scale XR products. The paper concludes that the choice should not be framed as WebXR versus engines, but as a continuum: WebXR is preferable when accessibility, interoperability, low-friction deployment, and long-term maintainability are primary goals, whereas native engines remain preferable when performance, platform-specific hardware access, and production-grade tooling dominate.

cs.CY↗

arXiv2026

Security and Privacy in the Musical Metaverse: Threat Analysis and Design Implications

The Musical Metaverse (MM) introduces immersive, real-time environments for collaborative musical interaction, characterized by ultra-low-latency constraints, continuous multimodal data streams, and heterogeneous devices. These properties create a distinctive security and privacy landscape that differs significantly from conventional XR or multimedia systems. This paper presents a multi-layer threat analysis of MM ecosystems, identifying key assets including live musical content, expressive interaction data, identity and session metadata, and intellectual property. Threats are analyzed across network, application, data/AI, device, intellectual property rights, and social layers, with particular attention to risks arising from expressive and neurophysiological data, which enable inference, re-identification, and potential privacy violations. We describe a stakeholder-driven survey involving 14 participants from 13 organizations, revealing that neurophysiological data leakage and real-time stream disruption are perceived as the most critical risks, followed by intellectual property infringement and avatar impersonation. We further evaluate the suitability of existing security protocols under strict latency constraints, showing that conventional approaches such as TLS over TCP are often incompatible with real-time musical interaction, while lightweight, stream-oriented mechanisms (e.g., SRTP, DTLS) provide a more suitable balance between security and performance. Based on these findings, we derive a set of design guidelines for MM systems, emphasizing latency-aware security, differentiation of interaction paths, data minimization, and edge-centric processing. The results support a security-by-design approach that enables trust and compliance without compromising real-time performance.

cs.CR↗

arXiv2026

Imagining the Musical Metaverse: a study on the needs and creative practices of composers and musicians

This study explores user needs for the Musical Metaverse (MM) through a series of workshops with electroacoustic composers, classical musicians, and music producers. Using a design approach based on the prompt "as if by magic," participants were invited to reflect on how the MM could impact their practice in composition, performance, and education domains. While groups maintain distinct priorities based on their roles, they share interests in educational applications and creative learning environments. Key themes include preference for mixed reality over purely virtual environments, virtual space as a creative paradigm, and tensions between democratizing tools and maintaining authentic musical experiences. Education emerged as the most promising initial use case, particularly for understanding complex musical concepts. However, participants expressed skepticism about fully virtual performances, emphasizing physical connections to instruments and spaces. Results suggest MM development should enhance rather than replace traditional practices while supporting embodied creativity.

cs.HC↗

arXiv2026

Application-Integrated Slicing towards 6G: The Musical Metaverse Use Case

Emerging immersive applications are expected to support heterogeneous groups of users with fundamentally different communication and computation requirements. Using the Musical Metaverse (MM) as a representative example, we show how such applications expose limitations of current 5G network slicing and orchestration frameworks, which remain largely service-centric and operate through decoupled application and network management mechanisms. While existing Quality of Service (QoS) and slicing techniques provide traffic differentiation, they cannot jointly account for application semantics, shared resources, and differentiated end-to-end requirements across multiple user classes within the same service instance. To address these limitations, we introduce application-integrated slicing, a framework that unifies application and network orchestration within a common end-to-end service model and enables differentiated KPI targets for multiple user classes within a single logical slice. Using a MM reference scenario deployed over cloud-edge network, we show that application-integrated slicing reduces both resource provisioning cost and QoS violation rates compared with conventional decoupled approaches.

cs.NI↗

arXiv2026

MR-DAW: Towards Collaborative Digital Audio Workstations in Mixed Reality

Digital Audio Workstations (DAWs) are central to modern music production but often encumber the musician's workflow, tethering them to a desk and hindering natural interaction with their instrument. Furthermore, effective remote collaboration remains a significant challenge, with existing solutions hampered by network latency and asynchronous file sharing. This paper investigates the potential of Mixed Reality (MR) to overcome these barriers, creating an intuitive environment for real-time, remote musical collaboration. We employ qualitative and speculative design techniques to better understand: 1) how players currently use DAWs, and 2) to imagine a speculative future of collaborative MR-DAWs. To facilitate this discussion, we developed and evaluated the usability of a design probe, MR-DAW. An MR system enabling multiple, geographically dispersed users to control a single, shared DAW instance while moving freely in their local spaces. Our networked system enables each remote musician to use a physical foot pedal for collaborative looping, merging a familiar, hands-free interaction with a shared virtual session. Based on interviews and system evaluations with 20 musicians, we analyze current practices, report on the user experience with our MR system, and speculate on the future of musical collaboration in MR. Our results highlight the affordances of MR for unencumbered musical interaction and provide a speculative outlook on the future of remote collaborative DAWs in the Musical Metaverse.

cs.HC↗

arXiv2025

MuMeNet: A Network Simulator for Musical Metaverse Communications

The Metaverse, a shared and spatially organized digital continuum, is transforming various industries, with music emerging as a leading use case. Live concerts, collaborative composition, and interactive experiences are driving the Musical Metaverse (MM), but the requirements of the underlying network and service infrastructures hinder its growth. These challenges underscore the need for a novel modeling and simulation paradigm tailored to the unique characteristics of MM sessions, along with specialized service provisioning strategies capable of capturing their interactive, heterogeneous, and multicast-oriented nature. To this end, we make a first attempt to formally model and analyze the problem of service provisioning for MM sessions in 5G/6G networks. We first formalize service and network graph models for the MM, using "live audience interaction in a virtual concert" as a reference scenario. We then present MuMeNet, a novel discrete-event network simulator specifically tailored to the requirements and the traffic dynamics of the MM. We showcase the effectiveness of MuMeNet by running a linear programming based orchestration policy on the reference scenario and providing performance analysis under realistic MM workloads.

cs.NI↗

arXiv2025

BrainForm: a Serious Game for BCI Training and Data Collection

$\textit{BrainForm}$ is a gamified Brain-Computer Interface (BCI) training system designed for scalable data collection using consumer hardware and a minimal setup. We investigated (1) how users develop BCI control skills across repeated sessions and (2) perceptual and performance effects of two visual stimulation textures. Game Experience Questionnaire (GEQ) scores for Flow, Positive Affect, Competence and Challenge were strongly positive, indicating sustained engagement. A within-subject study with multiple runs, two task complexities, and post-session questionnaires revealed no significant performance differences between textures but increased ocular irritation over time. Online metrics$\unicode{x2013}$Task Accuracy, Task Time, and Information Transfer Rate$\unicode{x2013}$improved across sessions, confirming learning effects for symbol spelling, even under pressure conditions. Our results highlight the potential of $\textit{BrainForm}$ as a scalable, user-friendly BCI research tool and offer guidance for sustained engagement and reduced training fatigue.

cs.HC↗

arXiv2025

How AI Generates Creativity from Inauthenticity

Artificial creativity is presented as a counter to Benjamin's conception of an "aura" in art. Where Benjamin sees authenticity as art's critical element, generative artificial intelligence operates as pure inauthenticity. Two elements of purely inauthentic art are described: elusiveness and reflection. Elusiveness is the inability to find an origin-story for the created artwork, and reflection is the ability for perceivers to impose any origin that serves their own purposes. The paper subsequently argues that these elements widen the scope of artistic and creative potential. To illustrate, an example is developed around musical improvisation with an artificial intelligence partner. Finally, a question is raised about whether the inauthentic creativity of AI in art can be extended to human experience and our sense of our identities.

cs.CY↗

arXiv2020

Quantum GestART: Identifying and Applying Correlations between Mathematics, Art, and Perceptual Organization

Mathematics can help analyze the arts and inspire new artwork. Mathematics can also help make transformations from one artistic medium to another, considering exceptions and choices, as well as artists' individual and unique contributions. We propose a method based on diagrammatic thinking and quantum formalism. We exploit decompositions of complex forms into a set of simple shapes, discretization of complex images, and Dirac notation, imagining a world of "prototypes" that can be connected to obtain a fine or coarse-graining approximation of a given visual image. Visual prototypes are exchanged with auditory ones, and the information (position, size) characterizing visual prototypes is connected with the information (onset, duration, loudness, pitch range) characterizing auditory prototypes. The topic is contextualized within a philosophical debate (discreteness and comparison of apparently unrelated objects), it develops through mathematical formalism, and it leads to programming, to spark interdisciplinary thinking and ignite creativity within STEAM.

math.HO↗

arXiv2019

Shall We (Math and) Dance?

Can we use mathematics, and in particular the abstract branch of category theory, to describe some basics of dance, and to highlight structural similarities between music and dance? We first summarize recent studies between mathematics and dance, and between music and categories. Then, we extend this formalism and diagrammatic thinking style to dance.

math.GM↗