arXiv Science⌕ Search

arXiv · 2611.15250

Multi-head Quantum Neural Networks: Replacing Transformer Blocks with NISQ Modules inside a 30-Billion-Parameter LLM

Abstract

We introduce the multi-head quantum neural network (QNN), a quantum module compatible with noisy intermediate-scale quantum (NISQ) devices that can replace transformer blocks inside pre-trained large language models (LLMs) with little loss in accuracy and with far fewer parameters than the blocks it replaces. Encoding a model's $D$ hidden neurons into a qubit register is the obstacle: amplitude encoding scales poorly in circuit depth and angle encoding in qubit count, so neither is practical at production-scale width. We instead partition the hidden space into $k$ disjoint heads, angle-encode each into an $n_q$-qubit register, process the heads in parallel, and couple them through constrained classical linear layers, the mixers. Performing surgery on Qwen3-30B-A3B-Instruct-2507, we show that a single multi-head QNN with $145\,281$ trainable parameters ($D=2048$, $k=32$, $n_q=8$, $L=4$) can replace a contiguous span of eight transformer blocks, displacing $16.33\%$ of the model's parameters. Ablations show that, for a single replaced block, the quantum heads (circuits followed by a learned readout), and not the classical mixers alone, recover the displaced function, whereas for fused spans the mixers alone come within $1$-$3\%$ of the full module in perplexity. Whether this benefit is specific to trained, entangling quantum circuits is not established: controls that switch the entangler off (two seeds), or freeze the circuit at its initial or at uniformly random angles (one seed), beat the full module on held-out perplexity, and a parameter-matched classical multilayer perceptron remains a few percent ahead of the full module. We train classically with exact statevector simulation, then execute the quantum heads of our fusion checkpoints on a $156$-qubit IBM Heron processor, showing that NISQ-scale quantum modules can be installed in a production-scale LLM and run on hardware.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Maforikan Amoussou, Borja Aizpurua, Sukhbinder Singh, Román Orús, Hans-Arno Jacobsen. 2026-11-11. Multi-head Quantum Neural Networks: Replacing Transformer Blocks with NISQ Modules inside a 30-Billion-Parameter LLM. https://arxiv.science/abs/2611.15250

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

A Diamond Quantum Oscilloscope

Quantum-enhanced metrology has achieved remarkable precision in parameter estimation, but extending these advantages to continuous time-dependent fields (waveform sensing) remains a formidable challenge due to the infinite-dimensional nature of the signal. Here, we report the first experimental realization of a room-temperature solid-state quantum oscilloscope based on a sequential quantum difference amplification strategy. Within a zero-order-hold reconstruction framework, our method completely bypasses the need for massive entanglement. Crucially, we demonstrate that the overall waveform reconstruction error unequivocally follows the theoretically predicted Heisenberg scaling of \(O(N^{-1/2})\), decisively surpassing the classical \(O(N^{-1/3})\) limit. This transformative precision requires approximately 30-fold fewer quantum resources than conventional methods to achieve the same waveform error. Furthermore, we demonstrate the reconstruction of a multicomponent waveform and distinctly resolve weak high-frequency components that are entirely drowned in the shot noise of classical sampling. These results transition quantum metrology from parameter estimation into a practical instrument, establishing a genuine quantum advantage in continuous waveform reconstruction and paving the way for critical applications such as nanoscale nuclear magnetic resonance imaging, where detecting extremely weak signals is crucial for resolving molecular structures.

quant-ph↗

Information geometry and entanglement under phase-space deformation through nonsymplectic congruence transformation

We investigate the relationship between Fisher-Rao (FR) information geometry and Gaussian quantum correlations under nonsymplectic phase-space transformations, with particular emphasis on noncommutative (NC) deformation implemented through the Bopp shift. Using the established affine invariance of the FR-metric, we examine the transformation properties of Gaussian covariance matrices and the positive partial transpose (PPT) criterion under invertible congruence transformations. For a bipartite Gaussian system, we demonstrate that keeping the canonical partial-transpose (PT) matrix fixed in the deformed phase space produces deformation-dependent spectral shifts that may cross the conventional PPT threshold. When the PT operation is transported together with the covariance and commutation matrices, the physical PPT spectrum and separability classification remain invariant under local Darboux transformations. We further investigate a two-dimensional anisotropic oscillator in NC phase space, where the deformation parameters modify the effective canonical Hamiltonian and consequently influence the Gaussian ground-state correlations and entanglement between canonical modes. The corresponding FR-geometry reveals that the original covariance manifold possesses constant negative scalar curvature. In contrast, the pure symmetric normal-form covariance family is intrinsically one-dimensional and flat. These results distinguish the statistical geometry of Gaussian covariance matrices from the physical quantum correlations determined by the Hamiltonian, commutation structure, and subsystem decomposition.

quant-ph↗

Entanglement preservation and Clauser-Horne nonlocality in electromagnetically induced transparency quantum memories

Understanding how decoherence-induced loss affects entanglement and nonlocal correlations in quantum memories is important for quantum information science. Here we develop a quantitative description of photonic quantum states retrieved from electromagnetically induced transparency (EIT) memories in a ground-state-decoherence-limited regime. Using the dark-state-polariton formalism, we derive the attenuation factor of the retrieved probe mode from the EIT dynamics. This attenuation depends on the coupling field and ground-state coherence decay and is not introduced as an independent phenomenological loss parameter. Within a specified quantum-limited pure-loss model, we use this attenuation to obtain the reduced photonic density operator. We show that, in the symmetric case, the complete retrieved state of an initially path-entangled single photon remains entangled for any nonzero storage efficiency, despite acquiring a vacuum component. In contrast, violation of the unconditional Clauser-Horne inequality under displaced on/off detection without post-selection requires an intrinsic efficiency above 89.7%. We further derive the retrieved N-qubit photonic density operator for multiple independently operating EIT memories. These results establish a quantitative connection between EIT memory dynamics, retrieved quantum states, and operational nonlocality benchmarks under explicit channel assumptions.

quant-ph↗