arXiv subjects

Benjamin Merkel

Publications and source records attributed to Benjamin Merkel.

9 recordsLinked to original sources

arXiv2025

Assembly of Experts: Linear-time construction of the Chimera LLM variants with emergent and adaptable behaviors

Requiring $10^{13}$-$10^{15}$ FLOPs to calculate one 8 bit weight in an LLM during pretraining is extremely expensive and seems inefficient. To better leverage the huge investments made into pretrained models, we develop the new "Assembly-of-Experts" (AoE) construction method to create capable child variants of existing Mixture-of-Experts parent models in linear time. Model weight tensors get interpolated individually, allowing to enhance or suppress semantic features of the parents. Varying the proportion of weights taken from the parent models, we observe some properties of the AoE child model changing gradually, while other behavioral traits emerge with a sharp transition. Surprisingly, nearly every generated model is functional and capable, which makes searching the model space straightforward. We construct the DeepSeek R1T "Chimera", a 671B open-weights hybrid model combining DeepSeek's V3-0324 and R1 model variants. The child inherits only the routed expert tensors of R1, but still achieves about R1-level intelligence. At the same time, it uses about 40\% fewer output tokens, close to V3 speed. Constructed without any fine-tuning or distillation, the Chimera exhibits surprisingly compact, orderly reasoning compared to its parent models.

cs.LG↗

arXiv2025

Cavity-enhanced spectroscopy of individual nuclear spins in a dense bath

Echo-based spectroscopy of the superhyperfine interaction of an electronic spin with nuclear spins in its surroundings enables detailed insights into the microscopic magnetic environment of spins in solids. Still, it is an outstanding challenge to resolve individual nuclear spins in a dense bath, in which many of them exhibit a comparable coupling strength. This simultaneously requires a high spectral resolution and a large signal-to-noise ratio. However, when probing spin ensembles, dipolar interactions between the dopants can lead to a concentration-dependent trade-off between resolution and signal. Here, we fully eliminate this limitation of previous optical-echo-envelope-modulation spectroscopy experiments by integrating the emitters into a high-finesse resonator, which allows for strong optical echoes even at very low concentrations. To demonstrate its potential, the technique is applied to erbium dopants in yttrium-orthosilicate (Er:YSO). Achieving an unprecedented spectral resolution enables precise measurements of the superhyperfine interaction with four of the Y nuclear spins densely surrounding each emitter. The achieved boost of the signal, enabled by the resonator, allows for extending the approach to the lowest concentration possible -- to the level of single dopants, thereby providing a tool for detecting and studying individual nuclear spins. Thus, our technique paves the way for an improved understanding of dense nuclear spin baths in solids.

quant-ph↗

arXiv2025

Mixture of Tunable Experts -- Behavior Modification of DeepSeek-R1 at Inference Time

We present the Mixture-of-Tunable-Experts (MoTE), a method that extends the Mixture-of-Experts architecture of Large Language Models (LLMs). Without additional training, MoTE enables meaningful and focused behavior changes in LLMs on-the-fly during inference time. By analyzing the digital LLM brain of DeepSeek-R1 using a technique we dub 'functional Token Resonance Imaging' (fTRI) -- inspired by fMRI and using prompts designed to elicit specific behavior (e.g., 'What happened {time}{place}?') -- we empirically identify distinctive experts associated with behaviors like refusal responses. Using MoTE we are able to intervene and control such specific behavior. We switched off the top 10 most refusal-relevant experts (0.07% of R1's 14,848 routed experts), achieving a 52% refusal reduction on sensitive reference prompts without performance degradation on MT-Bench. Random expert deactivation resulted in smaller behavioral shifts with increased noise, whereas forced expert activation led to significantly higher refusal rates. Our approach shares similarities with sparse autoencoders (SAEs) in terms of explainability and steerability. Unlike SAEs, MoTE does not require large training efforts, as within MoEs with a vast number of experts, specialization already emerged naturally during pretraining. Our findings suggest that significant functional mechanisms in Mixture-of-Experts architectures can at least partially be localized in a small number of specific experts, rather than being distributed throughout the model's weights. Expert subgroups can be tuned to trigger significant behavior variations, providing insights into the inner workings of LLMs.

cs.AI↗

arXiv2021

Spectral multiplexing of telecom emitters with stable transition frequency

In a quantum network, coherent emitters can be entangled over large distances using photonic channels. In solid-state devices, the required efficient light-emitter interface can be implemented by confining the light in nanophotonic structures. However, fluctuating charges and magnetic moments at the nearby interface then lead to spectral instability of the emitters. Here we avoid this limitation when enhancing the photon emission up to 70(12)-fold using a Fabry-Perot resonator with an embedded 19 micrometer thin crystalline membrane, in which we observe around 100 individual erbium emitters. In long-term measurements, they exhibit an exceptional spectral stability of < 0.2 MHz that is limited by the coupling to surrounding nuclear spins. We further implement spectrally multiplexed coherent control and find an optical coherence time of 0.11(1) ms, approaching the lifetime limit of 0.3 ms for the strongest-coupled emitters. Our results constitute an important step towards frequency-multiplexed quantum-network nodes operating directly at a telecommunication wavelength.

quant-ph↗

arXiv2021

Coherent control in the ground and optically excited state of an ensemble of erbium dopants

Ensembles of erbium dopants can realize quantum memories and frequency converters that operate in the minimal-loss wavelength band of fiber optical communication. Their operation requires the initialization, coherent control and readout of the electronic spin state. In this work, we use a split-ring microwave resonator to demonstrate such control in both the ground and optically excited state. The presented techniques can also be applied to other combinations of dopant and host, and may facilitate the development of new quantum memory protocols and sensing schemes.

quant-ph↗

arXiv2020

Laser stabilization to a cryogenic fiber ring resonator

The frequency stability of lasers is limited by thermal noise in state-of-the-art frequency references. Further improvement requires operation at cryogenic temperature. In this context, we investigate a fiber-based ring resonator. Our system exhibits a first-order temperature-insensitive point around $3.55$ K, much lower than that of crystalline silicon. The observed low sensitivity with respect to vibrations ($<5\cdot{10^{-11}}\,\text{m}^{-1} \text{s}^{2}$), temperature ($-22(1)\cdot{10^{-9}}\,\text{K}^{-2}$) and pressure changes ($4.2(2)\cdot{10^{-11}}\,\text{mbar}^{-2}$) makes our approach promising for future precision experiments.

physics.optics↗

arXiv2020

Coherent and Purcell-enhanced emission from erbium dopants in a cryogenic high-Q resonator

The stability and outstanding coherence of dopants and other atom-like defects in tailored host crystals make them a leading platform for the implementation of distributed quantum information processing and sensing in quantum networks. Albeit the required efficient light-matter coupling can be achieved via the integration into nanoscale resonators, in this approach the proximity of interfaces is detrimental to the coherence of even the least-sensitive emitters. Here, we establish an alternative: By integrating a 19 micrometer thin erbium-doped crystal into a cryogenic Fabry-Perot resonator with a quality factor of nine million, we can demonstrate 59(6)-fold enhancement of the emission rate, corresponding to a two-level Purcell factor of 530(50), while preserving lifetime-limited optical coherence up to 0.54(1) ms. With its emission at the minimal-loss wavelength of optical fibers and its outcoupling efficiency of 46(8) %, our system enables coherent and efficient nodes for long-distance quantum networks.

quant-ph↗

arXiv2020

Dynamical decoupling of spin ensembles with strong anisotropic interactions

Ensembles of dopants have widespread applications in quantum technology. The miniaturization of corresponding devices is however hampered by dipolar interactions that reduce the coherence at increased dopant density. We theoretically and experimentally investigate this limitation. We find that dynamical decoupling can alleviate, but not fully eliminate, the decoherence in crystals with strong anisotropic spin-spin interactions. Our findings can be generalized to all quantum systems with anisotropic g-factor used for quantum sensing, microwave-to-optical conversion, and quantum memory.

quant-ph↗

arXiv2020

Erbium dopants in silicon nanophotonic waveguides

The combination of established nanofabrication with attractive material properties makes silicon a promising material for quantum technologies, where implanted dopants serve as qubits with high density and excellent coherence even at elevated temperatures. In order to connect and control these qubits, interfacing them with light in nanophotonic waveguides offers unique promise. Here, we present resonant spectroscopy of implanted erbium dopants in such waveguides. We overcome the requirement of high doping and above-bandgap excitation that limited earlier studies. We thus observe erbium incorporation at well-defined lattice sites with a thousandfold reduced inhomogeneous broadening of about 1 GHz and a spectral diffusion linewidth down to 45 MHz. Our study thus introduces a novel materials platform for the implementation of on-chip quantum memories, microwave-to-optical conversion, and distributed quantum information processing, with the unique feature of operation in the main wavelength band of fiber-optic communication.

physics.app-ph↗