Field-induced intermediate phase in $\alpha$-RuCl$_3$: Non-coplanar order, phase diagram, and proximate spin liquid

Abstract

Frustrated magnets with strong spin-orbit coupling promise to host topological states of matter, with fractionalized excitations and emergent gauge fields. Kitaev's proposal for a honeycomb-lattice Majorana spin liquid has triggered an intense search for experimental realizations, with bond-dependent Ising interaction being the essential building block. A prime candidate is $\alpha$-RuCl$_3$ whose phase diagram in a magnetic field is, however, not understood to date. Here we present conclusive experimental evidence for a novel field-induced ordered phase in $\alpha$-RuCl$_3$, sandwiched between the zigzag and quantum disordered phases at low and high fields, respectively. We provide a detailed theoretical study of the relevant effective spin model which we show to display a field-induced intermediate phase as well. We fully characterize the intermediate phase within this model, including its complex spin structure, and pinpoint the parameters relevant to $\alpha$-RuCl$_3$ based on the experimentally observed critical fields. Most importantly, our study connects the physics of $\alpha$-RuCl$_3$ to that of the Kitaev-$\Gamma$ model, which displays a quantum spin liquid phase in zero field, and hence reveals the spin liquid whose signatures have been detected in a variety of dynamical probes of $\alpha$-RuCl$_3$.