An analog-electronic implementation of a harmonic oscillator recurrent neural network

Abstract

Oscillatory recurrent networks, such as the Harmonic Oscillator Recurrent Network (HORN) model, offer advantages in parameter efficiency, learning speed, and robustness relative to traditional non-oscillating architectures. Yet, while many implementations of physical neural networks exploiting attractor dynamics have been studied, implementations of oscillatory models in analog-electronic hardware that utilize the networks' transient dynamics so far are lacking. This study explores the feasibility of implementing HORNs in analog-electronic hardware while maintaining the computational performance of the digital counterpart. Using a digital twin approach, we trained a four-node HORN in silico for sequential MNIST classification and transferred the trained parameters to an analog electronic implementation. A set of custom error metrics indicated that the analog system is able to successfully replicate the dynamics of the digital model in most test cases. However, despite the overall well-matching dynamics, when using the readout layer of the digital model on the data generated by the analog system, we only observed $28.39\%$ agreement with the predictions of the digital model. An analysis shows that this mismatch is due to a precision difference between the analog hardware and the floating-point representation exploited by the digital model to perform classification tasks. When the analog system was utilized as a reservoir with a re-trained linear readout, its classification performance could be recovered to that of the digital twin, indicating preserved information content within the analog dynamics. This proof-of-concept establishes that analog electronic circuits can effectively implement oscillatory neural networks for computation, providing a demonstration of energy-efficient analog systems that exploit brain-inspired transient dynamics for computation.