arXiv subjects

Richard Naud

Publications and source records attributed to Richard Naud.

6 recordsLinked to original sources

arXiv2026

Large Language Models with At Most One Spike per Neuron

Leveraging their inherent sparse event-driven computation, spiking neural networks (SNNs) offer a promising path toward energy-efficient large language models (LLMs). Time-to-first-spike (TTFS) coding generates at most one spike per neuron within a time window, yielding extremely low firing rates. However, conventional TTFS SNNs are restricted to specific structures, making it challenging to encode certain blocks in LLM -- such as layer normalization and matrix multiplication --using TTFS. To overcome this limitation, we introduce a reference-based strategy specifically to encode the four core LLM components: embedding layers, layer normalization, attention-related operations and dropout. We construct a fully TTFS-based SNN architecture and train it end-to-end. Experiments on modern LLMs like BERT and GPT-2 demonstrate that our approach achieves performance comparable to ANN counterparts on natural language understanding and common-sense reasoning, while a clear gap remains on language modeling perplexity. To the best of our knowledge, this is the first work to scale a spiking LLM to 1.5 billion parameters using TTFS coding. We also report an estimate of spike-related energy; this is a spike-count proxy under an established cost model rather than a measurement on neuromorphic hardware.

cs.NE↗

arXiv2023

Connecting levels of analysis in the computational era

Neuroscience and artificial intelligence are closely intertwined, but so are the physics of dynamical system, philosophy and psychology. Each of these fields try in their own way to relate observations at the level of molecules, synapses, neurons or behavior, to a function. An influential conceptual approach to this end was popularized by David Marr, which focused on the interaction between three theoretical 'levels of analysis'. With the convergence of simulation-based approaches, algorithm-oriented Neuro-AI and high-throughput data, we currently see much research organized around four levels of analysis: observations, models, algorithms and functions. Bidirectional interaction between these levels influences how we undertake interdisciplinary science.

q-bio.NC↗

arXiv2023

Silences, Spikes and Bursts: Three-Part Knot of the Neural Code

When a neuron breaks silence, it can emit action potentials in a number of patterns. Some responses are so sudden and intense that electrophysiologists felt the need to single them out, labeling action potentials emitted at a particularly high frequency with a metonym -- bursts. Is there more to bursts than a figure of speech? After all, sudden bouts of high-frequency firing are expected to occur whenever inputs surge. The burst coding hypothesis advances that the neural code has three syllables: silences, spikes and bursts. We review evidence supporting this ternary code in terms of devoted mechanisms for burst generation, synaptic transmission and synaptic plasticity. We also review the learning and attention theories for which such a triad is beneficial.

q-bio.NC↗

arXiv2015

Fluctuations and information filtering in coupled populations of spiking neurons with adaptation

Finite-sized populations of spiking elements are fundamental to brain function, but also used in many areas of physics. Here we present a theory of the dynamics of finite-sized populations of spiking units, based on a quasi-renewal description of neurons with adaptation. We derive an integral equation with colored noise that governs the stochastic dynamics of the population activity in response to time-dependent stimulation and calculate the spectral density in the asynchronous state. We show that systems of coupled populations with adaptation can generate a frequency band in which sensory information is preferentially encoded. The theory is applicable to fully as well as randomly connected networks, and to leaky integrate-and-fire as well as to generalized spiking neurons with adaptation on multiple time scales.

q-bio.NC↗

arXiv2013

An Integral Equation Approach to the Dynamics of L2-3 Cortical Neurons

How do neuronal populations encode time-dependent stimuli in their population firing rate? To address this question, I consider the quasi-renewal equation and the event-based expansion, two theoretical approximations proposed recently, and test these against peri-stimulus time histograms from L2-3 pyramidal cells in vitro. Parameters are optimized by gradient descent to best match the firing rate output given the current input. The fitting method can estimate single-neuron parameters that are normally obtained either with intracellular recordings or with individual spike trains. I find that quasi-renewal theory predicts the adapting firing rate with good precision but not the event-based expansion. Quasi-renewal predictions are equal in quality with state-of-the-art spike timing prediction methods, and does so without resorting to the indiviual spike times or the membrane potential responses.

q-bio.NC↗

arXiv2013

Spike timing prediction with active dendrites

A complete single-neuron model must correctly reproduce the firing of spikes and bursts. We present a study of a simplified model of deep pyramidal cells of the cortex with active dendrites. We hypothesized that we can model the soma and its apical tuft with only two compartments, without significant loss in the accuracy of spike-timing predictions. The model is based on experimentally measurable impulse-response functions, which transfer the effect of current injected in one compartment to current reaching the other. Each compartment was modeled with a pair of non-linear differential equations and a small number of parameters that approximate the Hodgkin-and-Huxley equations. The predictive power of this model was tested on electrophysiological experiments where noisy current was injected in both the soma and the apical dendrite simultaneously. We conclude that a simple two-compartment model can predict spike times of pyramidal cells stimulated in the soma and dendrites simultaneously. Our results support that regenerating activity in the dendritic tuft is required to properly account for the dynamics of layer 5 pyramidal cells under in-vivo-like conditions.

q-bio.NC↗