Skip to main content

A Paper Roadmap for Understanding Surrogate Gradients

· 11 min read
p4r4d0xb0x
p4r4d0xb0x
Rustacean, AI, OSS Enthusiast

Surrogate gradients are a detour connecting spiking neural networks, or SNNs, with the training tools of modern deep learning. The forward pass uses real spikes, while the backward pass uses a differentiable fake gradient. That sounds simple in one sentence, but properly understanding the idea requires three pieces of background.

First, why are spiking neurons discontinuous dynamical systems? Second, why does backpropagation require differentiability? Third, how were SNNs trained before surrogate gradients, and what prevented those methods from progressing?

Dark glass terminal roadmap from spiking neuron models to backpropagation and surrogate gradients

The target paper to read​

The target paper is Neftci, Mostafa, and Zenke's Surrogate Gradient Learning in Spiking Neural Networks. Rather than proposing "one new algorithm," this article reads more like a review that explains the problems of SNN training step by step. Its arXiv abstract makes the same point: SNNs are difficult to train because they are binary and dynamical, and surrogate gradients provide a flexible and efficient way around that problem.

Readers generally get stuck in three places.

  1. Why does the spike function produce dead gradients during backpropagation?
  2. What does BPTT propagate through a neural network's time axis?
  3. Why does training work with a surrogate derivative that is not the "true derivative"?

The roadmap below is therefore arranged in dependency order, not chronological order.

Term explainer: Surrogate gradient

A method that inserts a differentiable substitute for a nondifferentiable spike function only during backpropagation so that weights can be updated.

Example: A real traffic light switches suddenly from red to green, but a driving simulator can smooth the boundary to provide feedback that you "should have stopped a little sooner."

Step 1: A spiking neuron is a dynamical system, not a function​

Gerstner & Kistler, Spiking Neuron Models​

This is one of the best starting points for SNNs. It organizes foundational terms such as leaky integrate-and-fire, the spike-response model, refractory periods, and synaptic current. This layer is necessary to read membrane potential u(t)u(t), thresholds, and resets naturally in surrogate-gradient papers.

The key point is that an SNN neuron is not simply a node computing y=f(x)y = f(x). It accumulates input current over time, emits a spike when its membrane potential exceeds a threshold, and then resets its state. The output spike behaves approximately like a Heaviside step function.

Izhikevich, Simple Model of Spiking Neurons​

The Izhikevich model demonstrates a good balance between the biological detail of Hodgkin–Huxley-style models and the computational efficiency of LIF-style models. Reading this paper builds an intuition that "there is not one spike model; its level of abstraction depends on its purpose."

From the surrogate-gradient perspective, the differences matter. Regardless of the neuron model, the training bottleneck is similar: The instant at which a spike occurs is discontinuous, and a computation graph containing that event is not smooth like an ordinary ANN.

Step 2: Backpropagation pushes the chain rule through a computation graph​

Rumelhart, Hinton, Williams, Learning representations by back-propagating errors​

This is the classic backpropagation paper. It should be read because it gives the most compressed account of why deep learning needs gradients. The effect of the loss on each weight is calculated by multiplying the local derivatives of each layer. In other words, differentiable components must be linked by the chain rule.

That is exactly where SNNs encounter a problem. For spike output s=H(u−ϑ)s = H(u - \vartheta), the derivative is almost always zero away from the threshold and undefined at the threshold. The gradient does not flow for most of the time and breaks mathematically at the important moment.

Werbos, Backpropagation through time​

An SNN is a recurrent dynamical system with a time axis. Ordinary backpropagation is therefore insufficient; BPTT over a computation graph unrolled through time is required. Werbos's BPTT paper provides the background for understanding how recurrent state receives credit assignment over time.

This is why "temporal credit assignment" repeatedly appears in surrogate-gradient papers. A spike affects not only the current loss, but also later membrane potentials and future spikes.

Bengio et al., Estimating or Propagating Gradients Through Stochastic Neurons​

Although this is not an SNN paper, it is crucial for understanding surrogate gradients. It asks how to estimate gradients when training hard nonlinearities or stochastic binary neurons. The straight-through estimator appears here.

Broadly speaking, SNN surrogate gradients belong to this family of ideas. The forward pass retains a hard decision, while the backward pass carries a useful gradient signal. This opens the view that a "signal usable for learning" matters more than an "exact derivative."

Term explainer: Straight-through estimator

An approximation that retains a hard discrete decision in the forward pass but lets the gradient pass through during backpropagation as if the decision were an identity or smooth function.

Example: The exam result is reported only as pass or fail, but study feedback also says how many points short the student was.

Step 3: Understand SNN training before surrogate gradients​

Bi & Poo, Spike-Timing-Dependent Plasticity​

STDP frequently appears as a biological learning principle for SNNs. A synapse strengthens or weakens according to the time difference between presynaptic and postsynaptic spikes. This paper shows why SNNs evolved alongside "local learning rules."

Its limits are also clear from the perspective of deep supervised learning. STDP is natural for local spike timing, but it does not directly solve global credit assignment—how a loss several layers later should assign responsibility to an earlier synapse.

Bohte, Kok, La Poutré, SpikeProp​

SpikeProp is an early supervised-SNN paper that attempted error backpropagation over spike timing. It shows that even before surrogate gradients, researchers tried to make spike time a differentiable object for learning.

SpikeProp-style methods, however, depend heavily on spike-time representations and particular conditions. Modern deep SNNs need more general learning rules for layers, convolutions, recurrent structures, and event streams.

Diehl et al., Fast-classifying, high-accuracy spiking deep networks​

This paper represents the ANN-to-SNN conversion approach. It first trains an ordinary ANN, then converts it to an SNN by interpreting activations as firing rates. The approach was practical because it avoided the difficulty of training an SNN directly while retaining the advantages of an ANN that trains well.

Conversion can, however, limit the benefits of latency and temporal coding. It readily relies on rate approximations instead of making active use of spike timing, and maintaining performance at low timestep counts is difficult. Surrogate gradients matter because they expand the path toward training an SNN directly as an SNN.

Step 4: Core papers in the surrogate-gradient family​

Zenke & Ganguli, SuperSpike​

SuperSpike is essential reading. It uses a surrogate gradient to train deterministic integrate-and-fire neurons with supervised learning in a multilayer network. Its arXiv abstract describes deriving a voltage-based three-factor learning rule through a surrogate-gradient approach.

The important point is not to "smooth the spike function and change the forward pass too." The forward spike remains unchanged. Instead, a meaningful pseudo-derivative is defined only near the threshold during the backward pass. This makes it possible to train nonlinear tasks over spike-timing patterns.

Shrestha & Orchard, SLAYER​

As the name "spike layer error reassignment" suggests, SLAYER is a training framework that reassigns errors across layers and time. As emphasized in the arXiv abstract, it handles both the nondifferentiability of the spike-generation function and temporal credit assignment. Its GPU implementation and convolutional-SNN experiments also demonstrate that surrogate-gradient methods can grow into practical tools for training deep SNNs.

Wu et al., Spatio-Temporal Backpropagation​

STBP has a clear perspective of propagating through both spatial and temporal domains. The Frontiers abstract explains that an approximate derivative resolves the nondifferentiability of spike activity, combining layer-by-layer spatial propagation with a timing-dependent temporal domain.

This is a useful paper for viewing an SNN as a temporal signal-processing model rather than merely "an ANN with spikes attached."

Bellec et al., Long short-term memory and learning-to-learn in networks of spiking neurons​

The LSNN paper shows that a recurrent SNN combined with BPTT and adaptation can achieve capabilities approaching an LSTM. Its abstract identifies a lack of optimization as one reason RSNNs had underperformed ANNs and explains that powerful optimization such as BPTT can approximate functional contributions.

Viewing surrogate gradients only as a "classification trick" is too narrow. This paper reveals a path from SNN training toward memory, adaptation, and learning to learn.

Step 5: Return to the target review​

Returning now to Neftci, Mostafa, and Zenke's review makes its structure much clearer.

  • Spikes are binary and dynamical, making them harder to train than ordinary ANNs.
  • Local plasticity is biologically natural but insufficient for deep supervised objectives.
  • Conversion is practical but can limit the benefits of spike timing and low latency.
  • A surrogate derivative preserves event-driven spikes in the forward pass while opening the gradient bottleneck in the backward pass.

The core of surrogate gradients is therefore not "perfectly modeling the brain." More precisely, it is an interface that brings event-driven SNNs as a computational model into the gradient-based optimization ecosystem.

Reading-order summary​

OrderPaperWhy read it
1Gerstner & Kistler, Spiking Neuron ModelsLearn the language of LIF, thresholds, resets, and spike trains
2Izhikevich, Simple Model of Spiking NeuronsUnderstand the abstraction levels of neuron models
3Rumelhart et al., Back-propagating errorsUnderstand the original form of gradient learning
4Werbos, Backpropagation through timeUnderstand credit assignment over time
5Bengio et al., Stochastic neurons and hard non-linearitiesUnderstand straight-through estimators and learning with hard decisions
6Bi & Poo, STDPUnderstand the benefits and limits of local plasticity
7Bohte et al., SpikePropReview an early attempt at supervised spike learning
8Diehl et al., ANN-to-SNN conversionUnderstand the practical path before direct training
9Zenke & Ganguli, SuperSpikeUnderstand the core implementation of surrogate derivatives
10SLAYER, STBP, LSNNUnderstand deep, temporal, and recurrent SNN extensions
11Neftci et al., Surrogate Gradient Learning in SNNsIntegrate the entire progression through the review

Questions to keep in mind while reading​

  1. Does this paper view spikes as rates or as timing?
  2. Is the learning signal local, or does it come from a global loss?
  3. Does it explicitly handle temporal credit assignment?
  4. Does it preserve spikes in the forward pass and approximate only the backward pass?
  5. How well does it align with the event-driven advantages of neuromorphic hardware?

Keeping these five questions in mind reveals that surrogate gradients are not merely a "differentiation trick," but a design option for training SNNs as practical AI models.

Sources​

// COMMENTS

Comments