Networks and dynamics

Wire thousands of neurons together at random, a fifth of them inhibitory, and they settle into firing that looks like noise but is the network's own. This chapter shows where that comes from, how the network changes state as its balance shifts, and why no two runs of it are ever alike.

A puzzle about cortex

Record a neuron in a living cortex and its spikes look random: the intervals between them vary about as much as a Poisson process’s, the purest kind of randomness. Yet chapter 3’s neuron, given a steady input, fires like a clock. Something about being in a network makes neurons irregular.

The answer most theorists accept is balance. A cortical neuron receives thousands of excitatory inputs and a comparable inhibitory drive. If excitation alone acted, the neuron would be driven far past threshold and fire regularly at its maximum rate. But inhibition cancels most of it. What remains is a mean input just below threshold plus large fluctuations, as the many inputs arrive at random moments. The neuron fires when a fluctuation carries it over, and fluctuations arrive irregularly.

Brunel’s network

Brunel (2000) built the simplest network that shows this, and worked out its behaviour analytically. It has $N_E$ excitatory and $N_I = N_E/4$ inhibitory LIF neurons. Each neuron receives a fixed number of connections chosen at random, 10% of each population: $C_E$ excitatory and $C_I = C_E/4$ inhibitory. An excitatory spike raises the membrane by $J = 0.1$ mV, an inhibitory one lowers it by $gJ$, and every spike arrives $D = 1.5$ ms after it was sent. Each neuron also receives Poisson spikes from outside the network through $C_E$ more excitatory synapses.

Two numbers set its behaviour. The first is the relative strength of inhibition, $g$. If every neuron fires at the same rate $\nu$, the mean recurrent input is proportional to

$$C_E\,\nu\,J - C_I\,\nu\,gJ = C_E\,\nu\,J\left(1 - \frac{g}{4}\right)$$

so at $g = 4$ excitation and inhibition cancel exactly, and above 4 inhibition wins. The second is the external drive. Brunel measured it in units of $\nu_\text{thr} = \theta / (J C_E \tau)$, the external rate that would bring the mean membrane exactly to threshold with no recurrent input at all; $\eta = \nu_\text{ext}/\nu_\text{thr}$.

Depending on $g$ and $\eta$, his network falls into one of four states. In his figure 8, with $N_E = 10{,}000$:

State	$g$	$\eta$	What the network does
Synchronous regular	3	2	Excitation dominates; neurons fire in near lockstep, each almost periodically
Asynchronous irregular	5	2	Balanced; each neuron fires irregularly and the population rate is flat
Synchronous irregular, fast	6	4	Strong inhibition and drive; the population oscillates fast while each neuron skips most cycles
Synchronous irregular, slow	4.5	0.9	Weak drive; the population oscillates slowly, neurons irregular within it

A smaller network

His network is too big to run smoothly in a browser, and shrinking it is not innocent. Brunel’s analysis assumes each neuron receives many small inputs: 1,000 excitatory synapses of 0.1 mV. A network a tenth the size gives each neuron 100 such inputs, and their total is a tenth of his. The recurrent input is then too weak to balance the external drive, and the network fires fast and regularly at his asynchronous irregular parameters.

One way to keep what matters is to scale the synapses with the size, so that each neuron’s total excitatory input stays at his $C_E J = 100$ mV: 100 inputs of 1 mV in a tenth of his network, 200 of 0.5 mV in the figure’s 2,500 neurons. The mean input is then the same as in his network, and the fluctuations are larger, since fewer, bigger inputs make a noisier sum.

brunel

g 5, η 2g 3, η 2g 6, η 4g 4.5, η 0.9

His total inputHis synapses, 0.1 mV

1,2502,5005,000

Inhibition g

5.0

External drive η

2.00

· rate … · CV … · Fano …

Each dot is a spike: 160 excitatory neurons above, 40 inhibitory below, over the last 400 ms. The trace below is the excitatory population’s rate, and the line under the figure measures the network as it runs, from its first 100 ms on: the median CV of its excitatory neurons’ intervals and the Fano factor of their count in 1 ms windows (both explained below). Pick one of Brunel’s four parameter sets, or move $g$ and $\eta$ yourself, and switch between his synapses and his total input. The network runs in a worker in your browser with sparx’s step: each membrane solved exactly over 0.1 ms, delta synapses that arrive after their delay and land before the threshold test, and the reset held for 2 ms. Its test holds it to sparx’s own run of a smaller Brunel network, spike for spike.

Measuring irregularity

Two numbers separate the states. The coefficient of variation of a neuron’s interspike intervals, $\text{CV} = \sigma_\text{ISI}/\mu_\text{ISI}$, is 0 for a clock and 1 for a Poisson process. The Fano factor of the population’s spike count in short windows, its variance over its mean, is near 1 when neurons fire independently and far above 1 when they fire together. Synchronous regular has a low CV and a high Fano factor; asynchronous irregular has a CV near 1 and a Fano factor near 1.

sparx’s brunel(order) builds his network at any size, brunel(2500) being his, and sparx.spiketrains measures it. This runs a tenth of it, 1,250 neurons, with his synapses and then with his total input:

import jax
import numpy as np

from sparx.graph import SpikeRaster, simulate
from sparx.graph.models import brunel
from sparx.spiketrains import cv_isi, population_fano, rates_hz


def measure(g, eta, j, order=250):            # 5 * order neurons
    network = brunel(order, g=g, eta=eta, j=j)
    result = simulate(network, network.init(jax.random.key(0)),
                      duration=600.0, key=jax.random.key(1),
                      monitors={"e": SpikeRaster("e")})
    spikes = np.asarray(result.records["e"])[2000:]  # after 200 ms
    return (f"{rates_hz(spikes, 0.1).mean():5.1f} Hz, "
            f"CV {np.median(cv_isi(spikes)):.2f}, "
            f"Fano {population_fano(spikes, 0.1):5.1f}")


# Brunel's synapses, 0.1 mV, but each neuron has 100 inputs, not 1,000
print("g 5, eta 2, j 0.1:", measure(5.0, 2.0, 0.1))
# Synapses ten times stronger: 100 inputs of 1 mV, as his 1,000 of 0.1
for g, eta in [(3.0, 2.0), (5.0, 2.0), (6.0, 4.0), (4.5, 0.9)]:
    print(f"g {g}, eta {eta}, j 1.0:", measure(g, eta, 1.0))

Run it in ColabSource

With his 0.1 mV synapses, his asynchronous irregular parameters give 85.8 Hz with a CV of 0.13: regular firing. With 1 mV synapses the same parameters give 73.5 Hz and a CV of 1.47, irregular, though the Fano factor of 14.7 says the population still fluctuates together more than independent neurons would. His synchronous regular parameters give 327 Hz, a CV of 0.09 and a Fano factor of 156, and his slow synchronous parameters 38.7 Hz, a CV of 1.18 and a Fano factor of 29.8. A tenth of his network is not his network, but the progression from clockwork to irregular firing as inhibition grows is there.

One spike changes everything

Balance has a second consequence. Van Vreeswijk and Sompolinsky showed that balanced networks are chaotic: their activity is irregular even when every input is constant, and the smallest change to it grows until the network’s spikes have nothing in common with what they would have been.

London and colleagues measured how sensitive a living cortex is. In rats, they added a single extra spike to one neuron and estimated that it caused about 28 additional spikes in the neurons it connects to.

Here are two copies of one network of 1,250 neurons with 1 mV synapses: the same connections, the same external input, and so, step after step, the same spikes. Both copies are drawn in one raster: a spike fired by both is grey, a spike fired by only the first copy is orange, and one fired by only the second is violet.

brunel × 2

Nudge one neuronStart over

Asynchronous irregularSynchronous regular

Nudge one neuron of the second copy over threshold, once. In the first 10 ms after the nudge, 1,026 spikes already differ between the copies, and 50 to 100 ms after it, 6,358 of their 6,450 spikes do, while the two copies’ population rates, shown below the figure, stay about the same. In the synchronous regular state the population rhythm survives, but 50 ms after the nudge two spikes in five still differ: which neuron fires on which cycle is lost.

For a simulator this means two runs of a chaotic network cannot agree spike for spike, unless every operation rounds identically. The meaningful comparison is the one this chapter used to tell the states apart: rates, CVs and Fano factors over many runs. Chapter 11 returns to this.

How sparx runs a network

A sparx.graph.Network is a Flax module made of populations, projections between them and inputs. Its variables split by role: connectome holds the edges and fixed weights, params any trainable weights, and state the membranes, synapses and spikes in flight. Every step runs in NEST’s order: delta synapses deliver what is due, membranes integrate and fire, spikes enter each population’s ring buffer of delays, other synapses receive their arrivals, plasticity updates, and monitors record. A spike sent in step $m$ over a delay of $D$ steps lands at the end of step $m + D$.

Spikes are delivered as events: each step takes the neurons that fired and adds their out-edges, so a quiet network costs little. A projection can also be stored as a dense matrix or a list of edges, and every format gives the same input up to the order of summation.

The same machinery runs Shiu et al.’s model of the whole fly brain, built from the FlyWire connectome: 127,400 neurons and 14.7 million connections between them. With 21 sugar-sensing neurons activated at 100 Hz, sparx’s rates over ten trials correlate with their published Brian2 runs at 0.9989, and the motor neuron MN9 fires at 67.1 Hz against their 67.0 ± 6.6. It takes about 30 s per simulated second on 4 CPU cores.

Try this

1. With $g = 4$ and $\eta = 2$, what is the mean recurrent input? What fires the neurons then?
2. In the raster, at $g = 5$ and $\eta = 2$, switch between his synapses and his total input. Which fires irregularly, and why does the number of inputs matter?
3. In the twin figure, nudge the network, wait until the copies have parted, and press “Start over”. The copies are identical again. Why can’t a real experiment do this?
4. Why do two chaotic networks keep the same population rate after they part?

Answers

1. Zero: excitation and inhibition cancel on average. The external input alone, at twice the threshold rate, then sets the mean twice as far above rest as threshold, and the neurons fire fast and regularly.
2. His total input. With 100 inputs of 0.1 mV, a neuron’s recurrent inhibition can cancel only a tenth of what it does in his network, so the external drive, which alone would hold the mean at twice the threshold, wins and the neurons fire like clocks. With 1 mV synapses the recurrent input is as strong as his, inhibition balances the drive, and fluctuations decide when each neuron fires.
3. The two copies share every random number: their connections and every external spike. A brain never repeats its input exactly, and its neurons have noise of their own, so every trial starts from a different state, and the chaos amplifies that difference.
4. The rate is set by the balance of mean inputs, which depends on the parameters, not on which neuron fired when. Chaos scrambles the details and keeps the statistics.

Summary

With inhibition balancing excitation, each neuron sits below threshold and fires irregularly on fluctuations, and Brunel’s network moves between four states as inhibition and drive change, though a small copy shows them only with its synapses scaled up to his total input. Balanced networks are chaotic: one nudged neuron makes two identical copies part within tens of milliseconds while their statistics stay the same. The next chapter puts the units back and looks at what real synapses and neurons add.

References

• N. Brunel, “Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons”, Journal of Computational Neuroscience 8, 2000, doi:10.1023/A:1008925309027. Figure 8.
• C. van Vreeswijk and H. Sompolinsky, “Chaos in neuronal networks with balanced excitatory and inhibitory activity”, Science 274, 1996, doi:10.1126/science.274.5293.1724.
• M. London, A. Roth, L. Beeren, M. Häusser and P. E. Latham, “Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex”, Nature 466, 2010, doi:10.1038/nature09086.
• P. K. Shiu et al., “A Drosophila computational brain model reveals sensorimotor processing”, Nature 634, 2024, doi:10.1038/s41586-024-07763-9.