Gradient
Lesson 7.1ModelMath~40 min

The artificial neuron

Weights, a bias and a squish. One neuron, fully dissected.

Module 7 · 0/11 done

Loosely inspired by the brain

A brain cell (a neuron) receives signals from other cells, and if the total stimulation is strong enough, it fires its own signal onward. In 1943 two scientists sketched a mathematical caricature of this, and almost every AI system today is built from millions of copies of it.

The artificial neuron is something you already know. It takes some inputs, multiplies each by a weight, adds them up with a bias, and passes the total through an activation function:

a=f(w1x1+w2x2+⋯+b)=f(w⋅x+b)\amber{a} = f\big(\green{w_1}\blue{x_1} + \green{w_2}\blue{x_2} + \dots + \green{b}\big) = f(\green{\mathbf{w}}\cdot\blue{\mathbf{x}} + \green{b})

A dot product, plus a bias, through a squish. That’s the whole thing.