Lesson 4.4MathCode~1 h

Gradients & going downhill

Slopes in many directions at once, and the ball that rolls to the bottom: gradient descent.

Module 4 · 0/7 done

Going downhill, one step at a time

Here is gradient descent in one dimension. The curve is a loss: how wrong a model is for each value of its parameter $\green{w}$ . We start somewhere, look at the slope, and take a step against it — downhill.

\green{w} \leftarrow \green{w} - \eta \cdot \pink{\frac{dL}{dw}}

Fig 4.6The anatomy of one step. Drag the point; the green arrow is where one step of gradient descent lands.