Gradient
Lesson 6.2ModelCode~1 h

The training loop

Predict, measure, nudge, repeat. Write gradient descent that fits a line by itself.

Module 6 · 0/11 done

Let the slope do the work

The loss L(w,b)\pink{L}(\green{w}, \green{b}) is a landscape over the two parameters. We know how to go downhill on a landscape: compute the gradient, step against it. All we need are the two partial derivatives. Using the chain rule on each squared error (wxi+b−yi)2(\green{w}x_i + \green{b} - y_i)^2:

∂L∂w=2n∑i(y^i−yi)⏟error xi\pink{\frac{\partial L}{\partial w}} = \frac{2}{n}\sum_{i} \underbrace{(\amber{\hat{y}_i} - y_i)}_{\text{error}}\, \blue{x_i}
∂L∂b=2n∑i(y^i−yi)\pink{\frac{\partial L}{\partial b}} = \frac{2}{n}\sum_{i} (\amber{\hat{y}_i} - y_i)