Lesson 6.8ModelCode~45 min

Nearest neighbours

The laziest learner: predict by asking the most similar examples. Distances, k, and the curse of dimensionality.

Module 6 · 0/11 done

Learning without training

Here’s a completely different way to learn. Don’t fit any weights. Just remember every example, and when a new point arrives, look up the k most similar examples you’ve seen and let them vote. That’s k-nearest neighbours, or kNN.

Fig 6.8Drag the ? to ask its neighbours. Then slide k: with k = 1 the regions hug every point (overfitting); with a huge k they flatten out (underfitting).