Blogs/K Nearest Neighbors (KNN) Classification

K Nearest Neighbors (KNN) Classification

peterwashington Nov 11 2021 3 min read 0 views

A very different, but potentially easier to understand classification method, is k-nearest neighbors.  This method is best described with a visual example. Let’s say we plot the length of the beak and wingspan of different bird species. Bird species of type A are represented as a circle and type B are represented as a square:

We want to classify a new unknown data point represented by the question mark above. When k is 3, we find the 3 data points that are closest to the test point we want to predict for. We circle these 3 points below:

Because the majority of the 3 closest points are of type A, we classify the point as type A.

The choice of k is important. If we change k to 1, notice that the classification changes to type B:

To solve this issue in practice, it is common to perform hyperparameter tuning. The k is called a hyperparameter, and we can iterate through several values of k to see which gives the best training performance. We will see more complex examples of hyperparameter tuning in subsequent chapters.