peter.washington

2 min read 21 views

Overfitting Intuition 1

Overfitting Intuition 1_image

Let’s say we have the following data, and we learn the following piecewise line as a regression predictor:

Now, let’s say we get the following test data (visualized as the white data points):

As we can see, the learned regression function is very specific to the training data (black points) without generalizing to the test data (white points). This is a general problem in machine learning called overfitting.

If we instead used a plain old straight line to fit the data, we would get a much more generalizable fit:

The goal of the methods we will cover in this chapter are to generate a generalized fit to the training data, like the line of above, rather than a super specific fit that will not generalize to other data points. We aim to accomplish this in an algorithmic way that is specific to the dataset.