Blogs/Overfitting Intuition 1

Overfitting Intuition 1

peter.washington Oct 04 2021 2 min read 38 views

Stanford applied ML PhD

Optimization Supervised
Overfitting1.png

Let’s say we have the following data, and we learn the following piecewise line as a regression predictor:

Overfitting Intuition

Now, let’s say we get the following test data (visualized as the white data points):

Overfitting Intuition

As we can see, the learned regression function is very specific to the training data (black points) without generalizing to the test data (white points). This is a general problem in machine learning called overfitting.

If we instead used a plain old straight line to fit the data, we would get a much more generalizable fit:

Overfitting Intuition

The goal of the methods we will cover in this chapter are to generate a generalized fit to the training data, like the line of above, rather than a super specific fit that will not generalize to other data points. We aim to accomplish this in an algorithmic way that is specific to the dataset.

 

Learn and practice this concept here: 

https://mlpro.io/problems/lasso-regularization/