peter.washington

4 min read 32 views

Non-Linear Regression

Non-Linear Regression_image

What if the x and y variables don’t have a linear relationship? For example, let’s say we want to build a regression model that lets us predict the population of a certain city at a certain time in the future. We were measuring the population every year for the past 5 years, and we plot our data:

Visualizing the data, it looks like it may follow an exponential pattern. Instead of fitting a line y = mx + b, let’s fit an exponential formula with 3 parameters a, b, and c:

We proceed with gradient descent as we did before, except we now calculate ∂y/∂a, ∂y/∂b, and ∂y/∂c, and update the values of a, b, and c accordingly.

After the gradient descent process, we end up with the following curve of best fit: