What if the x and y variables don’t have a linear relationship? For example, let’s say we want to build a regression model that lets us predict the population of a certain city at a certain time in the future. We were measuring the population every year for the past 5 years, and we plot our data:
Visualizing the data, it looks like it may follow an exponential pattern. Instead of fitting a line y = mx + b, let’s fit an exponential formula with 3 parameters a, b, and c:
We proceed with gradient descent as we did before, except we now calculate ∂y/∂a, ∂y/∂b, and ∂y/∂c, and update the values of a, b, and c accordingly.
After the gradient descent process, we end up with the following curve of best fit: