1

Minimizing Loss with Calculus

Unsolved
Calculus
Supervised

Difficulty: 6 | Problem written by ankita

Educational Resource: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture3.pdf


Problem reported in interviews at

Amazon
Apple
Facebook
Google
Netflix

Here, we will apply the Calculus to take the derivative of the loss function with respect to to the weights of the model and equate the derivative to zero to get optimum weight vector for which the loss would be minimized. In this case, we are taking mean squared loss as our loss function (defined below). 

Let's try out this method and calculate the optimum weights at which loss is minimized for a given set of training examples. The model we are trying to learn is:

f(x) = wX + wo

The corresponding loss function is:

Loss =  \(\tfrac{n}{2}\sum_{i=1}^{i=n}(y_{i}-f(x_{i}))^{2}\)

n = number of training examples

y = actual labels of training examples

You have to minimize loss with respect to w and equate the loss function's derivative to zero to get optimal w.

You are given as input:

X: X is a matrix that has n training examples

y: output labels for X

Output:

w: a weight vector that results in the minimum loss

Hint:

https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse

First element of W should be Wo i.e., if Y = wX + wo then W = [wo , elements of w]

To get the above W stack a column of ones to X at the beginning of X

Sample Input:
<class 'list'>
X: [[0.02019019, 0.2709028, 0.58130202, 0.96772135, 0.40203406, 0.68584788, 0.39907061, 0.8020824, 0.59922299, 0.89880733, 0.52469613, 0.2320425, 0.95274654, 0.57433355, 0.60232225, 0.12846666, 0.62600488, 0.75885062, 0.09451081, 0.5949559], [0.30851973, 0.92397012, 0.45213047, 0.31350788, 0.47674255, 0.99656559, 0.31500455, 0.67267856, 0.96399447, 0.03283226, 0.36633299, 0.17880501, 0.1446183, 0.66984436, 0.46706976, 0.03951692, 0.22784513, 0.82142612, 0.07370921, 0.33906808], [0.73137758, 0.66755197, 0.03242705, 0.29475688, 0.15821178, 0.26910982, 0.91078273, 0.70083964, 0.33650157, 0.26706248, 0.82314416, 0.17939914, 0.77290582, 0.42405843, 0.76003435, 0.30173729, 0.4576326, 0.57653883, 0.30827412, 0.85723128], [0.39220308, 0.17726624, 0.99721065, 0.76477702, 0.3495421, 0.44473093, 0.71881405, 0.3056014, 0.68752376, 0.38752906, 0.12147931, 0.13051651, 0.37855929, 0.180905, 0.59225714, 0.80858205, 0.92062923, 0.03778071, 0.81331967, 0.53359333], [0.07207221, 0.17216235, 0.10151733, 0.77153031, 0.95657225, 0.47837661, 0.30229184, 0.93057551, 0.04504919, 0.82969811, 0.16200046, 0.34971251, 0.04055285, 0.57460145, 0.58058546, 0.40074296, 0.55627212, 0.50018151, 0.10975096, 0.41768908], [0.64015041, 0.12854868, 0.09152847, 0.81948493, 0.64439314, 0.1571958, 0.86136143, 0.96147962, 0.25692185, 0.91043344, 0.0031553, 0.50970494, 0.09239349, 0.32355557, 0.02523059, 0.78752319, 0.42300025, 0.83668593, 0.45169828, 0.23836782], [0.15067413, 0.17847288, 0.34651435, 0.88828655, 0.70997458, 0.3427081, 0.22473914, 0.47540793, 0.12765958, 0.39060752, 0.60724015, 0.72244086, 0.51583481, 0.49931979, 0.2823342, 0.80178407, 0.62729361, 0.28726125, 0.34917165, 0.60606349], [0.79312299, 0.37268688, 0.58758082, 0.55694459, 0.35795187, 0.08427892, 0.80678922, 0.32206049, 0.55948097, 0.06842917, 0.46172133, 0.16606839, 0.150101, 0.84987818, 0.2154061, 0.17590738, 0.16686814, 0.81569503, 0.1565211, 0.04867117], [0.0049085, 0.37890289, 0.89008346, 0.9281833, 0.33230683, 0.91271647, 0.02732307, 0.54477629, 0.01319123, 0.30752729, 0.79310271, 0.39230414, 0.49702199, 0.95427926, 0.48890644, 0.61628695, 0.50168302, 0.36486363, 0.01473803, 0.77408549], [0.903083, 0.67791114, 0.61581447, 0.05392819, 0.91796546, 0.92341286, 0.40200506, 0.92874617, 0.35434179, 0.03242402, 0.04678233, 0.88053641, 0.79735547, 0.9512864, 0.73243777, 0.24907558, 0.26310536, 0.80493796, 0.68572655, 0.45102886]]
<class 'list'>
y: [[1], [0], [1], [0], [1], [1], [1], [0], [1], [1]]

Expected Output:
<class 'numpy.ndarray'>
[[-0.07140561] [ 0.0649857 ] [-0.07455911] [-0.25456612] [ 0.02489909] [-0.05027607] [ 0.09235117] [-0.0528404 ] [ 0.43180331] [-0.6064806 ] [ 0.34821837] [ 0.11929702] [ 0.31914999] [ 0.43665925] [ 0.09250638] [ 0.00743292] [ 0.16578263] [-0.0770846 ] [ 0.16158202] [-0.03285493] [ 0.3250566 ]]

This is a premium problem, to view more details of this problem please sign up for MLPro Premium. MLPro premium offers access to actual machine learning and data science interview questions and coding challenges commonly asked at tech companies all over the world

MLPro Premium also allows you to access all our high quality MCQs which are not available on the free tier.

Not able to solve a problem? MLPro premium brings you access to solutions for all problems available on MLPro

Get access to Premium only exclusive educational content available to only Premium users.

Have an issue, the MLPro support team is available 24X7 to Premium users.

This is a premium feature.
To access this and other such features, click on upgrade below.

Log in to post a comment

Comments
Ready.

Input Test Case

Please enter only one test case at a time
numpy has been already imported as np (import numpy as np)