1

Minimizing Loss with Calculus

Unsolved
Calculus
Supervised

Difficulty: 6 | Problem written by ankita

Educational Resource: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture3.pdf


Problem reported in interviews at

Amazon
Apple
Facebook
Google
Netflix

Here, we will apply the Calculus to take the derivative of the loss function with respect to to the weights of the model and equate the derivative to zero to get optimum weight vector for which the loss would be minimized. In this case, we are taking mean squared loss as our loss function (defined below). 

Let's try out this method and calculate the optimum weights at which loss is minimized for a given set of training examples. The model we are trying to learn is:

f(x) = wX + wo

The corresponding loss function is:

Loss =  \(\tfrac{n}{2}\sum_{i=1}^{i=n}(y_{i}-f(x_{i}))^{2}\)

n = number of training examples

y = actual labels of training examples

You have to minimize loss with respect to w and equate the loss function's derivative to zero to get optimal w.

You are given as input:

X: X is a matrix that has n training examples

y: output labels for X

Output:

w: a weight vector that results in the minimum loss

Hint:

https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse

First element of W should be Wo i.e., if Y = wX + wo then W = [wo , elements of w]

To get the above W stack a column of ones to X at the beginning of X

Sample Input:
<class 'list'>
X: [[0.02019019, 0.2709028, 0.58130202, 0.96772135, 0.40203406, 0.68584788, 0.39907061, 0.8020824, 0.59922299, 0.89880733, 0.52469613, 0.2320425, 0.95274654, 0.57433355, 0.60232225, 0.12846666, 0.62600488, 0.75885062, 0.09451081, 0.5949559], [0.30851973, 0.92397012, 0.45213047, 0.31350788, 0.47674255, 0.99656559, 0.31500455, 0.67267856, 0.96399447, 0.03283226, 0.36633299, 0.17880501, 0.1446183, 0.66984436, 0.46706976, 0.03951692, 0.22784513, 0.82142612, 0.07370921, 0.33906808], [0.73137758, 0.66755197, 0.03242705, 0.29475688, 0.15821178, 0.26910982, 0.91078273, 0.70083964, 0.33650157, 0.26706248, 0.82314416, 0.17939914, 0.77290582, 0.42405843, 0.76003435, 0.30173729, 0.4576326, 0.57653883, 0.30827412, 0.85723128], [0.39220308, 0.17726624, 0.99721065, 0.76477702, 0.3495421, 0.44473093, 0.71881405, 0.3056014, 0.68752376, 0.38752906, 0.12147931, 0.13051651, 0.37855929, 0.180905, 0.59225714, 0.80858205, 0.92062923, 0.03778071, 0.81331967, 0.53359333], [0.07207221, 0.17216235, 0.10151733, 0.77153031, 0.95657225, 0.47837661, 0.30229184, 0.93057551, 0.04504919, 0.82969811, 0.16200046, 0.34971251, 0.04055285, 0.57460145, 0.58058546, 0.40074296, 0.55627212, 0.50018151, 0.10975096, 0.41768908], [0.64015041, 0.12854868, 0.09152847, 0.81948493, 0.64439314, 0.1571958, 0.86136143, 0.96147962, 0.25692185, 0.91043344, 0.0031553, 0.50970494, 0.09239349, 0.32355557, 0.02523059, 0.78752319, 0.42300025, 0.83668593, 0.45169828, 0.23836782], [0.15067413, 0.17847288, 0.34651435, 0.88828655, 0.70997458, 0.3427081, 0.22473914, 0.47540793, 0.12765958, 0.39060752, 0.60724015, 0.72244086, 0.51583481, 0.49931979, 0.2823342, 0.80178407, 0.62729361, 0.28726125, 0.34917165, 0.60606349], [0.79312299, 0.37268688, 0.58758082, 0.55694459, 0.35795187, 0.08427892, 0.80678922, 0.32206049, 0.55948097, 0.06842917, 0.46172133, 0.16606839, 0.150101, 0.84987818, 0.2154061, 0.17590738, 0.16686814, 0.81569503, 0.1565211, 0.04867117], [0.0049085, 0.37890289, 0.89008346, 0.9281833, 0.33230683, 0.91271647, 0.02732307, 0.54477629, 0.01319123, 0.30752729, 0.79310271, 0.39230414, 0.49702199, 0.95427926, 0.48890644, 0.61628695, 0.50168302, 0.36486363, 0.01473803, 0.77408549], [0.903083, 0.67791114, 0.61581447, 0.05392819, 0.91796546, 0.92341286, 0.40200506, 0.92874617, 0.35434179, 0.03242402, 0.04678233, 0.88053641, 0.79735547, 0.9512864, 0.73243777, 0.24907558, 0.26310536, 0.80493796, 0.68572655, 0.45102886]]
y: [[1], [0], [1], [0], [1], [1], [1], [0], [1], [1]]

Expected Output:
<class 'numpy.ndarray'>
[[-0.07140561] [ 0.0649857 ] [-0.07455911] [-0.25456612] [ 0.02489909] [-0.05027607] [ 0.09235117] [-0.0528404 ] [ 0.43180331] [-0.6064806 ] [ 0.34821837] [ 0.11929702] [ 0.31914999] [ 0.43665925] [ 0.09250638] [ 0.00743292] [ 0.16578263] [-0.0770846 ] [ 0.16158202] [-0.03285493] [ 0.3250566 ]]

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Voluptatem natus esse asperiores deserunt blanditiis ipsum nemo laborum recusandae culpa, tempore ducimus hic ad perferendis ratione sed ab totam ullam voluptas. Omnis maiores deserunt ut quidem voluptate laboriosam molestias id tempora earum tenetur?

Fugiat nemo quia a consequatur alias libero architecto laborum doloribus, veniam dolores dolore voluptatum. Eum deserunt rerum aperiam adipisci optio illo quos?

Pariatur exercitationem veniam consequatur voluptatem nihil sunt facere eum soluta, tempore hic libero eos placeat ex assumenda, reprehenderit sunt adipisci in delectus atque cupiditate, quo consectetur animi rerum, corrupti molestias explicabo debitis facere cupiditate? Iste illum ab, debitis recusandae optio totam qui est laboriosam quisquam magni ipsam velit?

This is a premium feature.
To access this and other such features, click on upgrade below.

Ready.

Input Test Case

Please enter only one test case at a time
numpy has been already imported as np (import numpy as np)