2

Ridge Regression

Unsolved
Supervised

Difficulty: 4 | Problem written by ankita
Regularization is a form of regression that discourages learning a more complex or flexible model to avoid the risk of overfitting. Ridge regression is implemented by adding an additional term in the loss function. Here, the loss function is the linear least-squares function and the regularization is given by the L2-norm:

L(W,α)= \(||XW-y||^{2}+\alpha||W||^{2}\)

The partial derivative of the loss function is calculated with respect to ‘W’ and the function is set as equal to zero to get the desired ‘W’. The ‘W’ that we get after applying the math is:

W=\((X^{T}X+\alpha I)^{-1}X^{T}y\)

Where I is an identity matrix and α is defined by you.

We expect you to implement this algorithm manually and return the ‘W’ corresponding to the best-suited α and the loss for the corresponding parameters.

Input:

X: an array of training examples. 

y: an array of output corresponding to each training example

alpha: Different values of α for which you have to get minimum loss 

Output:

For each α, calculate the 'W' and hence the loss according to the function above on the whole training data and return the α, loss score and 'W' for the α with minimum loss on the training data.

Output should be in the following order:

α, loss score, numpy array of 'W'.

Sample Input:
<class 'list'>
X: [[0.60125408, 0.12577945, 0.01411056, 0.0936978, 0.89758782, 0.93564844, 0.8950285, 0.47727655, 0.71737809, 0.59242012], [0.13858243, 0.24482032, 0.86234713, 0.33979516, 0.0889971, 0.64927903, 0.57949765, 0.38865498, 0.02500174, 0.78624164], [0.46325961, 0.30377005, 0.15942722, 0.29213869, 0.59723249, 0.23040769, 0.38858347, 0.39792989, 0.22995737, 0.2646458], [0.26977591, 0.10918761, 0.21551369, 0.05295569, 0.84376132, 0.41889707, 0.12817272, 0.95816636, 0.95781923, 0.6110987], [0.73695696, 0.4750288, 0.82283346, 0.66565624, 0.37338998, 0.78591842, 0.0742858, 0.8773378, 0.48120651, 0.48274888], [0.09708786, 0.91395461, 0.76595178, 0.21224482, 0.15616675, 0.85005413, 0.70310046, 0.88231817, 0.74145004, 0.6024671], [0.75903593, 0.46548889, 0.31014249, 0.87254449, 0.27189493, 0.60013148, 0.43970967, 0.24510671, 0.42089798, 0.32563036], [0.62350623, 0.69931332, 0.96678008, 0.16620784, 0.21867862, 0.07789741, 0.9754635, 0.00814793, 0.87632045, 0.78894852], [0.93176233, 0.94406265, 0.27014565, 0.85860165, 0.53244571, 0.7849823, 0.47061634, 0.82533955, 0.97938773, 0.5625293], [0.30010037, 0.92693908, 0.23358359, 0.59492493, 0.14378321, 0.79335955, 0.54795122, 0.93509142, 0.49911903, 0.1223535], [0.12125732, 0.03845552, 0.53730161, 0.33617737, 0.00424432, 0.68671698, 0.84538236, 0.63718911, 0.99249623, 0.11882383], [0.34461216, 0.3353819, 0.96411776, 0.65207153, 0.24999972, 0.37552798, 0.47608931, 0.72491517, 0.80095516, 0.68978142], [0.18362188, 0.58009584, 0.31087527, 0.84663187, 0.49291746, 0.88991947, 0.81423266, 0.00850095, 0.13322052, 0.85356227], [0.04315464, 0.50934197, 0.01864857, 0.64816, 0.11961158, 0.89168548, 0.02568152, 0.47235023, 0.29562819, 0.7476345], [0.44860421, 0.14621354, 0.34438205, 0.68228521, 0.31818425, 0.91887658, 0.74396, 0.45496131, 0.03400659, 0.14556801], [0.87777354, 0.1715586, 0.12218999, 0.07434893, 0.24157273, 0.76799222, 0.38555271, 0.03912091, 0.37887198, 0.04369867], [0.70235542, 0.85722687, 0.61369535, 0.51259233, 0.91769692, 0.04260091, 0.92715519, 0.11154074, 0.86136391, 0.17674612], [0.64574304, 0.70516763, 0.01899496, 0.29888788, 0.20827288, 0.06270067, 0.28274384, 0.41546589, 0.30170273, 0.49650167], [0.06605403, 0.22370203, 0.72766904, 0.27268386, 0.71954316, 0.67180046, 0.018005, 0.62971642, 0.40960662, 0.2556009], [0.87758445, 0.9914289, 0.43224018, 0.87862905, 0.14434139, 0.94006928, 0.39777724, 0.00741952, 0.32963568, 0.53465108]]
Y: [[0.28536552], [0.34221503], [0.84590637], [0.26964398], [0.27359041], [0.9442081], [0.71474058], [0.18775526], [0.68799382], [0.17606394], [0.55740976], [0.9541375], [0.38676104], [0.48218884], [0.83872929], [0.75201683], [0.75870764], [0.60631138], [0.26194846], [0.73325989]]
alpha: [1, 5, 0.1]

Expected Output:
<class 'tuple'>
(0.1, 1.3435677397481878, array([[ 0.30920673], [ 0.00949388], [ 0.08465373], [ 0.26662866], [-0.00174554], [ 0.05644303], [ 0.26276322], [ 0.18520228], [-0.01781252], [-0.05082192]]))

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Error eum voluptatum magnam ea quis quaerat reprehenderit voluptatibus, rerum ab aliquam, aspernatur totam quasi itaque illo pariatur porro provident, dolorum delectus amet placeat modi nulla debitis sed, quidem consequuntur esse doloribus impedit libero repudiandae? Voluptatem corrupti ipsa nam assumenda, excepturi laborum esse nostrum fugit nulla dignissimos nihil quaerat aliquam iusto, doloribus est eaque nesciunt quae maxime delectus, dolores magnam repellat neque repudiandae alias sequi. Possimus facilis tenetur ipsa enim aliquid porro maxime ullam id voluptatibus, placeat in totam error dolorem quisquam. Amet necessitatibus quae, quisquam temporibus perferendis consequatur asperiores, facilis modi laudantium corrupti.

Nobis aspernatur soluta totam saepe ab quia repellendus a earum pariatur, inventore voluptate unde itaque architecto, atque aspernatur culpa eius illo nobis harum, natus impedit minima accusamus sint eaque quod iusto maiores nostrum ut. Quasi dolorum cumque rem necessitatibus tempora culpa consequatur vitae, omnis et voluptas quis dolorum nostrum praesentium beatae aut, illum tenetur fuga hic voluptas laudantium nisi facilis eaque?

Nisi rem laborum animi, molestias praesentium explicabo rem modi temporibus cum ducimus odio beatae, consectetur impedit saepe deleniti, harum cupiditate facere aperiam? Consectetur pariatur libero totam quod mollitia optio at, esse soluta deserunt nostrum qui neque, voluptatem magni veniam distinctio similique molestias accusamus non pariatur, est corrupti sint repellendus, nostrum in ratione. Consequatur molestiae ut velit, nobis qui vitae ducimus quod, assumenda temporibus ullam laudantium error minus cum, harum minima nobis necessitatibus, a doloribus voluptatem illum esse recusandae neque nihil? Veritatis eius illo rem mollitia ipsum perferendis fugit ea dicta nisi excepturi, ad id unde dolores quod totam expedita commodi tenetur assumenda iusto repellendus, pariatur est animi voluptatibus?

This is a premium feature.
To access this and other such features, click on upgrade below.

Ready.

Input Test Case

Please enter only one test case at a time
numpy has been already imported as np (import numpy as np)