2

Lasso Regression

Unsolved
Optimization
Supervised

Difficulty: 7 | Problem written by ankita
Regularization is a form of regression that discourages learning a more complex or flexible model, so as to avoid the risk of overfitting.

Lasso regression is implemented by adding an additional term in the loss function. Here, we use the linear least-squares function. The regularization is given by the L1-norm:

L(W,α)=\(||X.W-y||^{2}+\alpha||W||\)

We will apply batch gradient descent with the following derivative of W at each step.

WS = NumPy array corresponding to W. The ith element is: 1 if W[i] >= 0, otherwise -1

n is the number of training examples

W=\((-2*X^{T}.(Y-X.W)+\alpha*W_S)/n\)

We expect you to implement the whole algorithm manually and return the ‘W’ corresponding to the best-suited α and the loss for the corresponding parameters.

Input:

X: an array of training examples. 

y: an array of output corresponding to each training example

alpha: different values of α for which you have to get minimum loss 

iter: number of iterations

lr: learning rate

Output:

For each α calculate the 'W' and hence the loss according to the function above on the whole training data and return the α, loss score, and 'W' for the α with the minimum loss on the training data.

The output should be in the following order:

α, loss score, NumPy array of 'W'.

Hint:

The prediction is mentioned as X.W and the shape of X is (n,m) and shape of Y is (n,1). So, this automatically means W should be (m,1) which is a 2-D matrix.

Sample Input:
<class 'list'>
X: [[0.55345954, 0.26978505, 0.99572193, 0.17879061, 0.13353172], [0.12972127, 0.89863166, 0.97875147, 0.61299997, 0.88425275], [0.08780339, 0.90976317, 0.68283976, 0.02670151, 0.30560837], [0.82365932, 0.87099191, 0.52195797, 0.52162298, 0.40034739], [0.70355801, 0.89146552, 0.38555787, 0.07339327, 0.16111809], [0.27560237, 0.92967928, 0.6460444, 0.46355679, 0.69999201], [0.86036116, 0.66422329, 0.69960402, 0.7787864, 0.67299241], [0.86554358, 0.43671475, 0.0406369, 0.09743328, 0.13477061], [0.22106352, 0.57616507, 0.43354926, 0.63722607, 0.89919981], [0.30758308, 0.40788758, 0.0811379, 0.35161535, 0.37144102]]
Y: [[0.12954958], [0.88900561], [0.15619786], [0.19463617], [0.25362551], [0.81332185], [0.59385747], [0.63010439], [0.4483], [0.16408941]]
betas: [1, 0.1, 10, 20]
iter: 1000
lr: 0.001

Expected Output:
<class 'tuple'>
(0.1, 0.5849272455767055, array([[0.0933715 ], [0.22456382], [0.12905883], [0.12445942], [0.21719877]]))

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Ducimus aut corrupti minima voluptatibus, possimus necessitatibus unde debitis animi aliquid adipisci id similique autem magni, consequatur eum laboriosam inventore voluptates perferendis repellendus eos fuga amet porro vel, magnam quos tempore amet assumenda excepturi error illum laborum, ullam aperiam voluptatum corporis illo vero expedita ipsam doloremque voluptates. Nisi facere libero animi debitis labore unde tenetur, minus voluptatem quasi voluptate nobis doloribus nulla numquam quas.

Iure suscipit architecto tenetur, facere cum eligendi distinctio? Earum beatae eum vitae, dicta quas rerum repellat iste, cumque quos repellat eaque, libero aut obcaecati animi eaque quia architecto laboriosam nobis at.

Quia ipsa a, quam sed facere libero officiis voluptates debitis dignissimos ut facilis, quas nisi natus reprehenderit neque asperiores fuga porro quaerat corrupti quisquam voluptatem, minima quasi autem nesciunt itaque amet adipisci voluptas minus, ratione libero similique at dicta in facilis. Quo suscipit velit quisquam blanditiis quia libero veniam sint minima eius nisi, natus quisquam pariatur atque a vero illum nobis corrupti illo. Cumque suscipit aliquam sequi placeat quibusdam non eius in ipsum id et, dolorum esse eligendi voluptatem debitis dolor adipisci, cum esse laborum saepe quae excepturi quisquam odio error ullam quia, et cupiditate temporibus aut enim. Adipisci tempore ipsam sit accusantium aperiam nobis culpa veritatis repellat et reprehenderit, esse provident dignissimos ipsam, sequi quibusdam aperiam sit tempore voluptates accusantium ipsa nihil aliquam?

This is a premium feature.
To access this and other such features, click on upgrade below.

Ready.

Input Test Case

Please enter only one test case at a time
numpy has been already imported as np (import numpy as np)