2

Perceptron Training

Unsolved

Supervised

Difficulty: 7 | Problem written by ankita

Neural networks are composed of small units which are sometimes called neurons or perceptrons. These small units are supervised classifiers of their own.  

Here, our aim is to implement the perceptron training manually using NumPy only.

Given below is the perceptron criteria that is to be minimized to learn the weights:

Ep(w) =  \(\sum_{n\in M}^{}-w^{t}\Phi(x_{n})t_{n}\) 

M: misclassified points.

\(w\)Weights of the model

\(\Phi\)Feature function of x in our case it is identity function \(\Phi\)(x) = x

\(t_{n}\): True labels of training data labeled as +1 and -1 for the two classes

Input:

x: an array of training examples
y: an array of outputs corresponding to each training example (possible values are 0 and 1)
lr: the learning rate for the algorithm
iter: number of iterations the algorithm will perform

Output:
A list of updated weights after every iteration. Do not include the first W that is an array of zeros

Algorithm: 

  • Convert the given y from 0,1 to \(t_{n}\) which is +1 and -1 for the two classes
  • Calculate,  \(w^{T}x\) , then write y_p as an array of +1 for positive value at the corresponding location in the calculated vector and -1 for negative value
  • It can be seen for correct classification \(w^{T}\Phi(x_{n})t_{n}\)  is positive and negative for incorrect classification
  • For incorrectly classified points calculate derivative of Ep(w) wrt to w using the formula given below: Derivative of  Ep(w) wrt to w =  \(\sum_{n\in M}^{}-\Phi(x_{n})t_{n}\), where M is misclassified points
  • Update the weights using the formula: W = W - learning_rate*derivative of the perceptron criterion with respect to W
  • Continue the above step for the number of iterations specified in the function.

Hints:

The first element of W should be wo i.e., if Y = wX + wo then W = [wo, elements of w]
To get the above W, stack a column of ones to X at the beginning of X.

Sample Input:
X: [[0.26703489, 0.50235526], [0.18151398, 0.07526641], [0.97665297, 0.09124986]]
Y: [0, 1, 1]
iter: 5
lr: 0.1

Expected Output:
[array([[0.8 ], [0.8841833 ], [0.98334837]]), array([[0.6 ], [0.76836661], [0.96669675]]), array([[0.4 ], [0.65254991], [0.95004512]]), array([[0.2 ], [0.53673322], [0.93339349]]), array([[5.55111512e-17], [4.20916525e-01], [9.16741865e-01]])]

Input Test Case

Please enter only one test case at a time
numpy has been already imported as np (import numpy as np)

Output